Profiling: Different types of profiling

Key Takeaways

• Knowing how different types of profilers work.
• Understanding when to use a particular profiler type.

Definition

Profiling is analyzing the performance of your application. It measures the frequency and duration of your function calls.

*For the rest of the article, by profiling I mean CPU profiling.

The below code would find first ‘n’ prime numbers that ends with digit 7. I am going to use this sample code to demonstrate different profiler types.

def is_prime?(number)
  if number <= 1
    return false
  end
  (2..(number - 1)).each do |n|
    return false if number % n == 0
  end
  true
end

def is_number_ends_with_7?(number)
  last_digit = number % 10
  last_digit == 7
end

def get_n_primes_with_7_as_ending_digit(n)
  primes_with_7_as_ending_digit = []
  candidate = 0
  while primes_with_7_as_ending_digit.size < n
    if is_prime?(candidate) && is_number_ends_with_7?(candidate)
      primes_with_7_as_ending_digit << candidate
    end
    candidate += 1
  end
  primes_with_7_as_ending_digit
end

get_n_primes_with_7_as_ending_digit(1000)

Why do we need profiling?

We need profiling to

• Evaluate overall application performance
• Identify the bottlenecks in your application.
• Determine performance issues in a particular method

Different Types of Profiling

There are two types of profilers

1. Tracing
2. Sampling

Tracing(Instrumentation)

Tracing profilers insert hooks at the beginning and end of each method call to record when the execution of the method has started and when it has ended. They also record the time taken by the methods that are called inside a method. With this information, the profiler measures the actual time taken by the entire method, the number of times a method has been invoked and also break it down into time spent locally and time spent on each call to another method.

                                

Here's what the headers here mean:                                

• %self is the percentage of the total time spent in this method call.
• total is the total time (in seconds) spent in this method and its children.
• self is the time spent in this method, excluding child method calls.
• wait is the time spent waiting here.
• child is the time spent in child method calls.
• calls is the number of times this was invoked.
• name  is method/function name.

Pros

• Results are accurate as it captures every method.
• Accurate method invocation count.

Cons

• For short functions that are called multiple times, profiling time would be more than the actual execution time.
• Because of the overhead added by this profiler on application performance, it is not recommended to be used in production environment.

When to use Tracing Profiler?

As these profilers add a lot of overhead which affects the execution time of your application, it is suggested to be used in the development environment.

Sampling

Sampling profilers collect snapshots of the call stack at regular intervals and determine the percentage of time taken by a method.

                                        

Here's what the headers here mean:                                

• TOTAL is the number of samples in which this method and its children are present.
• (pct) is the percentage of total samples for this method and its children
• SAMPLES is the number of samples in which this method is present, excluding child method samples.
• (pct) is the percentage of total samples for this method excluding child methods.
• FRAME is the method/function name.

Pros

• As the profiler won’t be adding any hooks to the method calls, the overhead added is negligible, and the application runs practically at its real speed.

Cons

• The metrics given are approximations since they don’t track method by method. There is a possibility of not capturing fast running methods as their execution might be completed in between sampling interrupts.(eg: This profiler was unable to capture `is_number_ends_with_7?` method.)
• They cannot give you the method invocation count.

When to use Sampling Profiler?

As these profilers add a negligible amount of overhead to your application performance, they are a good choice for profiling in the production environment.

Comparing Execution times

The Pareto Principle

80% of the application performance is affected by 20% of its code. Instead of doing premature optimization, we need to measure where the bottlenecks and problem areas are in our application. We should not be optimizing our application until our metrics say so.        

Conclusion

In general, use sampling profilers for production, and use tracing profilers for development environment.

---