cProfile and Python: Finding the specific line number that code spends most time on

Question

Hi,

I'm using cProfile, pstats and Gprof2dot to profile a rather long python script.

The results tell me that the most time is spent calling a method in an object I've defined. However, what I would really like is to know exactly what line number within that function is eating up the time.

Any idea's how to get this additional information?

(By the way, I'm using Python 2.6 on OSX snow leopard if that helps...)

Answer 1

Suppose the amount of time being "eaten up" is some number, like 40%. Then if you just interrupt the program or pause it at a random time, the probability is 40% that you will see it, precisely exposed on the call stack. Do this 10 times, and on 4 samples, +/-, you will see it.

This tells why it works. This is an example.

Answer 2

cProfile does not track line numbers within a function; it only tracks the line number of where the function was defined.

cProfile attempts to duplicate the behavior of profile (which is pure Python). profile uses pstats to store the data from running, and pstats only stores line numbers for function definitions, not for individual Python statements.

If you need to figure out with finer granularity what is eating all your time, then you need to refactor your big function into several, smaller functions.

Answer 3

There is a line profiler in python written by Robert Kern.