Succesive calls to cProfile/pstats no updating properly

Question

Hi, I'm trying to make successive calls of some profiler code however on the second call to the function the update time of the profile file changes but the actual profiler stats stay the same. This isn't the code I'm running but it's as simplified an example I can come up with that shows the same behaviour.

On running, the first time ctrl+c is pressed it shows stats, second time same thing but rather than being fully updated as expected only the time is, and third time program actually quits. If trying, ideally wait at a few seconds between ctrl+c presses.

Adding profiler.enable() after the 8th lines does give full updates between calls however it adds a lot of extra profiler data for things I don't want to be profiling.

Any suggestions for a happy medium where I get full updates but without the extra fluff?

import signal, sys, time, cProfile, pstats

call = 0

def sigint_handler(signal, frame):
global call
if call < 2:
 profiler.dump_stats("profile.prof")
 stats = pstats.Stats("profile.prof")
 stats.strip_dirs().sort_stats('cumulative').print_stats()
 call += 1
else:
 sys.exit()
def wait():
time.sleep(1)

def main_io_loop():
signal.signal(signal.SIGINT, sigint_handler)
while 1:
 wait()

profiler = cProfile.Profile()
profiler.runctx("main_io_loop()", globals(), locals())

Answer 1

Calling profiler.dump_stats (implemented in cProfile.py) calls profiler.create_stats, which in turns calls profiler.disable().

You need to call profiler.enable() to make it work again. No, this is not documented.

The following seems to do what you want. Note that I got rid of the intermediate data file since pstats.Stats knows how to get the data from the profiler directly.

import signal, sys, time, pstats, cProfile

call = 0

def sigint_handler(signal, frame):
  global call
  if call < 2:
    stats = pstats.Stats(profiler)
    stats.strip_dirs().sort_stats('cumulative').print_stats()
    profiler.enable()
    call += 1
  else:
    sys.exit()

def wait():
  time.sleep(1)

def main_io_loop():
  signal.signal(signal.SIGINT, sigint_handler)
  while 1:
    wait()

profiler = cProfile.Profile()
profiler.runctx("main_io_loop()", globals(), locals())