Using cProfile results with KCacheGrind

Question

I'm using cProfile to profile my Python program. Based upon this talk I was under the impression that KCacheGrind could parse and display the output from cProfile.

However, when I go to import the file, KCacheGrind just displays an 'Unknown File Format' error in the status bar and sits there displaying nothing.

Is there something special I need to do before my profiling stats are compatible with KCacheGrind?

...
if profile:
    import cProfile

    profileFileName = 'Profiles/pythonray_' + time.strftime('%Y%m%d_%H%M%S') + '.profile'

    profile = cProfile.Profile()
    profile.run('pilImage = camera.render(scene, samplePattern)')

    profile.dump_stats(profileFileName)
    profile.print_stats()
else:            
    pilImage = camera.render(scene, samplePattern)
...

Package Versions

• KCacheGrind 4.3.1
• Python 2.6.2

Answer 1

It can be done using an external module called lscallproftree

This article explains how: CherryPy - CacheGrind

With my resulting code looking like so:

...
if profile:
    import cProfile
    import lsprofcalltree

    profileFileName = 'Profiles/pythonray_' + time.strftime('%Y%m%d_%H%M%S') + '.profile'

    profile = cProfile.Profile()
    profile.run('pilImage = camera.render(scene, samplePattern)')

    kProfile = lsprofcalltree.KCacheGrind(profile)

    kFile = open (profileFileName, 'w+')
    kProfile.output(kFile)
    kFile.close()

    profile.print_stats()    
else:            
    pilImage = camera.render(scene, samplePattern)
...

If anyone knows a way to do this that doesn't require an external (ie. not shipped with Python) module, I'd still be very interested to hear about it.

Answer 2

If what you're actually trying to do is see what parts of your code could be optimized for speed, and you can randomly pause it in the debugger, this method works. It may be surprising, but you don't need very many stackshots.

Answer 3

You could use profilestats.profile decorator ($ pip install profilestats) -- a simple wrapper for pyprof2calltree module (rebranding of lsprofcalltree.py):

from profilestats import profile

@profile
def func():
    # do something here

Script can be run as usual. profilestats creates two files: cachegrind.out.profilestats and profilestats.prof in KCachegrind-compatible and cProfile formats correspondingly.

Answer 4

With cProfile you can also profile existing programs, without making any separate profiling script. Just run program with profiler

python -m cProfile -o profile_data.pyprof script_to_profile.py

and open profile data in kcachegrind with pyprof2calltree, whose -k switch automatically opens data in kcachegrind

pyprof2calltree -i profile_data.pyprof -k

For example profiling whole paster server and webapp would be done like this

python -m cProfile -o pyprof.out `which paster` serve development.ini

pyprof2calltree can be installed with easy_install.