Catching a python app before it exits

Question

I have a python app which is supposed to be very long-lived, but sometimes the process just disappears and I don't know why. Nothing gets logged when this happens, so I'm at a bit of a loss.

Is there some way in code I can hook in to an exit event, or some other way to get some of my code to run just before the process quits? I'd like to log the state of memory structures to better understand what's going on.

Answer 1

You might try running your application directly from a console (cmd on windows, sh/bash/etc on unix), so you can see any stack trace, etc printed to the console when the process dies.

Answer 2

atexit is pronounced "at exit". The first times I read that function name, I read it as "a texit", which doesn't make nearly as much sense.

Answer 3

I'm not sure if you are able to modify the source code, but if so you might want to try this:

def debugexcept(type, value, tb):
    if hasattr(sys, 'ps1') or not (sys.stderr.isatty() and sys.stdin.isatty()) or type == SyntaxError:
        sys.__excepthook__(type, value, tb)
    else:
        import traceback, pdb
        traceback.print_exception(type, value, tb)
        print
        pdb.pm()


sys.excepthook = debugexcept

If you launch your python program from a command line you should be dumped into the python debugger when it dies, assuming something 'bad' has happened to cause an exception. I'm guessing maybe stderr/stdout have been captured and you're not seeing some exception?

ie search for something like:

sys.stdout = open('stdout.log', 'w')
sys.stderr = open('stderr.log', 'w')

If the process is dieing without an exception at all then that might be harder to find. One (very hard way) on windows would be to use something like windbg to attach to the process and set a breakpoint in the CRT at some relevant spot.

Good luck!