What is the best way to get a stacktrace when using multiprocessing?

Question

I'm wondering about the best way to get a stacktrace when there is an exception inside a function executed via the multiprocessing module. Here's an example:

import multiprocessing

def square(x):
    raise Exception("Crash.")
    return x**2

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=4)
    results = pool.map_async(square, range(5))
    for result in results.get():
        print result

This prints:

Traceback (most recent call last):
  File "/extra/workspace/Playground/src/multiproc/multiproc_debug.py", line 11, in <module>
    for result in results.get():
  File "/extra/Python 2.6/lib/python2.6/multiprocessing/pool.py", line 422, in get
    raise self._value
Exception: Crash.

So there is no useful stacktrace, which is quite annoying. My current solution for this:

import multiprocessing
import traceback

def square(x):
    try:
        # some more code...
        raise Exception("Crash.")
    except Exception, exception:
        print exception
        traceback.print_exc()
        raise
    return x**2

Is there a way to get this behaviour without all the boilerplate code? If not, what is the reason for not including this feature?

Edit: One could use a decorator for the boilerplate code, but I don't know if such a decorator is included in the standard library?

Answer 1

It looks like you should avoid raising the exception from your main function. Instead, you can catch it, treat it as a value returned to the main program, then raise it there. Re-throwing exceptions in Python has more details.