Does functools.partial not work with multiprocessing.Pool.map?

Question

I have code that, simplified down, looks like this:

run = functools.partial(run, grep=options.grep, print_only=options.print_only, force=options.force)

if not options.single and not options.print_only and options.n > 0:
    pool = multiprocessing.Pool(options.n)
    Map = pool.map
else: Map = map

for f in args:
    with open(f) as fh: Map(run, fh)

try:
    pool.close()
    pool.join()
except NameError: pass

That works fine when I run it in single process mode, but fails with errors like this

TypeError: type 'partial' takes at least one argument

mixed up together with long call stacks through the multiprocessing module. What's going on?

I'm using python 2.6.1.

Answer 1

Google tells me that this is a bug in Python; apparently fixed in Py3k. It's supposedly due to partial not being picklable.

There is a workaround.