yet another confusion with multiprocessing error, 'module' object has no attribute 'f'

Question

I know this has been answered before, but it seems that executing the script directly "python filename.py" does not work. I have Python 2.6.2 on SuSE Linux.

Code:

#!/usr/bin/python
# -*- coding: utf-8 -*-
from multiprocessing import Pool
p = Pool(1)
def f(x):
    return x*x
p.map(f, [1, 2, 3])

Command line:

> python example.py
Process PoolWorker-1:
Traceback (most recent call last):
File "/usr/lib/python2.6/multiprocessing/process.py", line 231, in _bootstrap
    self.run()
File "/usr/lib/python2.6/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.6/multiprocessing/pool.py", line 57, in worker
    task = get()
File "/usr/lib/python2.6/multiprocessing/queues.py", line 339, in get
    return recv()
AttributeError: 'module' object has no attribute 'f'

Answer 1

This one works:

#!/usr/bin/python
# -*- coding: utf-8 -*-
from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == "__main__":
    p = Pool(1)
    p.map(f, [1, 2, 3])

I'm not 100% sure why your code does not work, but I guess the reason is that child processes launched by the multiprocessing module try to import the main module (to have access to the methods you defined), and the if __name__ == "__main__" stanza is required not to execute the initialization code where you set up your pool.

Answer 2

Restructure your code so that the f() function is defined before you create instance of Pool. Otherwise the worker cannot see your function.

#!/usr/bin/python
# -*- coding: utf-8 -*-

from multiprocessing import Pool

def f(x):
    return x*x

p = Pool(1)
p.map(f, [1, 2, 3])