views:

42

answers:

1

I have python2.5 and multiprocessoring (get from http://code.google.com/p/python-multiprocessing/)

This simple code (get from docs), works very strange from time to time, sometimes it ok, but sometimes it throw timeout ex or hang my Windows (Vista), only reset helps :) Why this can happen?

from multiprocessing import Pool

def f(x):
    print "fc",x
    return x*x
pool = Pool(processes=4)  
if __name__ == '__main__':
    result = pool.apply_async(f, (10,))     # evaluate "f(10)" asynchronously
    print result.get(timeout=3)           # prints "100" unless your computer is *very* slow
+1  A: 

This is just a wild guess, but have you tried to move the Pool creation into the if block? I suspect that otherwise it might spawn an unlimited number of new processes, causing the freeze.

nikow
i was afraid to start this again :) but it works! thanx) really pull was creating over and over and this hangs.
Evg
@nikow: Why would processes be spawned indefinitely? I can see the 4 initial children each creating a new pool of 4 children upon importing the program, but I don't see how they would run the main code and call `f` for each of their children…
EOL
@EOL: I suspect that in every child the module code is executed. Every time a new Pool is created, spawning 4 new children. I had this happen on my machine, and saw a very large number of Python processes before the machine froze.
nikow
@nikow: yes, but why would the 4 new children (for each new pool) create a "very large number of processes"? I would expect the 4 normal children, plus 4*4 new processes, not more. I don't see this as a reason to freeze… I'm still puzzled. :)
EOL
@EOL: It's recursive. Each child accidentally creates a new pool with 4 new children. Note that the process in the first original children is no different from the one in their children. That's how `multiprocessing` is implemented.
nikow
@nikow: I see. This is a Windows "detail", as only 1 `Pool` is created with Python 2.7 on OS X…
EOL
@EOL: Yes, I think the reason is that Windows does not support forking processes.
nikow