multiprocessing Pool hangs when there is a exception in any of the thread

Question

I am new to Python and trying a multiprocessing.pool program to process files, it works fine as long as there are no exceptions. If any of the thread/process gets an exception the whole program waits for the thread

snippet of the code:

cp = ConfigParser.ConfigParser()
cp.read(gdbini)
for table in cp.sections():
    jobs.append(table)
#print jobs
poolreturn = pool.map(worker, jobs)
pool.close()
pool.join()

Failure Message:

---

Traceback (most recent call last):
  File "/opt/cnet-python/default-2.6/lib/python2.6/threading.py", line 525, in __bootstrap_inner
    self.run()
  File "/opt/cnet-python/default-2.6/lib/python2.6/threading.py", line 477, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/opt/cnet-python/default-2.6/lib/python2.6/multiprocessing/pool.py", line 259, in _handle_results
    task = get()
TypeError: ('__init__() takes exactly 3 arguments (2 given)', <class 'ConfigParser.NoOptionError'>, ("No option 'inputfilename' in section: 'section-1'",))

---

I went ahead added a exception handler to terminate the process

try:
    ifile=cp.get(table,'inputfilename')
except ConfigParser.NoSectionError,ConfigParser.NoOptionError:
    usage("One of Parameter not found for"+ table)
    terminate()

but still it waits, not sure whats missing.

Answer 1

Shouldn't you be using a finally block to do some cleanup?

Answer 2

never min , I could figure it out. I had to return some thing back to the thread so that it can return back to the pool thx- mk