WX.Python and multiprocessing

Question

I have a wx.python application that takes some files and processes them when a button is clicked. I need to process them in parallel.

I use this code inside the bound button function:

  my_pool = multiprocessing.Pool(POOLSIZE)

  results=[digest_pool.apply_async(self.fun, [args]) for file in list_files() ]

  my_pool.close()

  my_pool.join()

    for result in results :

       print result.get()

But it seems this code is not run at all, even if I print something on fun. I didn't get any result and my GUI application got stuck. Could someone help? What is the problem here and how can I solve it using the pool multiprocessing module inside my wx frame class?

Answer 1

It looks like you're running up against a pretty common problem encountered by people attempting to use threading with GUI toolkits. The core of the issue is that you must never block the main GUI thread in your code. The graphical toolkit needs to be able to constantly respond to events. When you do the my_pool.join() call, you're putting the main thread to sleep and the result is that your entire process will appear to lock up.

I'm not particularly familiar with wxWidgets but I'm sure there are a few patterns out there for how to use threads with it. It's easy to spin off background threads from the GUI thread but getting the results back is usually the trick. You'll need some sort of asynchronous "work done" event that you can send to the main GUI thread when the background operation completes. Exactly how that's done differs from toolkit to toolkit. I'm not sure what the mechanism is for wxWidgets but I'd expect a quick google search would turn up an answer (or perhaps a kind commenter will provide a link ;-)