I have some tasks in an application that are CPU bound and I want to use the multiprocessing module to use the multi-cores processors. I take a big task (a video file analysis) and I split it into several smaller tasks which are put in a queue and done by worker processes. What I want to know is how to report progress to the main process from these worker processes. For example I need them to send "I am at 1000ms of my analysis of file 1". What is the best way to do such progress reports ?
+6
A:
I would recommend a multiprocessing.Queue: nothing easier than for the worker processes to post their updates (presumably as tuples with the various aspect of their progress updates) there, while the main process just wait for such messages and when they come updates the GUI (or textual UI;-) to keep the user appraised of progress.
Alex Martelli
2010-09-21 00:49:01
Thanks. I think that it is the easiest solution. I'm currently implementing it and I'll ask again if something goes wrong. Sorry but I can't vote up your post because I don't have enough reputation.
Marc Demierre
2010-09-21 03:01:49
@Marc, I believe that with 11 or more (and I see you're at 18) you _can_ upvote. If it doesn't work for you you might want to ask on meta!
Alex Martelli
2010-09-21 15:05:51
I hadn't enough yesterday. Now I upvoted your post :)
Marc Demierre
2010-09-22 00:26:20