I need to import some data to show it for user but page execution time exceeds 30 second limit. So I decided to split my big code into several tasks and try Task Queues. I add about 10-20 tasks to queue and app engine executes tasks in parallel while user is waiting for data. How can I determine that my tasks are completed to show user data ASAP? Can I somehow iterate over active tasks?
+2
A:
I've solved this in the past by keeping the status for the tasks in memcached, and polling (via Ajax) to determine when the tasks are finished.
If you go this way, it's best if you can always "manually" determine the status of the tasks without looking in memcached, since there's always the (slim) chance that memcache will go down or will get cleared or something as a task is running.
Will McCutchen
2010-09-29 15:22:47
But how can I determine the status of the tasks?
Vladimir
2010-09-30 07:48:49
Currently I solved this creating table in datastore where I am tracking status of the tasks. Is there an easier way?
Vladimir
2010-09-30 09:10:46
The easier way might be to keep the status in memcache, as I suggested. The downside to this, if you can't determine the status of your tasks outside of the tasks themselves, is that memcache might occasionally fail/go down for maintenance/get flushed/etc., at which point you'd lose status information on any currently-executing tasks. This has not been a problem for me in practice.
Will McCutchen
2010-09-30 17:09:12