views:

48

answers:

1

I have created a cron job to scan rss/atom feeds but want to know if there is a timeout on this.

The documentation says that requests are limited to 30 seconds; is a cron job a regular request that is subject to this restriction?

Should I break the job up into one scan per cron job just to be on the safe side, or is there a better way?

I was thinking of maybe having the cron job spawn tasks into the task queue that would do the resource fetching (so each task could fetch one resource, and hopefully none would overrun the 30 second limit).

Any advice would be appreciated.

+3  A: 

Yes, the 30-seconds deadline applies to cron jobs, too (just as to regular requests and task queue jobs). The best way to "break things up" is generally to enqueue "continuation" requests on a task queue.

Alex Martelli
Is there anything special about a "continuation" request is it just a regular task?
download
@download, a regular task, with whatever parameters you need to continue where the previous one left off. For a great (advanced) example, and in an RSS feeds context too;-), check out http://code.google.com/p/pubsubhubbub/ .
Alex Martelli