I have a cron job which parses a # of RSS feeds. It turns out it takes too long so I'm getting a HardDeadlineExceededError. I tried just creating two cron jobs, one which parses the even feeds, one which parses the odd feeds, I still get the error.
What's the best way to break up this work so it fits within a 30 second window?
Increase the number of jobs to decrease the number of feeds per job. Not just two segments (odd and even), but more (feed number MOD x) until it fits into the window.
If this does not work (too many cron jobs), take a look at Task Queues. This way the cron job could push out the work into the queue, maybe one task per feed.
I recommend using the Task Queue -- a task can perform any number N of sub-tasks of your choice, making sure to get close but not too close to the hard-deadline, then it spawns another task to continue where the spawner left off.
What I ended up doing was creating a simple datastore object that tracks what RSS feed I parsed last. I parse one RSS feed and increment the variable every time the cron job runs.