Google App Engine for pseudo-cronjobs?

Question

I look for a possibility to create pseudo-cronjobs as I cannot use the real jobs on UNIX.

Since Python scripts can run for an unlimited period, I thought Python would be a great solution.

On Google App Engine you can set up Python scripts and it's free. So I should use the App Engine.

The App Engine allows 160,000 external URL accesses (right?) so you should have 160000/31/24/60 = 3,6 accesses per minute.

So my script would be:

import time
import urllib
while time.clock() < 86400:
    # execute pseudo-cronjob file and then wait 60 seconds
    content = urllib.urlopen('http://www.example.org/cronjob_file.php').read()
    time.sleep(60)

Unfortunately, I have no possibility to test the script, so my questions are: 1) Do you think this would work? 2) Is it allowed (Google TOS) to use the service for such an activity? 3) Is my calculation for the URL accesses per minute right?

Thanks in advance!

Answer 1

Duplicate, see http://stackoverflow.com/questions/145651/cron-jobs-on-google-appengine

Cron jobs are now officaly supported on GAE: http://code.google.com/appengine/docs/python/config/cron.html

Answer 2

Maybe I'm misunderstanding you, but the cron config files will let you do this (without Python). You can add something like this to you cron.yaml file:

cron:
- description: job that runs every minute
  url: /cronjobs/job1
  schedule: every minute

See Google's documentation for more info on scheduling.

Answer 3

Google has some limits on how long a task can run.

URLFetch calls made in the SDK now have a 5 second timeout, here

They allow you to schedule up to 20 cron tasks in any given day. Here

Answer 4

You may want to clarify which way around you want to do it

Do you want to use appengine to RUN the job? Ie, the job runs on google's server?

or

Do you want to use your OWN code on your server, and trigger it by using google app engine?

If it's the former: google does cron now. Use that :)

If it's the latter: you could use google's cron to trigger your own, even if it's indirectly (ie, google-cron calls google-app-engine which calls your-app).

If you can, spin up a thread to do the job, so your page returns immediatly. Dont forgot: if you call http://whatever/mypage.php, and your browser dies (or in this case, google kills your process for running too long), the php script usually still runs to the end - the output just goes no where.

Failing that, try to spin up a thread (not sure if you can do that in PHP tho - I'm a C# guy new to PHP)

And if all else fails: get a better webhost! I pay $6/month or so for dreamhost.com, and I can run cron jobs on their servers - it's included. They do PHP, Rails et al. You could even ping me for a discount code :) (view profile for website etc)