long time cron job via wget/curl ?

Question

Hi All

I am working on cron jobs for my php app and planning to use cron via wget/curl. Some of my php cron jobs can take 2-3 hours. How to let php work 2-3 hours from cron tab ? Is it good practice to run such long time jobs via cron wget/curl ? Anyone got experience on this ? I also have email queue and i needs to be run every 10 seconds, but cron tab is minute level. Any suggest on this case ?

Thanks for reading.

Answer 1

When you use wget/curl, you are requesting for a page from a webserver. Every webserver will have a time out period, so this might time out.

Also some of the hosting providers may stop the process running beyond certain minutes ( basically done to control the rogue threads).

So it is not advisable to schedule using wget/curl if the job takes more than few minutes.

Try scheduling it using actual scheduler. You can run php from command line

php [options] [-f] [--] [args...]

php command should be on the path.

Answer 2

You can use the following at the start of your script to tell PHP to effectively never time out:

set_time_limit(0);

What may be wiser is, if crontab is going to run the script every 24 hours, set the timeout to 24 hours so that you don't get two copies running at the same time.

set_time_limit(24*60*60);

Crontab only allows minute-level execution because, that's the most often you should really be launching a script - there are startup/shutdown costs that make more rapid scheduling inefficient.

If your application requires a queue to be checked every 10 seconds a better strategy might be to have a single long-running script that does that checking and uses sleep() occasionally to stop itself from hogging system resources.

On a UNIX system such a script should really run as a daemon - have a look at the PEAR System_Daemon package to see how that can be accomplished simply.