How can I split a PHP script that takes a long time to run into smaller chunks?

Question

I have a PHP script that initialises an image gallery. It loops through all images, checks if they are thumbnailed and watermarks them.

My shared hosting account only lets me have 30 seconds of execution per script, as set in the php.ini settings. I can't change that.

What can I do to get round this? Currently I refresh the page after every 5 images, this prevents the script timing out, but the browser recognises that the script won't complete and gives an error. That's ok, but it's not really user friendly.

Any suggestions?

Answer 1

three ideas:

1. you could offload this to a script that runs on the server (PHP CLI perhaps?) and have your current script check the progress (still running! check back later!).

2. if javascript is ok in your environment, perhaps you could make ajax calls to handle this as well. the first solution allows the user to keep wandering around the site though, not sure if that's an important consideration in your case.

3. you could get the PHP script to just do a certain batch (say 10 images at a time), then refresh the page with a progress report and go for 10 more, instead of an arbitrary time based refresh.

Answer 2

get a list of images and then loop through them for example you get a total of 500 imgs you can take 5 at a time and after you're finished with them output a meta refresh tag that will reload the script in 2 seconds for example with the image number you're currently at as a parameter .

<meta http-equiv="refresh" content="2;url=script.php?start=5">

and in your script also check what $_GET['start'] is and continue from there with next 5 images then output start=10 and so one... until you loop through all of them.

Answer 3

What about some kind of task list, for example:

1. First script makes a list of images that need thumbnails or watermarking
2. Second script checks this list and does the first 5 or so (however many can be done in the time limit) and removes them from the list
3. Have the script in step 2 return a value that tells you if the list is empty, if not then repeat step 2 (probably easiest with ajax)

Using this method, the only thing 'running' for the whole time is the page in your browser, each script only runs for 30 seconds.

Answer 4

You could use set_time_limit() although some hosts may not support this, but most do now.

Answer 5

I think that it's a sign of something else going awry if each page load is taking over 30 seconds. How many images do you have? Are you storing the watermarked/thumbnailed versions on disk? If you're not doing that, then there's your answer: sure, it might take up a bit of space, but it'll avoid this problem altogether, as well as allow clients to cache the images.

If you are sure that the way you're doing it is the only way, then look at the set_time_limit function to increase script running time.

Answer 6

Perhaps you should avoid batching and make a proxy-generation script that does the process for each image on-demand.

Rewrite the conversion code in a proxy PHP script which checks if the image is already thumbnailed and if not, does what needs to be done. The thumbnailed or watermarked file can be then be loaded and sent to the client using fpassthru().

Your webpage should only refer to the proxy-script, like

<img src="/image.php?DSC001.JPG">

If your provider supports Apache/rewriting, you can do even more magic by bypassing the proxy/generation script if the thumbnail has already been created.

Answer 7

You might also want to try optimizing your script. Are you doing too much in your loop? Could you move something out of the loop? What are you using to get the list of files? Is there a more light-weight way to do that? These are the sort of questions you can ask yourself about your script so that instead of splitting it up you might get the whole shebang to run in time.

Failing that, I agree with Owen's suggestion of using AJAX. When the user kicks the process off you'll output a page as normal and then send back requests to process files out of band and then write status updates to the page as they come in.

Answer 8

On a shared hosting env, I would not recommend you run a script that might take more than 30 seconds to process and do n't even think about set_time_limit. What I would recommend is that you either do your thumbnails and watermarking on picture upload ? or picture request and mark the picture as done so you don't have to do it again. The other option is to run a batch that will process for instance 10 pictures every minute or 10 minutes but run the script via cli in the crontab....

It's a good practice to always know how much resources your script might be using when running a batch script....especially in a shared hosting env....this can affect other users easily.