Script that will quit out every 5 minutes then renew itself where it left off

Question

Hey all,

I've got a UNIX question... I have a PHP script that I run in the terminal that parses a bunch of images and puts them in the database. This script has already been written

Due to restrictions on my hosting (Bluehost) I cannot run a process for more than 5ish minutes. I'm wondering how to set it up so that I will quit out of the script every 5 minutes then maybe sleep for a bit, then reopen itself where it left off.

I was planning on using a cronjob to rerun the command but how do I resume where I left off?

Thanks, Matt Mueller

Answer 1

Change your hosting provider....

Write a script to dump all the image file names into a text file. Set script to read the text file and remove the file name from the text file every time you finish processing that image. Any time the process is killed it should be safe to re-start.

Answer 2

You can have a table in the database or an external file that tells you where you left off.

Answer 3

Some ways I can see to do it (that spring immediately to mind - there may well be others):

1. As you add the images to your database, move them to a different location on the disk. Then your script won't see them the next time it runs.

2. As you add the images to your database, rename them from xx.jpg to xx.jpg-processed (for example). Then have your script ignore the processed ones.

3. Maintain a list of files that you've already processed and don't reprocess them.

4. Store the file name of the actual source file in the database along with the image, and query the database to prevent re-import.

Answer 4

Try CryoPID.

CryoPID allows you to capture the state of a running process in Linux and save it to a file. This file can then be used to resume the process later on, either after a reboot or even on another machine.