Safe Concurrent Write to a File Shared Between a Perl and a PHP script

Question

A Perl script (which I do not control) appends lines to the end of a text file periodically.

I need my PHP script (which will run as a cron job) to read the lines from that file, process them, and then remove them from the file. But, it seems like the only way to remove a line from a file with PHP is to read the file into a variable, remove the one line, truncate the file, and then rewrite the file.

But what happens if:

1. PHP reads the file
2. The Perl Script appends a new line.
3. The PHP script writes the modified buffer back over the file.

In that case the new line would be lost because it would be overwritten when the PHP script finishes and updates the file.

Is there a way to lock a file using PHP in a way that Perl will respect? It looks like the flock() function is PHP specific.

Answer 1

If the Perl script, which you cannot control, already implements file locking via flock, you are fine. If it doesn't (and I'm afraid that we have to assume that), you are out of luck.

Answer 2

Do you have any freedom to change the design? Is removing the processed lines from the file an essential part of your processing?

If you have that freedom how about letting the perl-produced file grow. Presumably the authors of the perl script have some kind of housekeeping in mind already? Maintaining your own "log" of what you have processed. Then when your script starts up it reads the perl file upto the point recorded in your "log". Process a record, update the log.

Answer 3

Maybe you instead of working on the same file could let your php script work on a copy? I imagine it could work with three files:

1. File written to by perl script
2. A copy of file 1
3. A processed version of file 2

Then when your php script starts, it checks if file 1 is newer than file 2, and if so makes a new copy, processes this (possibly skipping the number of lines already processed previously) and writes this to file 3.

Answer 4

Another possibility would be to instead of having the perl script write to a file, let it write to a named pipe and have your php script read out directly on the other end and let it write to a real file.