Reading data from a log file as a separate application is writing to it.

Question

I would like to monitor a log file that is being written to by an application. I want to process the file line by line as, or shortly after, it is written. I have not found a way of detecting that a file has been extended after reaching eof.

The code needs to work on Mac and PC, and can be in any language, though I am most familiar with C++ and Perl.

Does anybody have a suggestion for the best way to do it?

Answer 1

In Perl, the File::Tail module does exactly what you need.

Answer 2

A generic enough answer:

Most languages, on EOF, return that no data were read. You can re-try reading after an interval, and if the file has grown since, this time the operating system will return data.

Answer 3

You should be able to use read the standard io from tail -f

Answer 4

The essense of tail -f is the following loop:

open IN, $file;
while(1) {
  my $line = <IN>;
  if($line) {
    #process line...
  } else {
    sleep(1);
    seek(IN,0,1);
  }
}
close IN;

The seek call is to clear the EOF flag.

Answer 5

I'd have thought outputting the actions via tee, and thence tail'ing (or using the loop above) the file created by tee some use.