Java - RandomAccessFile (Emulating the Linux tail function)

Question

http://stackoverflow.com/questions/557844/java-io-implementation-of-unix-linux-tail-f has a similar problem; but the solution is not viable for log files that generate about 50-100 lines per second.

I have an algorithm that emulates the tail functionality in Linux. For example,

File _logFile = new File("/tmp/myFile.txt");
long _filePtr = _logFile.length();

while (true)
{
     long length = _logFile.length();
     if (length < _filePtr)
     {
            // means file was truncated
     }
     else if (length > _filePtr)
     {
            // means something was added to the file
     } 
// we ignore len = _filePtr ... nothing was written to file
}

My problem is when: "something was added to the file" (referring to the else if() statement).

else if (length > _filePtr)
{
     RandomAccessFile _raf = new RandomAccessFile(_logFile, "r");
     raf.seek(_filePtr);

     while ((curLine = raf.readLine()) != null)
           myTextPane.append(curLine);

        _filePtr = raf.getFilePointer();
        raf.close();
}

The program blocks at while ((curLine = raf.readLine()).... after 15 seconds of run-time! (Note: that the program runs right for the first 15 seconds).

It appears that raf.readLine() is never hitting NULL, because I believe this log file is being written so fast that we go into an "endless cat and mouse" loop.

What's the best way to emulate Linux's tail?

Answer 1

I would think that you would be best served by grabbing a block of bytes based on the file's length, then release the file and parse a ByteArrayInputStream (instead of trying to read directly from the file).

So use RandomAccessFile#read(byte[]), and size the buffer using the returned file length. You won't always show the exact end of the file, but that is to be expected with this sort of polling algorithm.

As an aside, this algorithm is horrible - you are running IO operations in a crazy tight loop - the calls to File#length() will block, but not very much. Expect this routine to take your app to it's knees CPU-wise. I don't necessarily have a better solution for you (well - actually, I do - have the source application write to a stream instead of a file - but I recognize that isn't always feasible).

In addition to the above, you may want to introduce a polling delay (sleep the thread by 100ms each loop - it looks to me like you are displaying to a GUI - a 100ms delay won't hurt anyone, and will greatly improve the performance of the swing operations).

ok - final beef: You are adjusting a Swing component from what (I hope) is code not running on the EDT. Use SwingWorker#invokeLater() to update your text pane.

Answer 2

It appears I have found the problem and created a solution.

Under the else if statement:

while ((curLine = raf.readLine()) != null)
    myTextPane.append(curLine);

This was the problem. the append(String) method of myTextPane (which is a derived class of JTextPane) evoked "setCaretPosition()" on every line append which IS BAD!!

That meant that setCaretPosition() was called 50-100 Hz trying to "scroll down." This caused a blocking overhead to the interface.

A simple solution was to create a StringBuffer class and append "curLine" until raf.readLine() read null.

Then, append the StringBuffer and voila ... no more blocking from setCaretPosition()!

Thanks to Kevin for bringing me towards the correct direction.

Answer 3

You could always exec the tail program:

    BufferedReader in = new BufferedReader(new InputStreamReader(
            Runtime.getRuntime().exec("tail -F /tmp/myFile.txt").getInputStream()));
    String line;
    while ((line = in.readLine()) != null) {
        // process line
    }