How to tail -f the latest log file with a given pattern

Question

Hello,

I work with some log system which creates a log file every hour, like follows:

SoftwareLog.2010-08-01-08
SoftwareLog.2010-08-01-09
SoftwareLog.2010-08-01-10

I'm trying to tail to follow the latest log file giving a pattern (e.g. SoftwareLog*) and I realize there's:

tail -F (tail --follow=name --retry)

but that only follow one specific name - and these have different names by date and hour. I tried something like:

tail --follow=name --retry SoftwareLog*(.om[1])  

but the wildcard statement is resoved before it gets passed to tail and doesn't re-execute everytime tail retries.

Any suggestions?

Answer 1

I haven't tested this, but an approach that may work would be to run a background process that creates and updates a symlink to the latest log file and then you would tail -f (or tail -F) the symlink.

Answer 2

[Edit: after a quick googling for a tool]

You might want to try out multitail - http://www.vanheusden.com/multitail/

If you want to stick with Dennis Williamson's answer (and I've +1'ed him accordingly) here are the blanks filled in for you.

In your shell, run the following script (or it's zsh equivalent, I whipped this up in bash before I saw the zsh tag):

#!/bin/bash

TARGET_DIR="some/logfiles/"
SYMLINK_FILE="SoftwareLog.latest"
SYMLINK_PATH="$TARGET_DIR/$SYMLINK_FILE"

function getLastModifiedFile {
    echo $(ls -t "$TARGET_DIR" | grep -v "$SYMLINK_FILE" | head -1)
}

function getCurrentlySymlinkedFile {
    if [[ -h $SYMLINK_PATH ]]
    then
        echo $(ls -l $SYMLINK_PATH | awk '{print $NF}')j
    else
        echo ""
    fi
}

symlinkedFile=$(getCurrentlySymlinkedFile)
while true
do
    sleep 10
    lastModified=$(getLastModifiedFile)
    if [[ $symlinkedFile != $lastModifiedFile ]]
    then
        ln -nsf $lastModified $SYMLINK_PATH
        symlinkedFile=$lastModified
    fi
done

Background that process using the normal method (again, I don't know zsh, so it might be different)...

./updateSymlink.sh 2>&1 > /dev/null

Then tail -F $SYMLINK_PATH so that the tail hands the changing of the symbolic link or a rotation of the file.

This is slightly convoluted, but I don't know of another way to do this with tail. If anyone else knows of a utility that handles this, then let them step forward because I'd love to see it myself too - applications like Jetty by default do logs this way and I always script up a symlinking script run on a cron to compensate for it.