Why doesnt "tail" work to truncate log files?

Question

I'm trying to manage my log file size using a cron script. I basically want to delete all but the last 2000 lines of the log file every night. I'm trying to run this command, but it seems to be emptying the entire file instead of doing what I want:

tail -2000 logfile.txt > logfile.txt

Does anyone know why this isn't working and/or how to accomplish what I want? Thanks!

Answer 1

You are overwriting the file before tail even starts to read it. The shell processes the > redirect operator, by clearing out the file first. Then it runs tail which has no data to read.

You can solve this by using a temporary file:

tail -2000 logfile.txt >logfile.tmp
mv logfile.tmp logfile.txt

Answer 2

Greg Hewgill is right, logfile.txt is being truncated before tail can work on it.

try:

tail -2000 logfile.txt > logfile2.txt; rm -f logfile.txt; mv logfile2.txt logfile.txt

Answer 3

Rather than doing this with your own cron file you might want to look into using logrotate for a more robust solution. You can rotate the logs, control how long to keep them around, mail them, compress old logs, and run scripts afterwards if you want to.

http://linuxcommand.org/man%5Fpages/logrotate8.html or type man logrotate from the command line

Create a new logrotate file in your /etc/logrotate.d/ directory. Here's an example:

/var/logs/myapp/logfile.txt {
  # keep 5 files
  rotate 5
  # rotate once the file is bigger than 2k
  size 2k
  # don't error if the file isn't there
  missingok
  # old files are compressed
  compress
  # set ownership and permissions on the new log file
  create 0640 myuser myuser
}

Answer 4

There is a problem with the accepted solution if the process keeps the log file open; you basically need to reuse the i-node. Mmrobins' response is good, logrotate should do the right thing.

To use tail, you can do something (similar to Pantonza's & Greg's idea), but retain the original file by truncating the original file in-place:

tail -2000 logfile.txt >logfile.tmp
cat logfile.tmp > logfile.txt
rm logfile.tmp

To avoid a temp file, you could read into a variable, then stuff it back:

bash -c 'X=$(<tail -2000 logfile.txt);echo "$x">logfile.txt'

In all cases, there's the possibility of a race condition between your truncation and the process appending to the file. Not sure if logrotate handles that, none of the tail solutions here do.