views:

28

answers:

3

I'm writing a simple application in vb.net and need to output some information to a log file for diagnostic purposes.

In order to ensure that the log file doesn't get too big, I need to trim it to a certain number of lines (say 5000). For performance reasons, the trimming doesn't have to occur each time a log entry is written out nor is it critical that trimming occurs (for example, if the application crashes prematurely).

However I'm not really sure the best way to trim it. Loading the entire log into memory and then re-writing only the last 5000 lines back out would work, but I'm sure there is a cleaner and better way to do this.

Can anyone please suggest a more efficient method? I'm fairly new at vb.net so code examples would be really appreciated.

+1  A: 

Rather than worrying about modifying data I've already written, I would just follow the time-honored tradition of starting a new log file at a certain point, either when the current one reaches a certain size, or on a schedule basis (hourly, daily or otherwise).

Just rename the current logfile from xyzzy.log to xxyyz-YYYYMMDDHHMMSS.log then start a brand new copy of xyzzy.log.

This also gives you a decent trade-off between maintaining a full audit of logs and preserving disk space. You can set up procedures to delete log files more than 30 days old, or keep the aggregate size below some threshold, or even hive off older logs to a secondary storage area.

These procedures don't even have to be part of your program - I have, in the past, simply had a scheduled task which cleans up old log files, a task that shared nothing with the program proper, other than the directory where the script resided.

This is probably the easiest way to do it short of integrating a third-party product (which isn't always possible) and, since log files are almost invariably for the programmer rather than the user, the naming standards shouldn't be too onerous.


A very simple solution I've used in the past (admittedly with Linux) is to actually not worry about renaming at all. Simply have the log function open the file using the current date, and remember what the date was. Every time you call the log function after that, check if the date has changed and, if so, open a new log file for subsequent logging. That gives you one log file per day with very little effort by the program.

Then I just had scripts using find -mtime to delete those over a certain age.

paxdiablo
Aside from using something like log4net below, this is exactly what I would do as well. This would mean zero reading of file data, and always writing (much more efficient). You could even add a few lines of code to delete files that are old based on a date, or based om timestamps keeping the last [x] files.
enforge
A: 

You should try using a 3rd party library, most of which should include this sort of functionality - e.g. log4net, NLog, Enterprise Library.

Most of them are straightforward to use, depending on how much logging you're doing, it will probably be quicker to use one of these than write such functionality yourself, it will be reliable and well tested, and provide other useful features too.

EDIT

My main experience is with Enterprise Library, with which I always used the technique @paxdiablo describes in his answer, starting a new log file every x days/weeks/Kb/etc. I'm not certain about truncating a single file to a certain size, but I imagine this is possible with those libraries.

Grant Crofton
A: 

One approach is to leverage an open-source solution for this. Log4Net is one that many of my peers have used over and over again. It has the functionality to handle what you are talking about. You add the reference, add a config file to set your logging options (how much to keep, etc), and call the library to log.

http://logging.apache.org/log4net/

enforge