I'm logging to a file using FileStream.Write(). When the file reaches a maximum size, I want to delete a load of the logs from the start of the file. What's the best way of doing this?
The best idea I've got so far is to create a second file. Write everything I want to keep from the original file into it. Then delete the original file, and rename the second file with the original files name.
Is there a more simple and efficient way I can do this?
If the maximum size is not too large, you can load the whole file into memory, delete the parts you want to delete and then directly overwrite the old file.
That sounds like a fair way to do it. Since "truncating" the beginning of files isn't directly possible on most filesystems, you're going to be forced to rewrite all the data, one way or another. Your two options are essentially:
Read the existing log, store it into memory, then overwrite the entire file having chopped off the initial part. This risks losing the data if an error occurs during the process (or if the OS/program crashes unexpectedly for some reason).
Your suggested method of creating a new file and then deleting the old one. This has the added benefit that if something goes wrong in creating the new one, your old one is still there and prefectly intact.
It definitely seems like you want to go with the second method.
If the size doesnt have to be exact, maybe its easier to count lines? When the max is reach, go back to the beginning and delete everything up to and including the newline character. Then go back to the end.
Of course, it would be even easier if you dont do any file writing at all until the program ends at which point you write all the cached lines. Although, that may not be feasible if your program is running for a long time and you want to see the log during run with tail or something similar.
Use the log4net library - it allows rolling log files with a maximum size per file and a maximum number of files.
I think you should just treat the log file like a ring buffer. Of course, because the lines might have variable lengths, you will certainly have half lines sometimes - you might consider just blanking them out.
I really would not consider rewriting the whole file every time a new line comes in - much to expensive. Maybe you should even use log4net or have a look at the source code to see how they do it.
Yes, and no ;-)
Which is important to you: efficiency, or simplicty?
More efficient: A traditional rolling log is a fixed-sized-record. You use a random access file in read/write mode, keep track of the current record pointer, and when current record == max records set current record back to zero. Obviously, this places restrictions on what you can log... DB's do this on "raw disk" to enable them to random-access variable record size. This also requires a custom log-reader application. Yeck!
Simpler (and still a tad more efficient than copying thousands of lines from file to file): write many but smaller log files. "Roll over" the log file periodically, or when it reaches a certain size... i.e. start a new log-file and if numLogs == maxLogs then delete the oldest one.
Cheers. Keith.