Rapid Opening and Closing System.IO.StreamWriter in C#

Question

Suppose you have a file that you are programmatically logging information into with regards to a process. Kinda like your typical debug Console.WriteLine, but due to the nature of the code you're testing, you don't have a console to write onto so you have to write it somewhere like a file. My current program uses System.IO.StreamWriter for this task.

My question is about the approach to using the StreamWriter. Is it better to open just one StreamWriter instance, do all of the writes, and close it when the entire process is done? Or is it a better idea to open a new StreamWriter instance to write a line into the file, then immediately close it, and do this for every time something needs to be written in? In the latter approach, this would probably be facilitated by a method that would do just that for a given message, rather than bloating the main process code with excessive amounts of lines. But having a method to aid in that implementation doesn't necessarily make it the better choice. Are there significant advantages to picking one approach or the other? Or are they functionally equivalent, leaving the choice on the shoulders of the programmer?

Answer 1

Look at pre-rolled logging implementations; they may save you lots of headaches. Obviously keeping the stream open means you might lose some of the end data if it crashes, but may gaing performance from buffering the IO more. Some implementations may also offer features such as asynchronous logging from a spooler/queue.

Answer 2

Repeated opening/closing a new StreamWriter for each write will generate a lot of resources for the GC plus impose overhead on the app due to the actual lookup of the file for every open operation. On the other hand, holding a single stream open will hold a lock on the file. So it depends.

You don't want your logging mechanism to become a performance bottleneck, so write to a single stream. Make it unbuffered or AutoFlush for critical debugging (but be advised, that also has a performance impact).

I'd follow the model of log4net, create a static log stream, and write to that singleton. Look into log4net anyway, so you don't roll your own. http://logging.apache.org/log4net/index.html

Answer 3

I would buffer/queue the data & write out to the file once it reaches a threshold & flush the whole queue when the application closes/exits normally.

The only issue in this is in case of an application crash, you might loose the items in the queue...

HTH.