As a followup on this question discussing the use of the ThreadPool vs a dedicated thread:
When would you use a dedicated thread (with lowered priority) for applicationwide logging and when would you use the ThreadPool?
A:
Generally speaking logging should usually be done synchronously - if your diagnosing an application error then when looking at logs you really need to be able to guarantee that:
- If a logging statement was executed, the log message was definitely written to the log.
- You know what order the log statements were executed in.
Now although its possible to get around #2 by working out the log message number (or similar) in advance of the logging, its not possible to get around #1 unless you synchronously log messages.
Besides, there should be no need to go to the effort of logging asynchronously - if your application performance is being affected by logging then it probably means that you are logging too much.
Kragen
2010-10-18 13:45:02
Logging involves a lot of IO when I don't want to interfere the main application execution, this would be a really a bad idea.
bitbonk
2010-10-18 13:46:45
I concur with bitbonk. Use a queue if you need to preserve the order of your log messages, but perform the actual IO asynchronously.
Chris
2010-10-18 13:51:19
@bitbonk - What happens when an unexpected error condition goes onto cause heap corruption and all your asynchronous log messages are lost? If IO from logging is interfering with application performance then you are probably logging waaaay too much (or you are confusing logging with auditing)
Kragen
2010-10-18 14:52:23
IO is *always* a performance issue since IO can be (unexpectedley) slow. I have no idea how heap corruption is supposed to occur in a purely managed application, but if for some reason an error occurs while logging I'd prefer my logmessages getting lost instead of breaking the application execution. If the error occurs in the executing application code in most cases I still should be able to log that exception.
bitbonk
2010-10-18 15:08:01
A:
What I would do is completely dependent on the requirements of my app and its logging component.
If logging is mission-critical (maybe you need the ability to replay recent traffic based on the log, for example) then a dedicated thread is more likely the right approach.
If logging is 'best effort', then ThreadPool would be fine subject to other constraints on your app's required performance and latency. Async I/O for the logger would be fine here. Since you suggest lower priority for your putative logger thread, this may match your app's profile.
If more critical work is happening on the ThreadPool then I would not overload it to do logging, esp. if logging itself is important - you could well be doing synchronous, flushed I/O to write out the logs and that's a possible bottleneck depending on the volume of stuff that you wish to log.
Steve Townsend
2010-10-18 13:47:54
If it is 'best effort' how about this argument FOR a dedicated thread: You can alter (lower) the priority of a dedicated thread. AFAIK you can't (or shouln't) do that with the threadpool.
bitbonk
2010-10-18 13:55:32
@bitbonk - I would not go to this trouble unless you must. A better argument for dedicated thread might be to preserve ordering. There are producer/consumer constructs built in to .Net to do that now, though. Are you on .Net 4.0?
Steve Townsend
2010-10-18 13:57:26
@bitbonk - you might be able to use `Microsoft.Practices.EnterpriseLibrary.Logging` - this supports async logging in .Net 3.5 - see here for scenarios - http://msdn.microsoft.com/en-us/library/ff664736(v=PandP.50).aspx
Steve Townsend
2010-10-18 14:36:34
Before adding a dependency on Entlib I'd rather add one for log4net. :) But so far I am confident that a small single class service would suffice for my needs.
bitbonk
2010-10-18 15:10:17
I could not see where `log4net` supports async logging, but some off-the-shelf library is probably what you need.
Steve Townsend
2010-10-18 15:12:58
A:
If logging is not critical and you want to do it asynchronously then I would recommend using a single background thread for logging and a producer/consumer queue to send log messages. This can achieve improved performance over threadpool since you have a single thread performing I/O on less-critical logs which would have less of a likelyhood to block higher-priority I/O on other threads.
You can also use this mechanism to make sure critical logs are committed before logging. Add them to the queue and then have a mechanism to wait until that particular message is committed.
Sam
2010-10-18 13:59:56