Minimize performance impact of event logging?

Question

I have a static logging function that uses StringBuilder to concatenate a bunch of query parameters before sending the string to a log. This process can get moderately long, as we may have ~10 parameters (.Append calls) and end up being ~200 chars long.

I want to minimize the performance impact of the logging function. (This logging function may be called multiple times per web request, and we measure the processing time for each web request)

How/Should/Can I build a "pool" of StringBuilders to improve performance?

I can also do all this logging asynchronously, right? How should I do that?

Answer 1

If you anticipate bursts of logging activity, and IF the event logging actually causes a performance problem that you have measured (see Premature Optimization under "Levels" of Optimization / Design Level), you can create a queue for logging requests and a separate thread that works off the queue. You would want the queue to block the caller if it's full, until it becomes not full.

If your logging requests are fairly steady rather than bursts of activity, you will still gain overall if there is generally a CPU core not otherwise in use. If all CPU cores are generally in heavy use, a separate thread will only add to overhead and complexity without providing benefits.

Answer 2

What you would probably want is an instance of a StringBuilder for every HttpRequest your app is processing. A simple way to do that is to create this StringBuilder in your Global.asax.

You should not worry about synchronization on this stringbuilder, because at any given time it will only be accessible by one httpRequest.

When the request processing is completed you will have to push the content of the stringBuilder to some central place (a log) and this where you WILL have to worry about synchronization, but if you will do it in the OnEndRequest event the impact will be minimal.

Do not forget to clean the stringBuilder after you moved its content to the log

Answer 3

Since logging is usually state-dependent, putting the actual construction of the log entry off into another thread usually isn't an option. You could, however, capture the relevant data to format asynchronously. While I won't implement the entire logging mechanism here (though you can see my ProcessQueue article on CodeProject for something that will make it easier), you could do something like this:

public static void LogAsync<T1>(T1 value, Func<T1, string> formatter)
{
    // asynchronously call formatter(value) and log the result
}

public static void LogAsync<T1, T2>(T1 value1, T2 value2, Func<T1, T2, string> formatter)
{
    // asynchronously call formatter(value1, value2) and log the value
}

...and so on

If the string construction is what's bogging down the logging, then (assuming that the various T types are immutable, or at least don't change between the call to LogAsync and when formatter is invoked) this should alleviate that.

Answer 4

No, you should not pool StringBuilder objects. They are cheap to create and use. If you pool them you will just turn them from short lived objects into long lived objects. It will actually be more work for the garbage collector to move one StringBuilder to the next generation to keep it than to clean up a whole bunch of them.

The use of StringBuilders will definitely not be the bottle neck. Most of the work will be storing the strings in the log, wherever that is.

Doing the logging asynchronously won't save you any performance in itself. The server still has to do the same work, and you just add the overhead of creating another thread.

What might make it more efficient is if you keep a buffer of strings to be logged, and store a bunch of them at the same time. Provided of course that storing several strings can actually be done more efficiently than storing them one and one.

You could keep a list of strings in a static variable, and store them when you have enough of them. (Synchronising the access of course, as several threads can access it.) Or simply gather the strings from one web request and save at one, which is a lot easier but still can save you some work.