Logging in multi-threaded application in java

Question

What's the best way and best tool for logging in multi-threaded environment, so that each thread has it's own logger instance and separate file. is this even possible?

Answer 1

I have a multithreaded application that identifies each thread in the log file(dunno about mutliple files, single file shows me concurency), identifying the thread is done automatically by the logging framework, it's Log4J

Edit: Nothing needs to be added in code, you just configure the appender in the logger's to include [%Thread] that will identify the thread from which you are logging the current message, this is example is from log4net:

<appender name="AspNetTraceAppender" type="log4net.Appender.AspNetTraceAppender" >
<layout type="log4net.Layout.PatternLayout">
    <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
</layout>

Here is a list of other common Java logging frameworks

Answer 2

You may try using a custom Log4J appender, which takes thread id as a parameter and filters messages based on what thread calls it. Create it on the fly, attach it to the logger.

There are multiple problems with this approach though:

1. Too many appenders would slow down the logging
2. AppServers usually have a thread pool. That means that the same thread over time would take part in executing totally unrelated requests, which will end up in the same log file.

I suggest you consider a simpler approach: log thread id into the same log file. It's fast and simple, log4j has a % flag for doing it. Later you can grep/split the log file per thread id if required.

Update:

Actually, you may have a single custom appender, which will open log files on demand when a new thread logs a record (the appender is executed within that thread context, just call Thread.currentThread().getName()). But you'll have to re-implement all usual log file tasks (rotation) or delegate it to standard appender for each file.

Answer 3

In some cases knowing the thread identifier is far less important than knowing the context of execution. This would especially be true with lots of threads and larger applications. What do you do when thread ID changes, but the actual execution context is actually the same (as someone mentioned with the thread pools)?. I'd rather use MDC (mapped diagnostic context) to track the execution context in logs. It's better because you control what the context is. At the end, you can set the layout of logs to include MDC and then easily filter out what's relevant. Check out this tool, it does quite a lot of things with MDC.