Have you seen an appender that would log to separate files based on NDC in Log4j?

Question

We have only a very small number of customers (fewer than 50) and we would like each one to have its own separate log file into which all of its server side logging info goes. I know you could use NDC and filters to direct log statements to different files using the standard appenders in Log4j but that would require quite a bit more setup (to create loggers for all existing customers) and it wouldn't automatically accomodate the addition of a new customer into the system.

Have you ever seen an appender which would split out log statements to separate files based on the Log4j NDC value? I figure I could build a new appender to do it but I can't say I want to if I can get something already built that works well.

Answer 1

I am not aware of any standard or even 3rd party appenders that can do this and while you are correct that you could write your own I would be inclined to just use one of the standard file appenders (rolling or otherwise) and use a tool that can filter lines of text (i.e. exclude any lines not matching a pattern from view) to read the log. Most editors that can filter can also set up filter sets so you can reuse existing patterns. I won't discuss which editor to use as I don't even know what OS you're using and everyone has their own preferences.

Answer 2

It sounds like you want a multi-file appender sort of thing, where you give a part of a filename and the rest is filled in with the NDC. I am not aware of anything like this. Probably you would have to roll your own.

If you roll your own, you probably want to create an Appender that internally uses a Map of RollingFileAppenders that is created dynamically. I don't know if an Appender has access to the NDC, however. This would probably be a non-trivial undertaking.

Answer 3

I never met such appender also, but I don't think it's a big deal to write your own. If you want to do it yourself - look at AppenderSkeleton, minimum you will have to do is to override append(..) method. When your appender will be called, you will get fully formed event which will contain everything needed, including NDC. Do whatever you like with it then.. I think MDC (mapped diagnostic context) in your case is viable alternative, it's more flexible IMHO.

If you're looking for commercial solution - look at logFaces, it was designed for handling similar situations.

Answer 4

SiftingAppender which ships with logback (log4j's successor) is designed precisely to handle this situation.

As its name implies, a SiftingAppender can be used to separate (or sift) logging according to a given runtime attribute. For example, SiftingAppender can separate logging events according to user sessions, so that the logs generated by every user go into distinct log files, one log file per user. For example, SiftingAppender can separate logging events into distinct log files, one file per user.

The documentation for SiftingAppender contains an example for separating logs based on user id.