Organizing logs with log4j

Question

Hey all,

I'm currently working on a Linux daemon written in Java. What is the common naming scheme for logs?

Right now I'm thinking of doing something like:

DEBUG = /var/log/myapp.debug
INFO = /var/log/myapp.info
WARN = /var/log/myapp.warn
ERROR = /var/log/myapp.err
FATAL = /var/log/myapp.err

Does anyone have any opinions / suggestions on the naming scheme? I will accept the answer that I end up going with.

Hopefully this question doesn't get closed since there are other topics similar that are opinion related (like what IDE do you prefer, etc)

Answer 1

For Java apps, I recommend two logs.

The first is the stderr and stdout of the start of the process. Configure log4j to send ERRORs and FATAL to the CONSOLE as well as FILE.

The second log should be managed by log4j, and include all DEBUG-FATAL errors. This will be the detail log which you'll need for debugging complex problems. In your log4j, adjust DEBUG and INFO filters to not show packages you don't care about. For example, you generally don't need org.apache or org.jboss at DEBUG level.

For structure, I would do the following:

/var/log/myapp/boot.log.$(date +%s)
/var/log/myapp/server.log.$(date +%Y%m%d)

Have the server log rotate daily, while the boot log will create a new file on each start.

It does not make sense to separate out DEBUG, INFO, etc.. into their own log files, as you usually need the context from different log levels to be in the same file.

If you don't capture the process stderr log, you'll won't be able to capture a stacktrace using kill -3, you won't see system error messages like Stale NFS handle, and you won't get details of a JVM crash.

Answer 2

I'd use the following scheme for application logs:

/var/log/<app>/myapp.log

In other words, everything goes into one file, regardless of the log level (there is already enough complexity when you have one log file per instance in clustered multi-tiered environments, no need for more). You usually use DEBUG during development, ERROR for production (sometimes with another pattern layout for monitoring purpose but this is another story). And the log level may be changed on demand for specific loggers.

Answer 3

I would strongly recommend that you use one log instead of several as you propose, because:

• Log output contains enough information (e.g. "[ERROR]") to filter by level.
• The relative order of logging statements of different levels either cannot be reconstructed (no timestamp is included) or is not trivial to reconstruct (timestamp included) when split into separate files by level.

As for the amount of logging that occurs... this should probably be dynamically configurable (maybe you want to use an XML or Java Properties configuration file for this?). WARN or ERROR is a reasonable logging level to use as a default. During development, it is sometimes useful to enable more fine-grained logging.

Answer 4

I wanted to make it easier to find specific issues without having to grep the file

If this is the purpose, I'd recommend looking at logFaces where management of log files is practically reduced to zero. You can instantly find precise information in the ocean of log data and create log files only when they really needed and contain only what you actually want. No need to create complex log management utilities when it can be done by the third party. All it takes is an appender to centralized log aggregator. Few lines in configuration file, no more. My opinion is a bit biased though :)