views:

281

answers:

3

I would like to build an Appender (or something similar) that inspects Events and on certain conditions creates logs new Events.

An example would be and Escalating Appender that checks if a certain amount of identical Events get logged and if so logs the Event with a higher logleve. So you could define something like: If you get more then 10 identical Warnings on this logger, make it an Error.

So my questions are:

  1. Does something like this already exist?

  2. Is an Appender the right class to implement this behavior?

  3. Are there any traps you could think of I should look out for?

Clarification: I am fine with the algorithm of gathering and analysing the events. I'll do that with a collection inside the appender. Persistence is not necessary for my purpose. My question #2 is: is an appender the right place for this to do? After all it is not normal behaviour to creat logging entries for an appender.

+2  A: 

You can create your own appender by implementing the Appender interface provided by log4j.

http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Appender.html

That would be one approach. Another would be to use an existing appender and then write some code that monitors the log. For example, you could log to the database and then write a process that monitors the log entries in the database and creates meta-events based on what it sees.

It depends most on what you're comfortable with. One question you'll have to deal with is how to look back in the log to create your meta-events. Either you'll have to accumulate events in your appender or persist them somewhere that you can query to construct your meta-events. The problem with accumulating them is that if you stop and start your process, you'll either have to dump them somewhere so they get picked back up or start over whenever the process restarts.

For example, let's say that I want to create a log entry every 10th time a NullPointerException is thrown. If I have the log entries in a database of some kind, every time an NPE is thrown I run a query to see how many NPEs have been thrown since the last time I created a log entry for them. If I just count them in memory every time one is thrown, if I restart the application after 5 are thrown, if I don't persist that number I'll lose count.

Rafe
A: 

As Rafe already pointed out, the greatest challenge would be persisting the actual events in the Appender, so that you'll know the time has come to trigger your event (e.g. escalate log level).

Therefore, I propose a following strategy:

  1. Use a custom JDBCAppender. Unlike the one bundled with Log4j, this one can log exceptions.
  2. Set-up an embedded database, like HSQLDB, and set-up a database with one table for event logging. It solves the persistence problem, as you can use SQL to find types of events that occurred.
  3. Run a separate thread that monitors the database, and detects desired event patterns.
  4. Use a LogManager to access desired Loggers and set their level manually.
javashlook
Jens stated: "Persistence is not necessary for my purpose."
Ceki
Persistence comment was added after I posted an answer. Still, HSQLDB can be configured to work in memory only (see: http://hsqldb.org/doc/guide/ch01.html#N101CA). Using that solution would enable moving event analyzing logic from out of the Appender, in both structural and temporal sense.
javashlook
In my experience, writing to DB appender (1 millisecond per event) compared to using a turbo filter (100 nanoseconds per event) would be at least 10'000 slower. Assuming a very fast data base, this ratio could be brought down but still would be several orders of magnitude slower.
Ceki
+1  A: 

Logback (log4j's successor) will allow you to enable logging for any event via TurboFilters. For example, assuming the same event occurs N or more times in a given timeframe, you could force the event to be accepted (regardless of its level). See also DuplicateMessageFilter which does the inverse (denying re-occurring events).

However, even logback will not allow the level of the logging event to be incremented. Log4j will not either. Neither framework is designed for this and I would discourage you from attempting to increment the level on the fly and within the same thread. On the other hand, incrementing the level during post processing is a different matter altogether. Signaling another thread to generate a new logging event with a higher level is an additional possibility. (Have your turbo-filter signal another thread to generate a new logging event with a higher level.)

It was not clear from your question why you wished the level to be incremented. Was the increment of the level a reason in itself or was it a means to a goal, that is having the event logged regardless of its level. If the latter, then logback's TurboFilters are the way to go.

HTH,

Ceki
incrementing the level is basically just an example of something I might like to do. "I would discourage you from attempting to increment the level on the fly and within the same thread" Does that include creating a new Evenet i.e. calling Logger.log(..), or just changing the level (I don't even know if this is possible at all.
Jens Schauder
TurboFilters look very promissing
Jens Schauder
You can't change the level of an existing event because level is a read-only field. Creating a new event while processing one can become extremely hairy. Hence my recommendation to avoid doing it. You can do whatever you want, including creating a new event, as long as it is from a different thread.
Ceki