Message logging, find out where the messages come from?

Question

At this point, we have three websites, an open api, some ten services, and numerous other parts of our infrastructure; and they all can send statistic messages into the queue.

But, there is a problem, we would really like to know where the messages come from, as we had some issues in the past, where a statistic was logged when that shouldn't be possible.

For the websites is fairly easy, we can just insert the url into the queue message, and have a list of regexes to determine whether a message type is allowed for that url.

But how can we do this things in the other applications, that don't have an url? The messages are sometimes logged very deep in the infrastructure, stuff that can be used by all applications, so that's kind of difficult. Does anyone has any good suggestions?

Answer 1

You are on the right track with URL. URL stands for "Unique Resource Locator".

Do you get my point?

Give each logging facility a Unique Identifier.

Process A ---> Alice

Process B ---> Bob

etc...

You can construct your log like this:

[Alice] some stuff happened. [Bob] some more stuff happened.

Answer 2

They must somehow connect to the queue. That would be a perfect point to specify their identity. Create a wrapper for your logging object (or the connector to the queue), so you can store this additional information and prepend it to any message that is put into the queue before it is sent.

[EDIT] In reply of your comment: My solution for a similar case: Pass the logger along somehow. In my case, I have a couple of static methods but I really need to know who uses them (not that they are called; that's obvious from the log message).

Or you can put a token in a ThreadLocal:

String token = "A";
try {
    pushToken(token);
    ...call method of B...
} finally {
    pullToken(token);
}

private static ThreadLocal context; 

public static void pushToken (String token) {
    if (context.get() == null)
        context.set(token);
}

public static void popToken (String token) {
    if (token.equals (context.get()))
        context.set(null);
}

That token will survive the way down the stack.

Answer 3

In case you're using log4j breed of logging APIs you might be interested in looking at MDC - mapped diagnostic context. It was designed particularly for this type of problems. Check out logFaces - it will let you easily fish out particular context data in real time or by doing queries. Otherwise, the ThreadLocal mentioned by Aaron seems reasonable too, it's just that you will have to do the coding yourself instead of using proved concept of MDC used in log4j for a long time and by many users..