Analyze log files from many languages using a single tool. And recommendations of logging frameworks.

Question

We have a system build on lots of languages. The ones we are interested in logging, in order of priority, are:

• C/C++
• PHP
• C#
• Bash
• Java

Wish list:

• If it is possible, we would like logging to be achieved from the above languages in such a way that we may use a single log viewing tool for all of them. Ideally they would be in the same format, but next to that in as few formats as possible and readable from as many log file viewers as possible.
• If it is possible logging to a single log file or a set of log files would be nice. With a possibility to filter based on the source language that is being logged.
• We would like to copy the log files (or should be log to a database and copy it instead?) from multiple servers to a single location. So that we can analyze the log files from many servers at the same time (to see if any of our servers execute a certain piece legacy code for example).
• Being able to change logging level at runtime would be nice.

Thank you for reading! It's quite a complex problem, I hope someone has wrestled with it before and has some valuable information!

Answer 1

If you are using linux, syslog is great! The project I am working on right now uses syslog for most of our logging; everything goes to the same place (/var/log/messages) and you are able to log from many different tools all to the same log file. Apparently you can also use syslog to log to a remote server.

Otherwise if you are on windows you might consider writing to the event log.

Answer 2

Splunk?

C.