Log in one place versus multiple logs

Question

For a web app or a standalone server app, which would you recommend, and why?

• have the giant application.log, where everything is logged;
• have many smaller logs:
  • security.log
  • performance.log
  • lifecycle.log
  • integration.log

Answer 1

Depends on your logging reviews. You should be in the best place to answer it. Do you need specific logs so you can identify issues with better accuracy, or do you just need an overall log for maintenance purposes?

I would always use different logs, provided they don't cause too much overhead.

Answer 2

I like using databases for logging. Four useful features:

1. You don't lose the time-ordering, as you'd experience when looking at multiple log files at once.
2. You can still filter by specific message types, if you want to.
3. You get integrity, so if your computer crashed just as you were writing a log file, you won't get a corrupted log, and it'll be replayed off the journal when your database starts up again.
4. Pruning the log is really easy! No need to use hacky log rotation programs that require your daemons to be SIGHUPed or anything.

Your mileage may vary. :-)

Answer 3

I'd suggest multiple logs for a couple of reasons:

• Reduce noise, if there are production troubleshooting situations where time counts.
• Different groups can get different logs. A systems group may want different logs than an applications group as each has their part of the system where they can optimize settings.

As for how to log, I'd suggest a mixture of database, e-mail and local text file just in case there are issues with things that require connectivity to another server.

Answer 4

If you expect lots of logging and you will quickly know where to look then split them up. One big log file can get unruly real quick, but if you are constantly looking in multiple log files for the relavent entry, that's no good either. So when you split them up, make sure it's in a way where when you get an error, you know exactly which file to open first.

Answer 5

I'd recommend using centralized logging system off-the-shelf and not inventing your own. This will allow you to easily dig out data in any form whenever you like. It also should save you time and headache. Managing log files is not only a burden on your application because you have to write additional code, but also for humans, who actually look at the text. Check out logFaces, it might solve some of your problems - it comes with it's own database, can be used with most commercial database brands and allows instant access to relevant log data.