MongoDB to store different (schema) log files

Question

Do you think that using a MongoDB Json Database to store log files from application is a good idea and why ?

The only advantage for me is the schema abstraction, but i think it's also a weakness we cannot ensure the integrity of a log file.

Answer 1

Obviously I'm biased (I work on MongoDB) but I think it it works very well for logs.

Reasons:

• It's fast for inserts and updates... you can do thousands per second
• As well as normal queries, you can run analytics and generate reports using JavaScript. You could have a cron job running nightly which does nice MapReduce things to your logs.
• You can use capped collections, which are collection that act like queues, to keep only the latest N KBs/MBs/GBs of logs

I'm not sure what you mean "ensure the integrity of a log file"... do you mean you are worried about not knowing what fields the document you're pulling out has? If so, I think you'll find it's no harder dealing with null fields in a relational database and much more flexible.

See also: the MongoDB blog post on logging.