Asynchronous thread-safe logging in C++

Question

I'm looking for a way to do asynchronous and thread-safe logging in my C++ project, if possible to one file. I'm currently using cerr and clog for the task, but since they are synchronous, execution shortly pauses every time something is logged. It's a relatively graphics-heavy app, so this kind of thing is quite annoying.

The new logger should use asynchronous I/O to get rid of these pauses. Thread-safety would also be desirable as I intend to add some basic multithreading soon.

I considered a one-file-per-thread approach, but that seemed like it would make managing the logs a nightmare. Any suggestions?

Answer 1

This is VERY possible and practical. How do I know? I wrote exactly that at my last job. Unfortunately (for us), they now own the code. :-) Sadly, they don't even use it.

I intend on writing an open source version in the near future. Meanwhile, I can give you some hints.

1. I/O manipulators are really just function names. You can implement them for your own logging class so that your logger is cout/cin compatible.
2. Your manipulator functions can tokenize the operations and store them into a queue.
3. A thread can be blocked on that queue waiting for chunks of log to come flying through. It then processes the string operations and generates the actual log.

This is intrinsically thread compatible since you are using a queue. However, you still would want to put some mutex-like protection around writing to the queue so that a given log << "stuff" << "more stuff"; type operation remains line-atomic.

Have fun!

Answer 2

I think the proper approach is not one-file-per-thread, but one-thread-per-file. If any one file (or resource in general) in your system is only ever accessed by one thread, thread-safe programming becomes so much easier.

So why not make Logger a dedicated thread (or several threads, one per file, if you're logging different things in different files), and in all other threads, writing to log would place the message on the input queue in the appropriate Logger thread, which would get to it after it's done writing the previous message. All it takes is a mutex to protect the queue from adding an event while Logger is reading an event, and a condvar for Logger to wait on when its queue is empty.

Answer 3

Have you considered using a log library.

There are several available, I discovered Pantheios recently and it really seems to be quite incredible.

It's more a front-end logger, you can customize which system is used. It can interact with ACE or log4cxx for example and it seems really easy to use and configure. The main advantage is that it use typesafe operators, which is always great.

If you just want a barebone logging library:

• ACE
• log4c*
• Boost.Log

Pick any :)

I should note that it's possible to implement lock-free queues in C++ and that they are great for logging.