multiple processes writing to a single log file

Question

This is intended to be a lightweight generic solution, although the problem is currently with a IIS CGI application that needs to log the timeline of events (second resolution) for troible shooting a situation where a later request ends up in the MySQL database BEFORE the earlier request!

So it boils down to a logging debug statements in a single text file.

I could write a service that manages a queue as suggested in this thread: http://stackoverflow.com/questions/119548/problem-in-writing-to-single-file-in-web-service-in-net but deploying the service on each machine is a pain

or I could use a global mutex, but this would require each instance to open and close the file for each write

or I could use a database which would handle this for me, but it doesnt make sense to use a database like MySQL to try to trouble shoot a timeline issue with itself. SQLite is another possability, but this thread

http://www.perlmonks.org/?node_id=672403

Suggests that it is not a good choice either.

I am really looking for a simple approach, something as blunt as writing to individual files for each process and consolidating them accasionally with a scheduled app. I do not want to over engineer this, nor spend a week implementing it. It is only needed occassionally.

Suggestions?

Answer 1

Tryb the simplest solution first - each write to the log opens and closes the file. If you experience problems with this, which you probably won't , look for another solution.

Answer 2

My suggestion is to preserve performance then think in asynchronous logging. Why not send your data log info using UDP to service listening port and he write to log file.

Answer 3

I would also suggest some kind of a central logger that can be called by each process in an asynchronous way. If the communication is UDP or RPC or whatever would be an implementation detail.

Answer 4

You can use file locking. Lock the file for writing, write the message, unlock.

Answer 5

Even thought it's an old post, has anyone got an idea why not using the following concept:

1. Creating/opening a file with share mode of FILE_SHARE_WRITE.
2. Having a named global mutex, and opening it.
3. Whenever a file write is desired, lock the mutex first, then write to the file.

Any input?