Is file append atomic in UNIX?

Question

In general, what can we take for granted when we append to a file in UNIX from multiple processes? Is it possible to lose data (one process overwriting the other's changes)? Is it possible for data to get mangled? (For example, each process is appending one line per append to a log file, is it possible that two lines get mangled?) If the append is not atomic in the above sense, then what's the best way of ensuring mutual exclusion?

Answer 1

A write that's under the size of 'PIPE_BUF' is supposed to be atomic. That should be at least 512 bytes, though it could easily be larger (linux seems to have it set to 4096).

This assume that you're talking all fully POSIX-compliant components. For instance, this isn't true on NFS.

But assuming you write to a log file you opened in 'O_APPEND' mode and keep your lines (including newline) under 'PIPE_BUF' bytes long, you should be able to have multiple writers to a log file without any corruption issues. Any interrupts will arrive before or after the write, not in the middle. If you want file integrity to survive a reboot you'll also need to call sync(2) after every write, but that's terrible for performance.

Answer 2

It's possible, but unlikely thet the log file will get mangled. I've been writing MT applications for years on Solaris and Linux and have never seen it happen. And if it does happen, i'ts only a log file - you can almost always live with it. Please don't start writing complex locking code for logfiles - it defeats their basic purpose, which is to record state changes with the mionimum of overhead.

Answer 3

Here is what the standard says: http://www.opengroup.org/onlinepubs/009695399/functions/pwrite.html.

If the O_APPEND flag of the file status flags is set, the file offset shall be set to the end of the file prior to each write and no intervening file modification operation shall occur between changing the file offset and the write operation.