Best way to atomically create files

Question

Hi SO,

The 'best practice' (as I see it) to atomically create a new file, is to open a temporary file (using tmpfile()), and then moving the file to it's final location.

However, this won't work well if the temporary file is on a different mountpoint, as this will result in the file gradually building up and additionally result in unneeded IO overhead.

Another option is to create a temporary file in the same directory as the final destination, but this has the disadvantage of creating a unusual file for a user (Apps such as MS Word and ViM do this, but I also consider this bad behaviour).

Is there a similar method as tmpfile() that will allow me to specify the mountpoint? I realize this probably doesn't exist built-into PHP, so a Posix/C-function or shell-call is also acceptable.

Answer 1

As you are talking about the "mountpoint" I'm assuming you are on a unix-like environment.

Maybe you can consider it a workaround or bad-behavior, but I think creating an hidden (.tmpfile) temp file on the same destination folder could be acceptable.

You can of course create a specific folder on the same mount point not accessible to the application suited for this task, simulating this way a tempfile() on the same mountpoint, if you don't want to see any spurious file in the destination dir.

Answer 2

No, there isn't such a method in the POSIX stack. tmpfile() and tmpname() use to normal temp dir. There is the tempnam(), where you can specific the target directory for the temp file. But is is basically a way to implement the second option you described.

Answer 3

I had to do something like this and went with a MySQL DB. Just stored the info I needed in a table and when I was finished I just deleted the record. Just a thought :)

Answer 4

The maildir protocol developed for qmail provides safe file creation for multiple writers to the same target directory, even across NFS. In this scheme, the "tempfile" directory is guaranteed to be on the same filesystem as the target dir.

The algorithm is conveniently implemented in an efficient shell utility, safecat, whose manpage presents the algorithm as:

safecat applies the maildir algorithm by writing data in six steps. First, it stat()s the two directories tempdir and destdir, and exits unless both directories exist and are writable. Second, it stat()s the name tempdir/time.pid.host, where time is the number of seconds since the beginning of 1970 GMT, pid is the program's process ID, and host is the host name. Third, if stat() returned anything other than ENOENT, the program sleeps for two seconds, updates time, and tries the stat() again, a limited number of times. Fourth, the program creates tempdir/time.pid.host. Fifth, the program NFS-writes the message to the file. Sixth, the program link()s the file to destdir/time.pid.host. At that instant the data has been successfully written.

In addition, safecat starts a 24-hour timer before creat-ing tempdir/time.pid.host, and aborts the write if the timer expires. Upon error, timeout, or normal completion, safecat attempts to unlink() tempdir/time.pid.host.