How to safely update a file that has many readers and one writer?

Question

I have a set of files. The set of files is read-only off a NTFS share, thus can have many readers. Each file is updated occasionally by one writer that has write access.

How do I ensure that:

1. If the write fails, that the previous file is still readable
2. Readers cannot hold up the single writer

I am using Java and my current solution is for the writer to write to a temporary file, then swap it out with the existing file using File.renameTo(). The problem is on NTFS, renameTo fails if target file already exists, so you have to delete it yourself. But if the writer deletes the target file and then fails (computer crash), I don't have a readable file.

nio's FileLock only work with the same JVM, so it useless to me.

How do I safely update a file with many readers using Java?

Answer 1

I don't know if this is applicable, but if you are running in a pure Vista/Windows Server 2008 solution, I would use TxF (transactional NTFS) and then make sure you open the file handle and perform the file operations by calling the appropriate file APIs through JNI.

If that is not an option, then I think you need to have some sort of service that all clients access which is responsible to coordinate the reading/writing of the file.

Answer 2

According to the JavaDoc:

This file-locking API is intended to map directly to the native locking facility of the underlying operating system. Thus the locks held on a file should be visible to all programs that have access to the file, regardless of the language in which those programs are written.

Answer 3

On a Unix system, I'd remove the file and then open it for writing. Anybody who had it open for reading would still see the old one, and once they'd all closed it it would vanish from the file system. I don't know if NTFS has similar semantics, although I've heard that it's losely based on BSD's file system so maybe it does.

Answer 4

I have been dealing with something similar recently. If you are running Java 5, perhaps you could consider using NIO file locks in conjunction with a ReentrantReadWriteLock? Make sure all code referencing the FileChannel object ALSO references the ReentrantReadWriteLock. This way the NIO locks it at a per-VM level while the reentrant lock locks it at a per-thread level.

FileLock fileLock = filechannel.lock(position, size, shared);
reentrantReadWriteLock.lock();

// do stuff

fileLock.release();
reentrantReadWriteLock.unlock();

Of course, some exception handling would be required.

Answer 5

There might be no need for locking. I am not too familiar with the FS API on Windows, but as NTFS supports both hard links and soft links, AFAIK, you can try this if your setup allows it:

Use a hard or soft link to point to the actual file, and name the file diferently. Let everyone access the file using the link's name.

Write the new file under a different name, in the same folder.

Once it is finished, have the file point to the new file. Optimally, Windows would allow you to create the new link with replacing the existing link in one atomic operation. Then you'd effectively have the link always identify a valid file, either the old or the new one. At worst, you'd have to delete the old one first, then create the link to the new file. In that case, there'd be a short time span in which a program would not be able to locate the file. (Also, Mac OS X offers a "ExchangeObjects" function that allows you to swap two items atomically - maybe Windows offers something similar).

This way, any program that has the old file already opened will continue to access the old one, and you won't get into its way creating the new one. Only if an app then notices the existence of the new version, it could then close the current and open it again, this way getting access to the new version.

I don't know, however, how to create links in Java. Maybe you have to use some native API for that.

I hope this helps anyways.

Answer 6

Something that should always work, no matter what OS etc, is changing your client software.

If this is an option, then you could have a file "settings1.ini" and if you want to change it, you create a file "settings2.ini.wait", then write your stuff to it and then rename it to "settings2.ini" and then delete "settings1.ini".

Your changed client software would simply always check for settings2.ini if it has read settings1.ini last, and vice versa.

This way you have always a working copy.