How to ensure file integrity on file write failures?

Question

Follow up to: How to safely update a file that has many readers and one writer?

In my previous questions, I figured out that you can use FileChannel's lock to ensure an ordering on reads and writes.

But how do you handle the case if the writer fails mid-write (say the JVM crashes)? This basic algorithm would look like,

WRITER:
  lock file
  write file
  release file

READER:
  lock file
  read file
  release file

If the JVM crashes during write file, sure the lock would be released, but now I have an incomplete file. I want something complete to always be readable. Either the old content the new content and nothing in between.

My first strategy was to write to a temporary file, and then copy the contents into the "live" file (while ensure good locking). The algorithm for this is,

WRITER:
  lock temp file
  write temp file
  lock file
  copy temp to file
  release file
  release temp
  delete temp

READER:
  lock file
  read file
  release file

One nice thing is the delete temp won't delete the temp if it has already been locked by another writer.

But that algorithm doesn't handle if the JVM crashes during copy temp to file. So then I added a copying flag,

WRITER:
  lock temp file
  write temp file
  lock file
  create copying flag
  copy temp to file
  delete copying flag
  release file
  release temp
  delete temp

READER:
  lock file
  if copying flag exists
    copy temp to file
    delete copying flag
    delete temp 
  end
  read file
  release file

There won't ever be two things accessing the copying file as it is guarded by the file lock.

Now, is this the way to do it? It seems very complicated to ensure something very simple. Is there some Java library that handles this for me?

EDIT

Well, I managed I make a mistake in my third attempt. The reader doesn't hold the lock to temp when it does copy temp to file. Also its not a simple fix to simply lock the temp file! That would cause the writer and reader to acquire locks in different orders and can lead to deadlock. This is getting more complicated all the time. Here's my fourth attempt,

WRITER:
  lock file
  write temp file
  create copying flag
  copy temp to file
  delete copying flag
  delete temp
  release file

READER:
  lock file
  if copying flag exists
    copy temp to file
    delete copying flag
    delete temp 
  end
  read file
  release file

This time the temp file is guarded by main lock, so it doesn't even need its own lock.

EDIT 2

When I say JVM crash, I actually mean say the power went out and you didn't have a UPS.

EDIT 3

I still managed to make another mistake. You shouldn't lock on the file you are writing to or reading from. This will cause problems, since you can't get both the read and write lock unless you use RandomAccessFile in Java, which does not implement Input/Output stream.

What you want to do instead is just lock on a lock file that guards the file you are read or writing. Here's the updated algorithm:

WRITER:
  lock
  write temp file
  create copying flag
  copy temp to file
  delete copying flag
  delete temp
  release

READER:
  lock
  if copying flag exists
    copy temp to file
    delete copying flag
    delete temp 
  end
  read file
  release

lock and release guards the file, the temp file and the copying flag. The only problem is now the reader lock can't be shared, but it never could be really. The reader always had a chance to modify the file, therefore it would have been wrong to make a shareable lock in the first place.

Answer 1

I don't think there is a perfect answer. I don't exactly know what you need to do, but can you do the writing to a new file, and then on success, rename files, rather than copying. Renaming is quick, and hence should be less prone a crash. This still won't help if it fails at the rename stage, but you've minimized the window of risk.

Again, I'm not sure if it is applicable or relevant to your needs, but can you write some file block at the end of the file to show all the data has been written?

Answer 2

I assume you have a large file which you are continously appending to. Crashes of the VM do not happen very often. But if they happens, you need a way to roll back the failed changes. You just need a way to know how far to roll back. For examply by writing the last file length to a new file:

WRITER:
  lock file
  write file position to pos-file
  write file
  remove pos-file
  unlock file

If the writer crashes one of your readers will get the read lock. They have to check for the pos-file. If they find one a crash occured. If they look inside the file they know how far to roll back the changes the get a consistent file again. Of course the roll back procedure has to happen in a similar way like the write procedure.

When you are not appending but replacing the file, you can use the same method:

WRITER:
  lock file
  write writing-in-progress-file
  write file
  remove writing-in-progress-file
  unlock file

Same rules as previously apply for the reader. When the writing-in-progress-file exists but the reader already acquired the read lock the written file is in a insonsistent state.