File Unlocking and Deleting as single operation

Question

Hello,

Please note this is not duplicate of http://stackoverflow.com/questions/1910624/file-r-w-locking-and-unlink. (The difference - platform. Operations of files like locking and deletion have totally different semantics, thus the sultion would be different).

I have following problem. I want to create a file system based session storage where each session data is stored in simple file named with session ids.

I want following API: write(sid,data,timeout), read(sid,data,timeout), remove(sid) where sid==file name, Also I want to have some kind of GC that may remove all timed-out sessions.

Quite simple task if you work with single process but absolutly not trivial when working with multiple processes or even over shared folders.

The simplest solution I thought about was:

write/read:
   hanlde=CreateFile
   LockFile(handle)
   read/write data
   UnlockFile(handle)
   CloseHanlde(handle)

GC (for each file in directory)
   hanlde=CreateFile
   LockFile(handle)
   check if timeout occured
     DeleteFile
   UnlockFile(handle)
   CloseHanlde(handle)

But AFIAK I can't call DeleteFile on opended locked file (unlike in Unix where file locking is not mandatory and unlink is allowed for opened files.

But if I put DeleteFile outside of Locking loop bad scenario may happen

GC - CreateFile/LockFile/Unlock/CloseHandle,
write - oCreateFile/LockFile/WriteUpdatedData/Unlock/CloseHandle
GC - DeleteFile

Does anybody have an idea how such issue may be solved? Are there any tricks that allow combine file locking and file removal or make operation on file atomic (Win32)?

Notes:

• I don't want to use Database,
• I look for a solution for Win32 API for NT 5.01 and above

Thanks.

Answer 1

You currently have data in the record that allows the GC to determine if the record is timed out. How about extending that housekeeping info with a "TooLateWeAlreadyTimedItOut" flag.

 GC sets TooLateWeAlreadyTimedItOut = true
 Release lock
    <== writer comes in here, sees the "TooLate" flag and so does not write
 GC deletes

In other words we're using a kind of optimistic locking approach. This does require some additional complexity in the Writer, but now you're not dependent upon any OS-specifc wrinkles.

I'm not clear what happens in the case:

 GC checks timeout
 GC deletes
 Writer attempts write, and finds no file ...

Whatever you have planned for this case can also be used in the "TooLate" case

Edited to add:

You have said that it's valid for this sequence to occur:

 GC Deletes
 (Very slightly later) Writer attempts a write, sees no file, creates a new one

The writer can treat "tooLate" flag as a identical to this case. It just creates a new file, with a different name, use a version number as a trailing part of it's name. Opening a session file the first time requires a directory search, but then you can stash the latest name in the session.

This does assume that there can only be one Writer thread for a given session, or that we can mediate between two Writer threads creating the file, but that must be true for your simple GC/Writer case to work.

Answer 2

For Windows, you can use the FILE_FLAG_DELETE_ON_CLOSE option to CreateFile - that will cause the file to be deleted when you close the handle. But I'm not sure that this satisfies your semantics (because I don't believe you can clear the delete-on-close attribute.

Here's another thought. What about renaming the file before you delete it? You simply can't close the window where the write comes in after you decided to delete the file but what if you rename the file before deleting it? Then when the write comes in it'll see that the session file doesn't exist and recreate it.

The key thing to keep in mind is that you simply can't close the window in question. IMHO there are two solutions:

1. Adding a flag like djna mentioned or
2. Require that a per-session named mutex be acquired which has the unfortunate side effect of serializing writes on the session.

What is the downside of having a TooLate flag? In other words, what goes wrong if you delete the file prematurely? After all your system has to deal with the file not being present...

Answer 3

I don't really understand how this is supposed to work. However, deleting a file that's opened by another process is possible. The process that creates the file has to use the FILE_SHARE_DELETE flag for the dwShareMode argument of CreateProcess(). A subsequent DeleteFile() call will succeed. The file doesn't actually get removed from the file system until the last handle on it is closed.