Writing files to USB stick causes file corruption/lockup on surprise removal

Question

I'm writing a background application to copy files in a loop to a USB stick with the "Optimize for quick removal" policy set. However, if the stick is removed part way through this process (specifically in the WriteFile() call below, which returns ERROR FILE NOT FOUND) the application hangs, the drive is then permanently inaccessible from any other application and the PC cannot be shutdown/logged off/restarted etc. All running instances of Windows Explorer also hang as a result.

I have traced the issue to the CloseHandle() call made after the stick is removed and the above error occurs. Its almost as if CloseHandle() is blocking indefinitely in the driver somewhere because the stick is no longer there? Anyway, I have managed to get past this issue by simply skipping the CloseHandle() call if WriteFile() returns ERROR FILE NOT FOUND. However, this leads to another problem where, every so often, a file gets irrecoverably corrupted and the only way to fix it is using chkdsk, or reformat the stick.

Note that this only happens on XP (SP2 and 3), Vista does not seem to suffer from the issue. A snippet of the code follows:

HANDLE hFile = CreateFile(szFile, 
                          GENERIC_WRITE, 
                          FILE_SHARE_WRITE | FILE_SHARE_READ | FILE_SHARE_DELETE,
                          NULL,
                          CREATE_ALWAYS,
                          FILE_FLAG_WRITE_THROUGH,
                          NULL);
if (hFile != INVALID_HANDLE_VALUE)
{
    if (!WriteFile(hFile, pBuffer, dwBufferSize, &dwWritten))
    {
        int nLastError = GetLastError();
    }

    // If usb stick is removed during WriteFile(), ERROR_FILE_NOT_FOUND usually      results.
    // If handle is closed at this point then drive is inaccessible.
    // If CloseHandle() is skipped, then file corruption occurs instead
    if (nLastError != ERROR_FILE_NOT_FOUND)
    {
        CloseHandle(hFile);
    }
}

I've tried pretty much every combination of flags for CreateFile() all to no avail. Has anybody seen this before or have any good ideas how to avoid either of the two problems occuring. Is what I'm seeing a driver problem which has been silently fixed under vista?

Thanks for any help.

Answer 1

About the hanging problem: You could spawn a separate thread for writing, supervise the writing process from the main thread and abort the writing thread if it takes suspiciously long. In other words: Implement the writing asynchronously and look for a timeout.

Answer 2

This seems to be a driver problem.

You must free all handles to a driver, to let it cleanup itself, and windows to unload it. When you don't do that, the driver thinks it is still responsible for the device, even though it changed.

You cannot escape this problem in User-Mode.

Abandoning the handle just transfers the problem to a later stage (e.g. Quitting your program, so that windows tries to close all your abandoned open handles.)

Answer 3

Its almost as if CloseHandle() is blocking indefinitely in the driver somewhere because the stick is no longer there?

Sounds reasonable. CloseHandle() will ultimately emit a file system IRP and you're not using non-blocking I/O, so that IRP will be synchronous, but it looks like where the actual file system has abruptly disappeared from underneath the file system driver, that IRP is never completed. Which means you're stuffed - the user-mode function call which lead to the file system IRP being issued will never return.

Try using non-blocking I/O - that will prolly get you around this problem, at least from the point of view of not hanging. You will still be experiencing resource loss and the like, since the IRP will still be passed down and almost certainly still won't be coming back up, but at least you won't be blocking on it.

BTW, "optimize for quick removal" is I would say designed to reduce the amount of caching that goes on and perhaps influence the order of writes to the file system to reduce the chance of corruption; I extremely doubt it is intended to preserve the file system in the event of file system departure during a write!

You should not be surprised that this kills the file system.

Answer 4

Could it be a problem with the filesystem driver rather than the hardware driver? You might find that if you use NTFS the problem goes away.