Random MoveFileEx failures on Vista (access denied, looks like caused by search indexer)

Question

I noticed that writing to a file, closing it and moving it to destination place randomly fails on Vista. Specifically, MoveFileEx() would return ERROR_ACCESS_DENIED for no apparent reason. This happens on Vista SP1 at least (32 bit). Does not happen on XP SP3.

Found this thread on the internets about exactly the same problem, with no real solutions. So far it looks like the error is caused by Vista's search indexer, see below.

The code example given there is enough to reproduce the problem. I'm pasting it here as well:

#include <windows.h>
#include <stdlib.h> 
#include <stdio.h> 

bool test() {
    unsigned char buf[] = {
        0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 
        0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99 
    }; 
    HANDLE h; 
    DWORD nbytes; 
    LPCTSTR fn_tmp = "aaa"; 
    LPCTSTR fn = "bbb"; 
    h = CreateFile(fn_tmp, GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, 0, OPEN_ALWAYS, 0, 0); 
    if (h == INVALID_HANDLE_VALUE) return 0; 
    if (!WriteFile(h, buf, sizeof buf, &nbytes, 0)) goto error; 
    if (!FlushFileBuffers(h)) goto error; 
    if (!CloseHandle(h)) goto error; 
    if (!MoveFileEx(fn_tmp, fn, MOVEFILE_REPLACE_EXISTING | MOVEFILE_COPY_ALLOWED | MOVEFILE_WRITE_THROUGH)) { 
        printf("error=%d\n", GetLastError()); 
        return 0; 
    } 
    return 1; 
error: 
    CloseHandle(h); 
    return 0; 
} 

int main(int argc, char** argv) { 
    unsigned int i; 
    for (i = 0;; ++i) { 
        printf("*%u\n", i); 
        if (!test()) return 1; 
    } 
    return 0; 
}

Build this as console app with Visual Studio. Correct behaviour would be infinite loop that prints test numbers. On Vista SP1, the program exits after random number of iterations (usually before 100 iterations are made).

This does not happen on Windows XP SP2. There's no antivirus running at all; and no other strange background processes (machine is pretty much vanilla OS install + Visual Studio).

Edit: Digging further via Process Monitor (thanks @sixlettervariables), I can't see anything particularly bad. Each test iteration results in 176 disk operations, majority of them coming from SearchProtocolHost.exe (search indexer). If search indexing service is stopped, no errors occur, so it looks like it's the culprit.

At the time of failure (when the app gets ERROR_ACCESS_DENIED), SearchProtocolHost.exe has two CreateFile(s) to the detination file (bbb) open with read/write/delete share modes, so it should be ok. One of the opens is followed by opportunistic lock (FSCTL_REQUEST_FILTER_OPLOCK), maybe that's the cause?

Anyway, I found out that I can avoid the problem by setting FILE_ATTRIBUTE_TEMPORARY and FILE_ATTRIBUTE_NOT_CONTENT_INDEXED flags on the file. It looks like FILE_ATTRIBUTE_NOT_CONTENT_INDEXED is enough by itself, but marking file as temporary also dramatically cuts down disk operations caused by search indexer.

But this is not a real solution. I mean, if an application can't expect to be able to create a file and rename it because some Vista's search indexer is messing with it, it's totally crazy! Should it keep retrying? Yell at the user (which is very undesirable)? Do something else?

Answer 1

I'd say it's either your anti-virus or Windows Indexing messing with the file at the same moment. Can you run the same test without an anti-virus. Then run it again making sure the temp file is created somewhere not indexed by Windows Search?

Answer 2

That usually means something else has an open handle on the file in question, maybe an active virus scanner running? Have you tried running something like Process Monitor from the Sysinternals site? You should be to filter all file operations and get a better picture of whats going on underneath the hood.

Answer 3

I suggest you use Process Monitor (edit: the artist formerly known as FileMon) to watch and see which application exactly is getting in the way. It can show you the entire trace of file system calls made on your machine.

(edit: thanks to @moocha for the change in application)

Answer 4

Windows has a special location for storing application files and I don't think its indexed (at least not by default). In Vista the path is:

C:\Users\*user name*\AppData

I suggest you put your files there if it is appropriate for your application.