How to empty/flush Windows READ disk cache in C#?

Question

If I am trying to determine the read speed of a drive, I can code a routine to write files to a filesystem and then read those files back. Unfortunately, this doesn't give an accurate read speed because Windows does disk read caching.

Is there a way to flush the disk read cache of a drive in C# / .Net (or perhaps with Win32 API calls) so that I can read the files directly from the drive without them being cached?

Answer 1

I found this article and it seems that this is a complicated program because you also have to flush other caches.

Answer 2

Why DIY?

If you only need to determine drive speed and not really interested in learning how to flush I/O buffers from .NET, you may just use DiskSpd utility from http://research.microsoft.com/barc/Sequential_IO/. It has random/sequential modes with and without buffer flushing.

The page also has some I/O related research reports you might find useful.

Answer 3

Constantin: Thanks! That link has a command-line EXE which does the testing I was looking for.

I also found a link off that page to a more interesting article (in Word and PDF) on this page: Sequential File Programming Patterns and Performance with .NET

In this article, it talks about Un-buffered File Performance (iow, no read/write caching -- just raw disk performance.)

Quoted directly from the article:

There is no simple way to disable FileStream buffering in the V2 .NET framework. One must invoke the Windows file system directly to obtain an un-buffered file handle and then ‘wrap’ the result in a FileStream as follows in C#:

    [DllImport("kernel32", SetLastError=true)]
    static extern unsafe SafeFileHandle CreateFile(
        string FileName,           // file name
        uint DesiredAccess,        // access mode
        uint ShareMode,            // share mode
        IntPtr SecurityAttributes, // Security Attr
        uint CreationDisposition,  // how to create
        uint FlagsAndAttributes,   // file attributes
        SafeFileHandle  hTemplate // template file  
        );

    SafeFileHandle handle = CreateFile(FileName,
                            FileAccess.Read,
                            FileShare.None,
                            IntPtr.Zero,
                            FileMode.Open,
                             FILE_FLAG_NO_BUFFERING,
                            null);

    FileStream stream = new FileStream(handle, 
        FileAccess.Read, 
        true, 
        4096);

Calling CreateFile() with the FILE_FLAG_NO_BUFFERING flag tells the file system to bypass all software memory caching for the file. The ‘true’ value passed as the third argument to the FileStream constructor indicates that the stream should take ownership of the file handle, meaning that the file handle will automatically be closed when the stream is closed. After this hocus-pocus, the un-buffered file stream is read and written in the same way as any other.

Answer 4

const int FILE_FLAG_NO_BUFFERING = 0x20000000;
return new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read,64 * 1024,
(FileOptions)FILE_FLAG_NO_BUFFERING & FileOptions.Asynchronous
& FileOptions.SequentialScan);

Answer 5

Response of Fix was almost right and better than PInvoke. But it has bugs and doesn't works...

To open up File w/o caching one needs to do following:

const FileOptions FileFlagNoBuffering = (FileOptions)0x20000000;

FileStream file = new FileStream(fileName, fileMode, fileAccess, fileShare, blockSize,
    FileFlagNoBuffering | FileOptions.WriteThrough | fileOptions);

---

Few rules:

1. blockSize must be hard drive cluster size aligned (4096 most of the time)
2. file position change must be cluster size aligned
3. you can't read/write less than blockSize or block not aligned to it's size

And don't forget - there is also HDD Cache (which slower and smaller than OS cache) which you can't turn off by that (but sometimes FileOptions.WriteThrough helps for not caching writes). With those options you have no reason for flushing, but make sure you've properly tested that this approach won't slow things down in case your implementation of cache is slower.