Improving File Read Performance (single file, C++, Windows)

Question

I have large (hundreds of MB or more) files that I need to read blocks from using C++ on Windows. Currently the relevant functions are:

errorType LargeFile::read( void* data_out, __int64 start_position, __int64 size_bytes ) const
{
 if( !m_open ) {
  // return error
 }
 else {
  seekPosition( start_position );
  DWORD bytes_read;
  BOOL result = ReadFile( m_file, data_out, DWORD( size_bytes ), &bytes_read, NULL );
  if( size_bytes != bytes_read || result != TRUE ) {
   // return error
  }
 }
 // return no error
}

void LargeFile::seekPosition( __int64 position ) const
{
 LARGE_INTEGER target;
 target.QuadPart = LONGLONG( position );
 SetFilePointerEx( m_file, target, NULL, FILE_BEGIN );
}

The performance of the above does not seem to be very good. Reads are on 4K blocks of the file. Some reads are coherent, most are not.

A couple questions: Is there a good way to profile the reads? What things might improve the performance? For example, would sector-aligning the data be useful? I'm relatively new to file i/o optimization, so suggestions or pointers to articles/tutorials would be helpful.

Answer 1

It's not at all clear what you mean by "coherent" here.

In any case, you can start by considering how you're really using data from the file. If you're mostly reading through a file in consecutive blocks from beginning to end, you might benefit from passing FILE_FLAG_SEQUENTIAL_SCAN when you call CreateFile. Conversely, if you mostly using one block from one place, and then most likely not using another block that's (nearly) consecutive with that, you might benefit from passing FILE_FLAG_RANDOM_ACCESS instead. If you're reasonably certain that after you read one block of data, you will not be using that same block of data again any time soon, you might benefit from using FILE_FLAG_NO_BUFFERING.

Another possibility is to simply read bigger chunks at a time, if you can make use of the extra data. Reading only 4K at a time will tend to impose a fair amount of overhead.

Finally, if you can disconnect the processing from the reading itself, you might want to consider using asynchronous reads. The basic idea is that you can tell it to read some data, and when the data has been read you can process the block of data -- but in between, you can do other things -- like issue more reads, so the disk will almost always stay busy reading data.

Answer 2

The performance of the above does not seem to be very good. Reads are on 4K blocks of the file. Some reads are coherent, most are not.
If the reads aren't coherent, performance is going to be poor. It might be a good idea to consider memory mapped files or using more coherent blocks if possible.

Is there a good way to profile the reads?
ProcMon will give you a good look at the reads themselves. I'm unsure exactly what kind of information you're looking for though.