How do you write the end of a file opened with FILE_FLAG_NO_BUFFERING?

Question

I am using VB6 and the Win32 API to write data to a file, this functionality is for the export of data, therefore write performance to the disk is the key factor in my considerations. As such I am using the FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH options when opening the file with a call to CreateFile.

FILE_FLAG_NO_BUFFERING requires that I use my own buffer and write data to the file in multiples of the disk's sector size, this is no problem generally, apart from the last part of data, which if it is not an exact multiple of the sector size will include character zero's padding out the file, how do I set the file size once the last block is written to not include these character zero's?

I can use SetEndOfFile however this requires me to close the file and re-open it without using FILE_FLAG_NO_BUFFERING. I have seen someone talk about NtSetInformationFile however I cannot find how to use and declare this in VB6. SetFileInformationByHandle can do exactly what I want however it is only available in Windows Vista, my application needs to be compatible with previous versions of Windows.

Answer 1

I'm not sure, but are YOU sure that setting FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH give you maximum performance?

They'll certainly result in your data hitting the disk as soon as possible, but that sort of thing doesn't actually help performance - it just helps reliability for things like journal files that you want to be as complete as possible in the event of a crash.

For a data export routine like you describe, allowing the operating system to buffer your data will probably result in BETTER performance, since the writes will be scheduled in line with other disk activity, rather than forcing the disk to jump back to your file every write.

Why don't you benchmark your code without those options? Leave in the 0-byte padding logic to make it a fair test.

If it turns out that skipping those options is faster, then you can remove the 0-padding logic, and your file size issue fixes itself.

Answer 2

I believe SetEndOfFile is the only way.

And I agree with Mike G. that you should bench your code with and without FILE_FLAG_NO_BUFFERING. Windows file buffering on modern OS's is pretty darn effective.

Answer 3

Well I am amazed! Using Windows buffering instead of doing it all myself is MUCH faster. I have been writing out a 1Gb file to test with, using my own buffer and the FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH options took on average 21.146 seconds, without those settings and using the Windows buffer the average time dropped to 13.53 seconds, that's over 30% faster!

Note to self: No need to re-invent the wheel. ;-)

Thank you 'Mike G' for your swift and accurate answer. And thanks also to you 'gabr', no need to bother with SetEndOfFile at all now.