Normal file related functions like fseek, ftell etc in Windows MSVC6 can handle files only upto 2GB (As per my current understanding, Please correct me if I am wrong).
I want to work with files >2GB. How should I go about it? What are the functions available?
I am not sure but the limit is 4 GB, OS API and the standard libraries using theses API and the filesystem used.
The ftell, fseek functions are using 32 bit integers so you won't be able to handle file bigger than 4GB. You would have to use the OS API directly.
So you have to be careful what function you use, for example for getting the file size you have to use the ex function GetFileSizeEx, so you have to make sure you use function that use 64 bit file offset. Same for SetFilePointerEx
Last word you have be aware that some filesystem limit the maximum filesize, the FAT32 won't handle file bigger than 4 GB by design, NTFS would handle any size but the API generally is made for 4 GB or less big file.
The limit probably originates with the file system. FAT32 has a limit of 4GB, whereas NTFS has a much higher limit (in the terabytes).
So the size of the file you can handle will depend on which file system the hard disk has been formatted with, and which operating system you are using, although you are almost certainly using an operating system which can handle the upper limits of NTFS (Windows 2000 or above).
For the most part, you have to ignore all the file functions built into the standard library, and use only functions in the Win32 API -- e.g. instead of fwrite or ostream::write, you'll need to use WriteFile. Likewise, to seek in a file you'll need to use SetFilePointer instead of fseek or seekp. Most of the Win32 API can handle files larger than 4 GB -- and the few that can't have substitutes that can handle larger files.
You can use windows APIs for file handling like CreateFile, ReadFile, WriteFile. This also gives you option to have overlapped and non-overlapped operations.
It's actually 16TB (for anyone finding this in future). I just created a 6710886400 that's 6GB file using overlapped I/O - the following snippet shows how to work with the offsets OVERLAPPED ol; __int64 fileOffset; ol.hEvent = CreateEvent(0, TRUE, FALSE, 0); fileOffset = __int64(TEST_BUFFER_SIZE) * i; ol.Offset = (DWORD)fileOffset; ol.OffsetHigh = (DWORD)(fileOffset >> 32); printf("[%d %I64d] ", i, fileOffset); result = WriteFile(hFile, buffer, TEST_BUFFER_SIZE, &written, &ol);
to get the size I can perform...
DWORD dwHigh, dwLow =GetFileSize(hFile, &dwHigh);
__int64 FileSizeInBytes = __int64(dwHigh * (MAXDWORD + 1.0L)) + dwLow;
HINT: If you start getting "invalid parameter" return codes/error from an API, you are probably mucking the math and passing in negative offsets.
(some innocent variables and exception handler policing action was removed from this sample to protect basic byte rights)