Fopen failing for binary file

Question

I have a huge binary file which is 2148181087 bytes (> 2gb)

I am trying to do fopen (file, "r") and it failed with

Can not open: xyz file (Value too large to be stored in data type)

I read on the man page EOVERFLOW error is received when the file size > 2gb.

The weird thing is, I use a different input file which is also "almost" as big as the first file 2142884400 bytes (also >2gb), fopen works fine with this.

Is there any cutoff on the file size for fopen or is there any alternate way to solve this?

Answer 1

The cutoff is 2GB which, contrary to what you may think, is not 2,000,000,000 (2x1000³).

It's 2,147,483,648 (2x1024³). So your second file, which works, is actually less than 2GB in size).

2GB, in the computer world, is only 2,000,000,000 in the minds of hard drive manufacturers so they can say their disks are bigger than they really are :-) - it lets them say their disks are actually 2.1GB.

Answer 2

The "alternative way to solve this" depends on which operating system/library you are using.

For the GNU C library, you can use fopen64 as a replacement for fopen; it uses 64-bit file handles (there's also a macro to have fopen use 64-bit file handles).

For Windows, you'll probably have to switch to the Win32 file management API, with which you can use CreateFile.