Are files dumped with fwrite portable across different systems?

Question

Can I assume a file generated with fwrite and read using fread is portable across different systems. 32bit/64bit windows,osx,linux.

//dumping
FILE *of =fopen("dumped.bin","w");
double *var=new double[10];
fwrite(var, sizeof(double), 10,FILE);
//reading
file *fo=fopen()
double *var=new double[10];
fread(var,sizeof(double),10,of);

And what about structs

struct mat_t{
    size_t x;
    size_t y;
    double **matrix;
}

Are these portable?

Answer 1

This file will definitely not be portable. That's why things like htons exist. You have to make sure your file format very clearly specifies size and endianness. Since the individual types aren't portable, structs aren't either (not even considering struct packing).

Answer 2

fwrite and fread are plenty portable, but you will have problems with things like sizeof(double), which may vary across systems. Make sure that every binary field you write has a defined size which isn't dependent on compiler or OS--you can largely accomplish this by using the types that explicitly give their size, such as uint32_t.

You also have to worry about endianness, but there are macros ntoh, ntohl, hton, and htonl that you can use to swap endianness for you, and they should be defined to work correctly for whatever system you compile them on.

Answer 3

Short Answer: NO

Long Answer:

You are writing out the binary representation of the data.
This is not portable across platforms or OS or even compilers.

All the objects you have written have things that can change:

int:        size and endianess.
double:     size and representation.
structure:  Each member has to be looked at individually.
            The structure itself may be padded different on different compilers.
            Or even the same compiler with different flags.
pointers:   Are meaningless even across processes on the same machine.
            All pointers have to be converted into something meaningful like
            a named object that is provided separately. The transport will then
            have to convert named objects into pointers at the transport layer
            at the destination.

You have two main options:

• Streaming the data.
  This is basically converting the structure into a textual representation and sending the string. For small object, API structures this is current standard method of doing cross platform/language communication (though the data is usually wrapped in some format like XML or Json).
• Convert to a network neutral binary format
  For this we have htonl() and family of functions() for converting integers. Doubles are harder and usually converted into two integers (the values depend on accuracy requirements). Strings are converted to a length followed by a sequence of characters etc. Then each is written individually to the stream. This can be much more compact than streaming (and thus efficient). The down side is that you are tightly coupling both ends to a very specific format and thus makes the solution particularly brittle and harder to correct in error situations.

Answer 4

You're forgetting Windows assumes file writes are in text mode, so you need to change the "w" to "wb" in the second parameter to fwrite()

Answer 5

If you are writing structures, and they are not self-contained (meaning they have pointers, sub-structures), you are much better off using some kind of serialization library. There is the Boost Serilization library, which takes an object-oriented approach. The output can be easily switched from text to binary, which makes debugging a lot easier.

There is also HDF5 (Hierarchical Data Format), a set of libraries that is routinely used in scientific computing on some unusual hardware (not your standard x86). I've never used this, so I can't speak to how easy it is to use, but will handle huge data sets (multiple TB).