Any way to read big endian data with little endian program?

Question

An external group provides me with a file written on a Big Endian machine, and they also provide a C++ parser for the file format.

I only can run the parser on a little endian machine - is there any way to read the file using their parser without add a swapbytes() call after each read?

Answer 1

In general, there's no "easy" solution to this. You will have to modify the parser to swap the bytes of each and every integer read from the file.

Answer 2

It depends upon what you are doing with the data. If you are going to print the data out, you need to swap the bytes on all the numbers. If you are looking through the file for one or more values, it may be faster to byte swap your comparison value.

In general, Greg is correct, you'll have to do it the hard way.

Answer 3

You could write a parser that wraps their parser and reverses the bytes, if you don't want to modify their parser.

Be conscious of the types of data being read in. A 4-byte int or float would need endian correction. A 4-byte ASCII string would not.

Answer 4

In general, no.

If the read/write calls are not type aware (which, for example fread and fwrite are not) then they can't tell the difference between writing endian sensitive data and endian insensitive data.

Depending on how the parser is structured you may be able to avoid some suffering, if the I/O functions they use are aware of the types being read/written then you could modify those routines apply the correct endian conversions.

If you do have to modify all the read/write calls then creating just such a routine would be a sensible course of action.

Answer 5

the best approach is to just define the endianess in the file format, and not say it's machine dependent. the writer will have to write the bytes in the correct order regardless of the CPU it's running on, and the reader will have to do the same.

Answer 6

Try persuading the parser team to include the following code:

int getInt(char* bytes, int num)
{
    int ret;
    assert(num == 4);
    ret = bytes[0] << 24;
    ret |= bytes[1] << 16;
    ret |= bytes[2] << 8;
    ret |= bytes[3];
    return ret;
}

it might be more time consuming than a general int i = *(reinterpret_cast<*int>(&myCharArray)); but will always get the endianness right on both big and small endian systems.

Answer 7

Back in the early Iron Age, the Ancients encountered this issue when they tried to network primitive PDP-11 minicomputers with other primitive computers. The PDP-11 was the first little-Endian computer, while most others at the time were big-Endian.

To solve the problem, once and for all, they developed the network byte order concept (always big-Endia), and the corresponding network byte order macros ntohs(), ntohl(), htons(), and htonl(). Code written with those macros will always "get the right answer".

Lean on your external supplier to use the macros in their code, and the file they supply you will always be big-Endian, even if they switch to a little-Endian machine. Rewrite the parser they gave you to use the macros, and you will always be able to read their file, even if you switch to a big-Endian machine.

A truly prodigious amount of programmer time has been wasted on this particular problem. There are days when I think a good argument could be made for hanging the PDP-11 designer who made the little-Endian feature decision.

Answer 8

Your question somehow conatins the answer: No!

I only can run the parser on a little endian machine - is there any way to read the file using their parser without add a swapbytes() call after each read?

If you read (and want to interpret) big endian data on a little endian machine, you must somehow and somewhere convert the data. You might do this after each read or after the whole file has been read (if the data read does not contain any information on how to read further data) - but there is no way in omitting the conversion.