Serialize a C# class to binary be used by C++. How to handle alignment?

Question

I am currently serializing a C# class into a binary stream using BinaryWriter.

I take each element of the class and write it out using BinaryWriter. This worked fine as the C++ application reading this binary file supported packed structs and hence the binary file could be loaded directly.

Now I have got a request to handle alignment as a new application has popped up which cannot support packed structs. What's the best way to convert the C# class and exporting it out as a binary keeping both 2 byte as well as 4 byte alignment in mind?

The user can choose the alignment.

Answer 1

You might want to consider not overlaying structs onto memory buffers. Just read the bytes in a deserialization step instead. Something like:

struct A {
    uint8_t  aByte;
    uint32_t aDWord;
    uint16_t aWord;
};

void serialize(FILE* fp, A const& aStruct) {
    fwrite(&aStruct.aByte,  sizeof(aStruct.aByte),  1, fp);
    fwrite(&aStruct.aDWord, sizeof(aStruct.aDWord), 1, fp);
    fwrite(&aStruct.aWord,  sizeof(aStruct.aWord),  1, fp);
}

void deserialize(FILE* fp, A& aStruct) {
    fread(&aStruct.aByte,  sizeof(aStruct.aByte),  1, fp);
    fread(&aStruct.aDWord, sizeof(aStruct.aDWord), 1, fp);
    fread(&aStruct.aWord,  sizeof(aStruct.aWord),  1, fp);
}

instead of:

void serialise(FILE* fp, A const& aStruct) {
    fwrite(&aStruct, sizeof(aStruct), 1, fp);
}

void deserialise(FILE* fp, A& aStruct) {
    fread(&aStruct, sizeof(aStruct), 1, fp);
}

The first example isn't dependent on structure packing rules where the second example is. I would recommend using the one that isn't. Most languages (not sure about C#) give you some way to read and write raw bytes so do all of the serialization/deserialization memberwise instead of a singular memory block and the packing/padding problems go away.

Answer 2

In serializations of objects in any language alignment should not be an issue, as you know ahead of time how much data is being written and read.

For example, take the following struct:

struct data
{
   char c;
   unsigned int i;
   double d;
}

Depending on how the compiler does memory layout, the size of this struct in memory can be anywhere from 13 to 20 (packed - 32-bit alignment), but that's the memory layout. As far as disk layout is concerned, you are (assuming binary) always going to be writing:

1. write 1 byte for c
2. write 4 bytes for i
3. write 8 bytes for d

Hence when the other side reads it in, be it either Python or C# or whatever else should do the following:

1. read 1 byte and convert to internal char representation
2. read 4 bytes and convert to an internal unsigned int representation (remember if in java there is no unsigned int)
3. read 8 bytes and convert to an internal real or floating point representation

This is pretty much the canonical solution. You should never rely on mass block writes of structures in any language if portability between languages and architectures is an issue.

Also the above does not take into account endianess, which is something you need to consider when serializing integers - usually as easy as converting to network byte order and then back, for writing and reading respectively.

Answer 3

Consider using a C++ library on the writer side. If the reader side is constrained, create structures with the very same memory layout in unmanaged C++ and dump them to the binary file. You probably have to use #pragma pack(1) to disable the default packing and put some char[] dummies between your structure elements to resemble the alignment on the reader side. Don't forget about endianness. Then just call the writer library using DllImport or via C++/CLI (it's a matter of taste).