Object to Network serialization - with an existing protocol

Question

I'm writing a client for a server program written in C++. As is not unusual, all the networking protocol is in a format where packets can be easily memcopied into/out of a C++ structure (1 byte packet code, then different arrangements per packet type).

I could do the same thing in C#, but is there an easier way, especially considering lots of the data is fixed-length char arrays that I want to play with as strings? Or should I just suck it up and convert types as needed? I've looked at using the ISerializable interface, but it doesnt look as low level as is required.

Answer 1

I wrote a post on this in 2004 which covers some of the options available for converting a binary stream to a .NET memory structure. I reposted it on my new blog since the old blog site no longer exists.

http://taylorza.blogspot.com/2010/04/archive-structure-from-binary-data.html

Basically you have three options

1. Use C++ style memory pointers in C# which requires the /unsafe switch
2. Use the .NET marshaling to allocate an unmanaged block of memory, copy the bytes to the unmanaged memory and then use Marshal.PtrToStructure to marshal the data back to the managed heap mapping it into your structure.
3. Use a BinaryReader to manualy read the byte stream and pack the data into the structure. Personally, this was my preferred option.

When you consider the options you should also take into account how the byte ordering might affect you.

As an example I will use the IP header as an example, since at the time of the post I was working with Raw TCP packets.

You need to define your .NET structure that the binary data will be mapped to. For example the IP Header looks like the following.

[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct IpHeader
{  
  public byte VerLen;  
  public byte TOS;  
  public short TotalLength;   
  public short ID;  
  public short Offset;  
  public byte TTL;  
  public byte Protocol;  
  public short Checksum;  
  public int SrcAddr;  
  public int DestAddr;
}

Note that the StructLayout attribute is only required for first two options, and of course you will need to set the packing as appropriate for the structure that is being serialized from the server.

So in C/C++, given a pointer to a block of memory that contains the data bytes that map to the C/C++ structure you can use the following bit of code to view the block of data as a structure piece of memory, where packet is a byte* to the memory.

IpHeader *pHeader = (IpHeader*)packet;

Doing the same is C# using /unsafe option and the struct defined above you count use the following code.

IpHeader iphdr;
unsafe
{  
  fixed ( byte *pData = packet)  
  {    
    iphdr = *(IpHeader*)pData;  
  }
}
//Use iphdr...

The marshaling option would look like the following

IntPtr pIP = Marshal.AllocHGlobal( len );
Marshal.Copy( packet, 0, pIP, len );
iphdr = (IpHeader)Marshal.PtrToStructure( pIP, typeof(IpHeader) );
Marshal.FreeHGlobal( pIP );

And finally you can use the BinaryReader to do this entirely in managed code.

MemoryStream stm = new MemoryStream( packet, 0, len );
BinaryReader rdr = new BinaryReader( stm );

iphdr.VerLen = rdr.ReadByte();
iphdr.TOS = rdr.ReadByte();
iphdr.TotalLength = rdr.ReadInt16();
iphdr.ID = rdr.ReadInt16();
iphdr.Offset = rdr.ReadInt16();
iphdr.TTL = rdr.ReadByte();
iphdr.Protocol = rdr.ReadByte();
iphdr.Checksum = rdr.ReadInt16();
iphdr.SrcAddr = rdr.ReadInt32();
iphdr.DestAddr = rdr.ReadInt32();

As I mentioned earlier, you might need to consider byte ordering. For example, the above code is not quite correct because the IpHeader does not use the same byte order as is assumed by ReadInt16. ReadInt32 etc. Resolving the issue with the above solution is as simple as using IPAddress.NetworkToHostOrder.

iphdr.VerLen = rdr.ReadByte();
iphdr.TOS = rdr.ReadByte();
iphdr.TotalLength = IPAddress.NetworkToHostOrder(rdr.ReadInt16());
iphdr.ID = IPAddress.NetworkToHostOrder(rdr.ReadInt16());
iphdr.Offset = IPAddress.NetworkToHostOrder(rdr.ReadInt16());
iphdr.TTL = rdr.ReadByte();
iphdr.Protocol = rdr.ReadByte();
iphdr.Checksum = IPAddress.NetworkToHostOrder(rdr.ReadInt16());
iphdr.SrcAddr = IPAddress.NetworkToHostOrder(rdr.ReadInt32());
iphdr.DestAddr = IPAddress.NetworkToHostOrder(rdr.ReadInt32());