Tips on building a byte protocol

Question

I'm communicating data between devices, and I have to program the protocol as an array of bytes.

Any tips when building protocols at a low-level? .. Eg:

• Use a 2 byte header, to send the length of the message before the data bytes.
• Use a CRC/data validation scheme. (How do I do this? Any simple checksums?)

Answer 1

It depends on the QoS (Quality Of Service) characteristics of the underlying transport layer.

If the underlying channel is reliable, then a CRC is probably overkill (assuming some form of integrity checking is done at the lower protocol layer).

If you are asking about how to delineate your payload from a byte stream, then there are several possibilities one of which might be just to BASE64 encode/decode your stream. Then again, depending on your requirement, BASE64 might translate to too much overhead.

Of course you can always use a HEADER (Unique Sequence+payload length+CRC) with a low probability of occurence in your payload but then you need to apply a scrambler over your payload to minimize the probability of duplicating your HEADER etc.

---

If you are looking to build a protocol for an unreliable byte-stream oriented procotol, then why reinvent the wheel? Why not use something like PPP?

Answer 2

Think carefully whether you can have a human readable protocol
Then consider if you can use compression rather than raw bianry

Answer 3

Significant part depends on what is provided on lower layers. Here are examples to explain what an why I count significant:

• I once took part in ITU H.223 protocol implementation. The worst thing there was data stream could be even byte or bit stream based and lower layers provided NOTHING beside raw bits. Protocol had several possible layers in dependence on transport reliability and functionality requirements for upper layers.

• Another example is ITU H.225 protocol over TCP where transport was reliable but it was stream of bytes. So it MUST be good message delimitation logics implemented.

• Well, UDP based SIP. Datagrams. Very useful. But lot of troubles there were related to messaging reliability. So sequencing and request / response tracking (transaction, leg ...) was very significant.

• OK, for some reasons we used RPC protocol. Good thing if you are lazy. Some limitations related to applications logics but I'd recommend to look on this even just to learn.

• NASA IPC was thing I looked on not so long time ago. Good, very simple related to upper layers. But it is centralized what is limitation.

Another key point is: You should design protocol looking at first on your needs (analyze upper layer requirements). What would you given with knowledge CRC-32 is best way to protect data integrity over analyze if you really need to do this (as example)?

Well, I think these mentions might help you think about subject with slightly different point of view.

Answer 4

1. Think wisely and about all the cases before setting the structure.
2. Make the header a little bigger, even if sending zero bytes in them.
3. Split the header into few parts. This of course depends on your requirements, for example a Control Byte, Message Length Byte, Format Byte ...etc

About the checksum, depends on the underlying protocol. But you can implement one yourself. Encrypt, Hash, Crunch, Flip, 2s Complement the message and store the result in one Check byte