error detection/correction/recovery in serial protocols

Question

I have some designing to do for a serial protocol and am running into some questions that I figure must have been considered elsewhere.

So I'm wondering if there are some recommendations for best practices in designing serial protocols. (Please either state a fact that is easily verifiable, or cite a reputable source if you make a claim.) General recommendations for websites/books are also welcome.

In particular I have to deal with issues like

• parsing a stream of bytes into packets
• verifying a packet is correct (easy with a CRC, for instance)
• identifying reasonable types of errors that can occur (e.g. in a point-to-point serial stream, sporadic single bit errors, and dropped series of bytes, are both likely, but extra phantom bytes are unlikely; whereas with a record stored in flash memory or on a disk drive the types of errors that predominate are different)
• error correction or recovery (if I detect an error in a packet, can I correct it? If not, can I resync to the boundary of the next packet?)
• how to make variable-length packets robust to error correction / recovery.

Any suggestions?

Answer 1

Have you considered FEC (Forward Error Correction)?

This procedure is very often used in "physical" level communication protocols such as WDM (Wavelength Division Multiplexing) / OTN (Optical Transport Network).

Answer 2

Packet delimiting

For syncing to packet boundaries, typically you have a byte or byte sequence that identifies the packet boundary, which cannot occur within the packet itself. If the packet data happens to contain that identifier, then you have to "escape" (aka byte stuff) it.

Examples:

• PPP Encapsulation
• Consistent Overhead Byte Stuffing (COBS), or maybe COBS/R, which encodes data packets so no zero bytes are present, thus you can use zero bytes for packet delimiting

Packet verification

Various options are:

• Checksum
  • Adler-32
  • Fletcher
• CRC (the more bits the better the check)

Error correction etc

Good questions. I've not had much experience with that.