Protocol/Packet Design Questions

Question

I'm looking to a design a protocol for a client-server application and need some links to some resources that may help me.

The big part is I'm trying to create my own "packet" format so I can minimize the amount of information being sent. I'm looking for some resources to dissect their protocol, but it seems some completely lack packet design, such as SMTP (which just sends strings terminated by CLRF). What are the advantages/disadvantages of using a system like SMTP over a system that uses a custom made packet? Couldn't SMTP use only a couple bytes to cover all commands through bit flags and save bandwidth/space?

Just trying to get my head around all this.

Answer 1

SMTP, HTTP and other TCP based protocols do not concern themselves with packet design because they are stream based. So it makes more sense to talk about protocol design.

As for why use text based protocols vs binary protocols...

Readability of the protocol by packet sniffing programs like Wireshark is very useful.

Also it is often very useful to be able to simply telnet into your port and be able to communicate with the server by specifying plain text.

Also with a protocol like HTTP the actual resource is usually the payload of the communication, the resource can be in binary or any other specified format. So having just the headers and status in plain text is not a bad thing.

Answer 2

• TCP is a stream based protocol, not packet based.
• Using text with lines makes ad hoc debugging a lot easier
• Using text with lines makes it possible to exercise your protocol with telnet

Answer 3

True, but SMTP wasn't particularly optimized for space, nor is it a packet-based protocol. It sits atop TCP, and uses the stream functionality of TCP. You need to decide what is desirable in your protocol: is it performance sensitive? latency? bandwidth?

Is it going to need to run as superuser? If not, you'll probably want to use UDP or TCP.

Are you going to need guarantees on delivery? If so, TCP is probably your best option, unless you are dealing with fairly extreme performance or size issues.

Few protocols these days design individual packets, though many do send very specific data structures across the wire using TCP, or, less commonly, UDP.

If you want to really optimize for space or bandwidth, consider condensing your data as much as possible into individual bits and byte, and defining and packing structures to send it across TCP. Modern network adapters are so optimized for TCP anyway, that there is often little advantage to other strategies.

Answer 4

First of all, are you about to implement an enhanced transport protocol (like RTP on top of UDP) or an application protocol (like HTTP/SMTP)?

There are several things you should think about in both cases concerning your design of the protocol or the demands of your application: Stream based or packet based, unidirectional / bi-directional, stateful and sessionful or stateles, reliable or best effort, timing demands, flow/congestion control, secure or plain.

Towards an application layer protocol, you should also think about: Textual or binary data, mapping of application data to network data units/packets, security demands and integrity, etc.