Python Sockets - Creating a message format

Question

I have built a Python server to which various clients can connect, and I need to set a predefined series of messages from clients to the server - For example the client passes in a name to the server when it first connects.

I was wondering what the best way to approach this is? How should I build a simple protocol for their communication?

Should the messages start with a specific set of bytes to mark them out as part of this protocol, then contain some sort of message id? Any suggestions or further reading appreciated.

Answer 1

Depending on the requirements, you might want to consider using JSON: use "newline" terminated strings with JSON encoding. The transport protocol could be HTTP: with this, you could have access to all the "connection related" facilities (e.g. status codes) and have JSON encoded payload.

The advantages of using JSON over HTTP:

1. human readable (debugging etc.)
2. libraries available for all languages/platforms
3. cross-platform
4. browser debuggable (to some extent)

Of course, there are many other ways to skin this cat but the time to working prototype using this approach is very low. This is worth considering if your requirements (which aren't very detailed here) can be met.

Answer 2

First, you need to decide whether you want your protocol to be human readable (much more overhead) or binary. If the first, you probably want to use regular expressions to decode the messages. For this, use the python module re. If the latter, the module struct is your friend.

Second, if you are building a protocol that is somehow stateful (e.g. first we do a handshake, then we transfer data, then we check checksums and say goodbye) you probably want to create a some sort of FSM to track the state.

Third, if protocol design is not a familiar subject, read some simple protocol specifications, for example by IETF

If this is not a learning excercise, you might want to build up from something else, like Python Twisted

Answer 3

Read some protocols, and try to find one that looks like what you need. Does it need to be message-oriented or stream-oriented? Does it need request order to be preserved, does it need requests to be paired with responses? Do you need message identifiers? Retries, back-off? Is it an RPC protocol, a message queue protocol?

Answer 4

See http://www.faqs.org/docs/artu/ch05s02.html and http://www.faqs.org/docs/artu/ch05s03.html for a good overview and discussion on data file formats and protocols.