How to write a simple text based protocol, preferably in C

Question

I'm interested in writing a small piece of sample code that communications via standard TCP/IP means to an application server. The idea here is that the client can speak to the application server and be authenticated by simply speaking in a specific text based protocol. The traffic will be encrypted, no username/password. If another app tries to communicate with the app server and doesn't use the correct text based protocol, the app server will just silently discard packets.

Looking to get pointed in the right direction to get this done.

Answer 1

You can look at some of the sample code from TCP/IP Sockets in C.

It has many examples of doing client/server communication in C. Without more details, it's difficult to know what you really want to handle...

Answer 2

You can use a simplified version of TLV (Tag Length Value).

The basic idea is to define a set of message types which are represented by a code of fixed size (the T for Tag). Depending the type of message the contents of it (the V for Value) can very so you specify its length (the L for Length) before the contents. The Length field also has fixed size

Suppose you have one message used to send user data to the server. You can define a message like:

0x01 0x0018 0x11 0x0003 tom 0x12 0x000F [email protected]

Tag: 0x10 User data. Length: 0x0018 Value: sub tags

    Tag 0x11: user name Length: 0x0003 Value = tom
    Tag 0x12: email. Length: 0x000F. Value = [email protected]

Edited:

I was about to forget: Merry Christmas :)

Answer 3

if your not comfortable with the limitited functionality (verbs) provided HTTP just add more verbs. This is what the REST architecture is for.

If you want to continue down your path of folly (your talking about reinventing HTTPS), then use protocol buffers to create a protocol -- it will save you hours of grief.

-- edit --

If your objective is to understand the programming involved with web-servers, you might want to read apache's code dissected by the FMC group into a collection of models. I have read this PDF multiple times -- it is an absolute gold mine.

Answer 4

Take a look at BEEP.

You might also find some good examples at four.livejournal.com; he's gotten good results writing an HTTP parser using the Ragel state machine generator, and also by hand.

Answer 5

For communicating between bespoke apps, you can just send your text format in TCP packets. You can use an extremely simple text format, but you should make sure that it starts with some text that clearly identifies to your server that it is a packet from your client, and not from an imposter. (Clearly this is not terribly good security, but that's not the point of your question).

A good place to start is to use XML for your text-based format. This is dead simple to write/read, and is flexible and extensible so you can easily add more information to your packets at a later date - the biggest thing you can get wrong is to use a communications format that can't be extended!

Once you have basic comms working, you can enhance the format to send more information, add encryption and other security measures, and consider moving to a binary (more secure, more compact and efficient) format. BUt you can work your way to this stage in small easy steps.

So the right direction:

• Get two programs talking via TCP. Just a simple packet with the text "bob" in it is enough at this stage, just to verify that the messaging is working. There are any number of simple tutorials on the web to get this going, and it's just a few lines of code once you work out what's needed.

• Then build your packets. Start with the simplest approach that gives you a unique ID (to verify that the packet is from the right program) and a means to add new data to the packet easily in future. Xml is ideal for this. Don't worry about security, just concentrate on the actual "conversation" you wish to convey between the programs - what data they wish to exchange and how to encode it.

• Step by step improve the communications protcol until it achieves what you want - smaller, faster, binary, more robust, fault tolerant, secure, etc. Each of these steps will be an interesting little challenge and by the time you've done them all you'll have learned a lot.

Answer 6

I've recently read a book on this topic. It's called "TCP IP Sockets in C", by Michael J. Donahoo and Kenneth L. Calvert. You if you can afford it, it's a nice tutorial/reference book to have.

If you'd like you can try create the client<->server pair in Java, as it is easier to grasp the idea, and then rethink the solution at a lower level in C.

Answer 7

Look at the chapter on text protocols in 'The Art of UNIX Programming' by E S Raymond. It covers a lot of the relevant ideas at a high level, with good examples, and explanations of why they are good examples. It mentions BEEP.

Answer 8

All the other comments are good, and stuff like BEEP, or doing some custom TLV encoding can get you along way, as well as using something like Google protocol buffers, but none of these are what I'd really call real simple.

A very simple text based protocol could just use a new line as the message delimiter. This is how IRC does it. Its not the most efficient, but if your messages are reasonably small it could work quite well. You could also prefix your message with a much shorter line telling the receiver how long the next message is.

If you want to use a light framework, look at libevent. It can assist in your IO and do line delimited reading for you.

If the language (protocol) is not already determined for you, then that is what you should design first, or look at something that already exists - XML, JSON chunks, netstrings, etc.