Is it a good thing for a custom rest protocol to be binary based instead of text based like Http?

Question

Have you ever seen a good reason to create a custom binary rest protocol instead of using the basic http rest implementation?

I am currently working on a service oriented architecture framework in .Net responsible for hosting and consuming services. I don't want to be based on an existing framework like Remoting or WCF, because I want total flexibility and control for performing custom optimization.

So here I am trying to find the best protocol for handling this SOA framework. I like the request/response stateless connection nature of REST and the uri for defining resources, but I dislike the text based nature of HTTP.

Here are my arguments for disliking HTTP, correct me if I'm wrong:

• First an evidence, parsing text is less efficient than parsing binary. I'd prefer a fixed length binary header containing the content length and a binary content.

• Second, there is no notion of sequence number for http requests, so the only way of associating a response with its request is the socket connection used to send the request and receiving the response. This means there can only be one pending request at a time for a specified socket, so if a service consumer want to send multiple requests in parallel to a service, it need to open multiple socket to the server. A custom rest protocol could define a sequence number for requests, so request and response would be associated with the sequence number instead of the socket and there could be multiple requests sent in parallel on the same socket. I think there is no way to do this standardly with HTTP, it could be done with a custom text based protocol, but why not make it binary based to gain performance.

To add a little more context, my SOA framework does not need to be accessible from a non .Net consumer, so I have no restriction about using the .Net binary formatter or another custom binary formatter.

So am I making any sense for wanting a custom binary rest protocol? If you think I'm wrong please tell me your arguments.

Thanks.

Answer 1

The data of your packets can be whatever you like; be it XML, Plaintext, JSON, or just a binary format. I see no reason to force any specific one of these on yourself (use whatever is most fitting).

Though, when saying 'binary format', most people hear a fixed format of field-lengths and other really annoying things. Generally, if you aren't desperate from a data-transfer point of view, I see no reason to go this way [though you may do a binary serialisation of .net objects so they can be re-instantiated [or look at the 'protocol buffers' lib for .net]].

Summary: Seems okay to me. Whatever floats your boat.

Answer 2

REST is an architectural style for constructing web services. Roy Fielding articulated it based on his experience in designing HTTP, but it transcends HTTP. You can deploy a RESTful service over ordinary email exchange for instance.

The REST representations of your resources can be anything you like, though Roy really stresses that people should try to use very carefully designed, standard representations. There's nothing wrong with binary. In fact image representations like JPEG and PNG are binary. Google's Protocol Buffers gives you ways of creating compact binary representations of structured data too.

So the short answer is that you can certainly be RESTful and use binary representations and a home-grown binary substitute for HTTP.

I would actually very strongly recommend you use HTTP for efficiency, though. If you use your own protocol then you lose all the leverage provided by the wonderful HTTP caching infrastructure that stands between your server and its clients. The full client load falls right on your server instead of getting spread out over the intermediate caches.

10/4/2010: In our HTTP-based REST APIs we now support Java Serialized Object and Kryo binary representations in addition to XML, JSON, and XHTML. The Kryo serialization library performs significantly better than the others without requiring any special protocol. Another alternative to cut down on bandwidth is to use HTTP compression along with a textual representation.

Answer 3

Have you ever seen a good reason to create a custom binary rest protocol instead of using the basic http rest implementation?

I'm not sure I understand your terms. A REST representation can be completely binary, and still transported over pure HTTP. There's no need to invent a new protocol just to transmit binary data. Just reduce your requirements to a well-documented media type (which you're free to invent).

Regarding binary resource representations, Fielding himself argues that binary is not only acceptable, but may be required in some situations.

Regardless of whether you go straight binary, Base64 mixed with text, or text-only, don't forget the hypertext constraint if you plan on calling what you do "REST".

So am I making any sense for wanting a custom binary rest protocol?

If you mean you'd like to create a custom, binary-only, hypertext-driven media type - that's perfectly reasonable.

If you're talking about inventing some custom extension to HTTP, I would avoid that unless absolutely necessary (and what you describe doesn't sound to me like it rises to that level).

Answer 4

I want total flexibility and control

Based in this statement then I would suggest one of two choices. Either straight TCP/IP sockets, or WCF. These options give you by far the most flexibility. WCF is a great solution if you want to be transport agnostic and you want to start with a clean state as far as an application protocol is concerned.

REST imposes a set of constraints that restrict how your distributed application behaves in order to gain certain beneficial characteristics. If you see your service end points as ways to invoke operations then I suggest that REST will not fit your needs well at all.

Somehow, I don't think whether you send binary or text payloads should be your biggest concern at this point.