Serializing objects for asynchronous messaging

Question

I'm considering using AMQP (using qpid) to enable a mixture of Python and Java services communicate with each other. Basic text messaging seems simple enough but, as with every other messaging technology I've investigated, that's where it seems to stop. Except for building instant messaging applications, I would have thought sending strings wasn't a particularly useful thing to do yet example after example demonstrates sending unformatted text around.

My instinct then is to use XML (de-)serialization or something similar (JSON, YAML, Protocol Buffers etc.) which has good library support in both languages. Is this a best practice and, if so, which (de-)serialization protocol would people recommend? Or am I missing the point somewhere and should be quite content sending small bits of text?

Answer 1

XML or JSON are probably the easiest. Protocol buffers is cool but I'd treat it as an optimisation to think of later on if you really need to (as its a bit harder to use being essentially a binary wire format).

BTW you might want to look at Stomp rather than AMQP; its got way more client libraries and supported message brokers. e.g. Apache ActiveMQ which is way more popular than qpid or rabbitmq - or indeed any JMS provider would work just fine.

Answer 2

Owen, may I offer a few words about RabbitMQ.

AMQP is a binary protocol and you can certainly do much more than send strings around! Which Python client do you plan to use? We recommend Barry Pederson's client for most uses: http://barryp.org/software/py-amqplib/ You are most welcome to come to the RabbitMQ list and ask any questions you like about anything in relation to your post and the comments :-)

As James points out, JSON is goodness. RabbitMQ supports JSON-RPC over HTTP connecting to an AMQP back end. People also use RabbitMQ with Orbited for comet type apps.

In addition we are fans of, and support XMPP, and STOMP too which James invented. STOMP is handy for a certain class of messaging apps and RabbitMQ supports it for both direct and topic based routing. We've found it a fine way to interop with ActiveMQ, preferring it to JMS in that scenario.

I hope you find the right server for your use cases, and recommend you try out different combinations, for best results.

Cheers,

alexis

Answer 3

For what it's worth, I've been having a good experience using AMQP + Protocol Buffers.

If the sender is serializing the messages, you will probably need to include an id in the header so that you know how to de-serialize on the receiving side. However, this isn't too much trouble to accomplish.