Implementing message priority in AMQP

Question

Hello all,

I'm intending to use AMQP to allow a distributed collection of machines to report to a central location asynchronously. The idea is to drop messages into the queue and allow the central logging entity to process the queue in a decoupled fashion; the 'process' is simply to create or update a row in a database table.

A problem that I'm anticipating is the effect of network jitter in the message queuing process - what happens if an 'update' accidentally gets in front of an 'insert' because the time between the two messages being issued is less than the network jitter?

Reading the AMQP spec, it seems that I could just apply a higher priority to 'inserts' so they skip the queue and get processed first. But presumably this only applies if a queue actually exists at the broker to be skipped. Is there a way to impose a buffer or delay at the broker to absorb this jitter and allow priority to be enacted before the messages are passed on to the consumer(s)?

Or do I have to go down the route of a resequencer as ActiveMQ suggests [1]?

Thanks, Matt.

[1] http://activemq.apache.org/how-can-i-support-priority-queues.html

Answer 1

The lack of ordering between multiple publishers has nothing to do with network jitter, it's a completely natural thing in distributed applications. Messages from the same publisher will always be ordered. If you really need causal ordering of actions performed by different nodes then either a resequencer or a global sequence numbering scheme are your only options. Note that you cannot use sender timestamps for this, which is what everyone seems to try first..