Open Source Queuing Solutions for peek, mark as done and then remove

Question

I am looking at open source queuing platforms that allow me do the following: I have multiple producers, multiple consumers putting data into a queue in a multithreaded environment with the specific use case: I want the ability for consumers to be able do the following

1. Peek at a message from the queue(which should mark as the message as invisible on the queue so that other consumers cannot consume the same message)
2. The consumer works on the message consumed and if it is able to do the work successfully, it marks the message as consumed which should permanently delete it from the queue.
3. If the consumer dies abruptly after marking the message as consumed or fails to acknowledge successful consumption after a certain timeout, the message is made visible on the queue again so that another consumer can pick it up.

I've been looking at RabbitMQ, hornetQ, ActiveMQ but I'm not sure I can get this functionality out of the box, any recommendations on a system that gives me this functionality?

Answer 1

By way of introduction I can say that I've built and designed many message based systems from the ground up, using many technologies including CORBA, COM and native sockets.

In many of these it is the design that sits on the technology that is important.

Bearing this in mind I would probably choose to start with RabbitMQ and maybe enhance it if needed.

In many ways it is a headful to understand AMQP but it is worth the time, and I believe that it will allow you to make this work.

Even if you can't get the exact functionality out of the box the important question is can you make it do this, which I believe I could. It's opensource after all.

Answer 2

What you're asking for is standard JMS behaviour - which would be implemented out of the box by any compliant JMS implementation.

Answer 3

RabbitMQ does this out of the box, except for the timeout-based redelivery. If the connection is dropped while a message is unacknowledged, the message will be requeued for delivery to some other consumer of the queue. You can either use pull-mode ("Basic.Get") or push-mode/subscribe-mode ("Basic.Consume") to get the server to feed you messages.