ActiveMQ message grouping performance

Question

Has anyone used the Message Grouping feature in ActiveMQ?

http://activemq.apache.org/message-groups.html

This would be a really useful feature for a project I'm working on, but I'm curious how well this feature scales and performs. In our system, we would need to group messages into groups of about 3-5 messages, so we would be continuously adding groups as the process runs. In this case, it seems like we'd eventually just run out of memory trying to store all the groups.

I'm interested in any experiences/thoughts/pros/cons.

Answer 1

I've used Message Groups on many projects and it works great. Though for full disclosure I was one of the folks pushing for Message Groups and did much of the initial implementation work.

The use case of Message Groups came from partitioning large topic hierarchies; such as dealing with financial stock symbols and the like. We wanted message groups to be able to use very fine grained correlation expressions (JMSXGroupID strings) - so you could use the date, stock symbol and product type as groupID - or the customer or business transaction ID or whatever.

To avoid having to keep every group ID string in memory, the default provider uses hash buckets - so we only store the mapping of hash buckets to consumers - not the individual strings. So it scales to as many group IDs as you want to use! It also means we don't have to 'clean' the old message group IDs out etc

Answer 2

I too am interested in this topic. I'd like to know: if there is one listener on a queue, and the messages sent to the queue belong to different message groups (for example, let's say there are four mesages groups "1", "2", "3", "4"), will the listener be instantiated four times, i.e. in four separate threads?

Also, I'm sorry to be dense, but James, what do you mean by hash buckets? Do I need to understand that before I proceed? I'm afraid I need things dumbed down a bit. I just want to use this stuff, not reinvent it.

Thanks,

Mike