JMS Messaging Performance: Lots of Topics/Queues vs. Extensive Filtering (Message Selectors)

Question

I'm working on a project that is going to make heavy use of JBoss Messaging (JMS). I'm tasked with building an easy to use wrapper around Messaging for other developers and am thinking about using JMS's Message Selectors to provide a filtering technique to keep unnecessary sending of messages to a minimum. I'm curious if anyone has any experience with doing so in regards to performance? My fear is that the JMS provider may get bogged down with the Message Selectors, effectively defeating the whole purpose. It would be much nicer however than creating a long list of Topics/Queues for every message type.

Ultimately I'll undoubtedly end up using some combination of the two, but am concerned with the impact on performance whichever way I lean more towards.

Answer 1

Hmm, I have my doubts. JMS is pretty easy-to-use. I've seen this tried, and the easier-to-use solution was harder to use and buggy.

Answer 2

Different implementation, but I'll pass along a conversation I had with a high-level architect for BEA's JMS products. I mentioned using selectors and he commented something along the lines of "fine, if you don't want it to perform".

Our app was doing 10's of messages/sec. He's probably used to seeing the tough problems with 100-1000's per second. Unless you're in those higher ranges or have really slow hardware, either many queues/topics or selectors will probably work OK.

On Don's point about JMS being easy to use... We went with a wrapper to abstract things. Once you get into issues like robust reconnection and correctly dealing with multithreading/async listeners, there are many wrong ways to write the code. It was well worth it for us to wrap the details so clients could stay innocent of most of the details.

Answer 3

From my experience with the JBoss MQ implementation, message selectors were used by the clients to filter messages. Obviously, this means every message in a Topic still goes to every recipient, even if they ignore it. On the other hand, different queues and topics on the server will affect server performance.

I'd say proliferation of selectors will affect client & network load and proliferation of topics & queues will affect server load. Obviously, network load, message consumer load, and message producer load all scale differently.

Beyond the simple case, the wrappers get tricky; I'd recommend you wrap the error handling and JMS API into a simple message passing API conceptually structured to meet your particular needs. Then, under the covers, you can change to any of the different designs above with a minimum of fuss.

Answer 4

As Martin mentioned, by default most JMS implementations will process message selectors on the client, unless they are part of a durable subscription, when most JMS implementations will process them on the server as well to avoid too many messages getting persisted when there's a significant reduction in the number of messages that get past the selector. Some systems (like SonicMQ) allow you to specify that message selectors should be processed on the server, which is a good option in a case where you have excess CPU available on your message brokers but not on your consumers.

Bear in mind that while topic-based selection is usually faster, it can be quite cumbersome, because if you want to listen to 5 different things, you have to have 5 different MessageConsumers. Each of those in a naive driver implementation is a different thread, and that can start to add up. For that reason, it is often useful to support both from publication so that some clients can listen only to the topics that they want, and others can listen to the topic hierarchies they want (e.g. foo.#) with message selectors (or code-based selectors).

However, you should always test against your application and your broker. Every broker handles the situation differently, and every application functions differently. You can't just say "always use technique X" because each technique for client-directed message processing has different tradeoffs. Benchmark, benchmark, benchmark.

One thing to bear in mind with message selectors is that they aren't dynamically changeable, so you have the possibility of losing messages or having to manually manage a complicated switchover scenario. Imagine the following use case:

1. You are listening to a message selector of the form (Ticker in ('CSCO', 'MSFT'))
2. User wants to start listening to AAPL
3. You have to shut down the old MessageConsumer and start a new one with a selector in the form (Ticker in ('CSCO, 'MSFT', 'AAPL'))
4. During the switchover, you either lose messages (because you shut down the old one before starting the new one) or you have to manually remove duplicates (because you have the new one started before the old one)

Answer 5

My two cents:

I asked myself exactly the same question concerning ActiveMQ.

• First, I did not use selectors and created lots of topics. Performance was horrible as the broker could not handle 100's of topics without a lot of resources.
• then I used a combination of topics/selectors. I now have a small number of topics. The selection works well. But the load is not very heavy, no more than 10 msg/s

I did develop an abstraction layer, allowing the developers to code without asking questions, and we did tests by switching the implementations.