views:

47

answers:

2

I have a java class Processor that is listening to a jms topic and it is struggling to keep up with the speed in which messages are arriving so we've decided to go concurrent:

A single class listening to the topic who's job is to hand the messages out to a pool of worker threads, effectively being a load balancer. It also has to prevent 2 workers processing messages for the same customer.

I expected there to be quite a lot of information on the internet about this but everything seems to suggest the use of EJBs where the app server manages the pool and does the balancing. I'm sure this must be a really common problem, but can't seem to find any libraries or design patterns to assist. Am I making more out of it than it is and should just delve in and write my own code?

+1  A: 

Why don't you just use a queue instead of a topic and have several instances of the same application handle messages from this queue ?

pgras
The reason we have a topic at the moment is that we have multiple applications running that filter the messages out so rather than receiving 100 msg/s it gets only a few. You have a good point though, a topic is unnecessary if we could keep up. However, something would still have to sit in the middle to prevent two instances processing the same customer?
James
@James: no sure what you mean by "customer", do you have several message per customer ??? You may use "message selectors" to filter out messages per consumer...
pgras
A message has a customerId. Since many instances are processing messages there would many customerId being processed. What I was trying to explain was that it is essential that the same customer id is not being processed by 2 instances at any one time, so some kind of locking and message prioritisation is necessary that I envisage must have to 'sit in the middle' as I put it?
James
A: 

This is an easy problem to solve with a pool of listeners. That's what the app server would be doing for you.

I'd get a good app server and use its MDBs to solve this quickly. Size the pool to keep up and you'll be fine.

If you insist on writing your own code, get a good open source pool implementation and use it.

If it must be non-EJB, consider Spring. It has message driven POJOs that could be just what you need.

duffymo