How to "copy" a JMS message to 2 destinations?

Question

I have a requirement that a single JMS message sent by a client must be delivered reliably (exactly-once) to two systems. These 2 systems are not HA-enabled, so the best suggestion that I came up with is to:

1. create single queue where client posts to

2. set up two "intermediate" queues

3. use a custom "DuplicatorMDB" that will read messages from the client queue and post them to two queues within the same transaction.

client->JMSDQ->DuplicatorMDB->Q1->MDB->System1
                           \->Q2->MDB->System2

Is there any existing functionality like that? What would be the proper way to balance the system to keep it stable if one or both of the backend systems are down?

The application server is WebLogic 10.

I can't use topics for this because in a cluster topics will cause too much message duplication. If we have 2 instances, then with topics it'll go like this:

 
client->Topic-->MDB1@server1->System1
           | \->MDB2@server1->System2 
           \---->MDB1@server2->System1 
            \--->MDB2@server2->System2 

Thus every message will be delivered twice to System1 and twice to System2 and if there'll be 8 servers in a cluster, each message will be delivered 8 times. This is what I'd really like to avoid...

Finally I got some time to test it and here is what I observed: 2 nodes in a cluster. 2 JMS servers: jms1 on node1, jms2 on node2. Distributed topic dt. MDB with durable subscription and jms-client-id=durableSubscriber. Started the system: 0 messages, mdb@node1 is up, mdb@node2 trying to connect periodically, but it can't because "Client id, durableSubscriber, is in use". As expected.

Sent in 100 messages: jms1@dt messages current = 0, messages total = 100, consumers current = 1 I can see that node1 processed 100 messages.
jms2@dt messages current = 100, messages total = 100 , consumers current = 1 i.e. "duplicate" messages are pending in the topic.

Sent in another 100 messages, 100 were processed on the node1, 200 pending on node2.

Rebooted node1, mdb@node2 reconnected to dt and started processing "pending" messages. 200 messages were processed on node2.

After node1 is up, mdb@node1 can't connect to the dt, while mdb@node2 is connected.

jms1@dt messages current=0, messages total = 0, consumers current = 0
jms2@dt messages current=0, messages total = 200, consumers current = 1

Send in 100 more messages, I see that all 100 messages are processed on node2 and discarded on node1.

jms1@dt messages current=0, messages total = 100, consumers current = 0
jms2@dt messages current=0, messages total = 300, consumers current = 1

Now I reboot node2, mdb@node1 reconnects to dt. After reboot mdb@node2 reconnects to dt and mdb@node1 gets disconnected from dt.

jms1@dt messages current=0, messages total = 100, consumers current = 1
jms2@dt messages current=0, messages total = 0, consumers current = 1

I send in 100 messages, all are processed on node2 and stored in the topic on node1:

jms1@dt messages current=100, messages total = 200, consumers current = 1
jms2@dt messages current=0, messages total = 0, consumers current = 1

Then I shut down node2 and I see 100 "pending messages" being processed on node1 after mdb@node1 reconnects to the topic.

So the result is: I sent 400 messages, 700 were processed by MDB out of which 300 were duplicates.

It looks like the MDB reconnection works good as expected, but the messages may be duplicated if a node hosting the "active" MDB goes down.

This might be a bug or a feature of weblogic JMS implementation.

Answer 1

I haven't used Weblogic, but most JMS solutions have the concept of Queues and Topics. You want a JMS Topic. Subscribers register and the topic ensures that the message is delivered to each subscriber once.

Configuration details.

Update: If you are running into issues in a clustered environment, I would make sure that everything is configured properly (here is a guide for JMS Topic Clustering). It definitely sounds strange that Weblogic would fail so miserably when clustering. If that does not work, you could look into a 3rd party Messaging Queue, such as RabbitMQ, which support JMS and will definitely not have this issue.

Answer 2

This is the kind of behaviour that an ESB implementation should enbale. In terms of processing overhead there would be no great difference, but it can be useful to have separation of concerns between "plumbing" and application code.

As it happens, the WebSphere JMS implementaion has support for installing mediations that address such requirements. I don't know whether WebLogic has something similar, or whether their associated ESB products are an option for you, but I would recommend investigating those capabilities. You currently have a simple requirement, and your code is surely sufficient, however it's quite easy to imagine how a few minor additional requirements (could we just convert this field from dollars to pounds before we transmit to that destination, could we not send messages with this content to that destination ...) and lo! you find yourself writing your own ESB.

Answer 3

[...] Thus every message will be delivered twice to System1 and twice to System2 and if there'll be 8 servers in a cluster, each message will be delivered 8 times. This is what I'd really like to avoid...

This is right for non-durable subscriptions, but not for durable. For durable, all MDBs share the same connection-id and subscription-id (based on the MDB name by default), so only one MDB will be able to attach and receive messages at a time. The first MDB to try will succeed in connecting, the others will detect a conflict and fail, but keep retrying. So using a durable topic subscription should do the trick.