Confused as to when you would use JMS (or a queue in general) versus a database

Question

When you store a message in a queue, isn't it more of meta data information so whoever pulls from the queue knows how to process the data? the actual information in the queue doesn't always hold all the information.

Say you have an app like Twitter, whenever someone posts a message, you would still need to store the actual message text in the database correct?

The queue would be more used to broadcast to other subscribers that a new message has arrived, and then those services could take further action.

Or could you actually store the tweet text in the queue also? (or you COULD, but that would be silly?)

Could a queue message have status fields, which subscribers can change as they process their part of the work flow? (or would you do that in the db?)

Just trying to get some clarification of when you would use a queue versus db.

Answer 1

I think your twitter example is good. You want the database for long term data. There wouldn't be much point in putting the tweet in the message because it has to go in the database. However, if you were running a chat room then you could go ahead and put the message in the JMS queue because you're not storing it long term anywhere.

It's not that you can't put the tweet in the JMS it's that you need to put it in the database anyways.

Answer 2

I recommend that you look at Gregor Hophe's book, Enterprise Integration Patterns, which explains many different patterns for messaging-based approaches.

Answer 3

In general, a queue is used to 'smooth' out publish rate versus consume rate, by buffering incoming requests that can't be handled immediately. A queue is usually backed by some sort of non-volatile storage (such as a database table). So the distinction is not so clear cut.

Use a database when you want to perform many searches against your 'queue', or provide rich reporting.

Answer 4

I would use the queue whenever you can utilize a "fire-and-forget" pattern. In your Twitter example, I would use the queue to post the message from the client. The queue processor can then store it to the database when it gets to it.

If you require some sort of immediate success/failure status, then the message queue isn't for you.

Answer 5

When a process wants to farm data and the processing of that data out to another process (possibly on a different host), there are 2 strategies:

1. Stuff all your data into the queue item and let the receiving app worry about storing it in the database, among with whatever other processing.

2. Update your database, and then queue a tiny message to the other process just to notify it that there's new data to be massaged.

There are a number of factors that can be used to decide on which strategy:

• If your database is fully ACID (one would hope) but your queueing system (QS) is not, your data would be safer in the DB. Even if the queue message gets lost in a server crash, you could run a script to process unprocessed data found in the DB. This would be a case for option 2.

• If your data is quite large (say, 1 MB or more) then it might be cruel to burden your QS with it. If it's persistent, you'll end up writing the data twice, first to the QS's persister and later to the DB. This could be a drag on performance and influence you to go for option 1.

• If your DB is slow or not even accessible to your app's front end, then option 1 it is.

• If your second process is going to do something with the data but not store it in a DB, then option 1 may be the way to go.

Can't think of any more, but I hope you get the idea.

Answer 6

We used JMS extensively at my last job where we were passing data around from machine to machine. In the end, we were both sending and storing the data at the same time; however, we stored far less data than we sent out. We had a lot of metadata surrounding the real values.

We used JMS as simply a messaging service and it worked very well for that. But, you don't want to use JMS to store your data as it has no persistence (aside from being able to log and replay the messages perhaps).

One of the main advantages that JMS gives you is the ability to send out your messages in the correct and appropriate order and ensure that everybody receives them in that order. This makes synchronization easy since the majority of the message handling is done for you.