Which MQ implementation would be suitable

views:

answers:

Which MQ implementation would be suitable

I have to implement Message Queue on site which do 20K/Day write operation in Database. We are willing to introduce message queues to take load off these write operations. I dont know much about Message Queue Implementation. If we implement it on one server only will it be helpful? or suggest some better impentation.

Also which MQ technology would be useful? We are php Shop with Apache

+1 A:

There's lots to choose from (nobody's mentioned rabbitmq so far) however if you are really only processing 20,000 transactions a day then using message queueing, particularly on a single server is not the right way to solve the problem.

symcbean 2010-10-21 12:47:35

@symcbean: Then what would you

Volatil3 2010-10-21 12:51:09

Look at why the system can't currently cope with this volume. What database? What tuning has been done already? How is it configured?

symcbean 2010-10-21 13:51:07

I think it's a suitable pattern. But I'm interested what you would suggest.

Till 2010-10-21 14:35:54

What about kestrel,made by Twitter?

Volatil3 2010-10-22 05:18:01

So instead of writing data to (real) database, you are proposing writing data to (queue) database, retrieving data from (queue) database, writing data to (real) database. And you expect this to be faster when running on a single machine?

symcbean 2010-10-22 08:22:30

There's a replication setup of One master and multiple slaves neither I said that only single Queue instance would be initiated.

Volatil3 2010-10-22 11:15:55

@symcbean: I agree that performance on a single server wouldn't really improve, but performance and scalability are two different concerns. I guess offloading to a queue is really faster regardless than writing to a database were locks are a natural thing.

Till 2010-10-23 11:48:29

@Till: You're damn right. We are experiencing Db locks which often cause query to get stuck.

Volatil3 2010-10-24 08:55:50

So breaking it down, it sounds like you also have some MySQL issues.

Maybe we can help you optimize this part of the stack first before you setup a queue? Especially because I'm not yet sure how your application uses MySQL and how or where you plan to queue items to offload from a potential write-lock.

I just answered a similar question a few days ago:

http://stackoverflow.com/questions/1178736/mysql-maximum-memory-usage/3987796#3987796

Let me know if this helps or how we can help further.

Till 2010-10-24 12:16:39

ansaurus

tags:

views:

answers:

Which MQ implementation would be suitable

related questions