Any reason why I shouldn't use couchdb for message passing or realtime activity streams?

While using ampq or xmpp (rabbitmq or ejabbered that could have couchdb as backends) seems like a good fit to deliver real time updates about friend state in a social gaming platform where updates are small but frequent, I can't help but think why wouldn't couchdb be a good platform to deliver such updates?

The main advantage I could think of is its ability to filter updates based on friends and availability of changes api, which makes developing such an application and managing it (including replication) quite easy compared to ampq or xmpp where you have to think about how to manage the pubsub nodes and who is subscribed to them at any point in time.

However, I can't help but think this is too good to be true, I can't find information on what couchdb's shortcomings are. Somehow, it feels like using MySQL for message passing which is why I am hesitant to using it.

Anyone have any experience in using couchdb for such applications? would you recommend another platform to use?

Here's some problems you'll run in to with "small but frequent" updates and CouchDB.

CouchDB has an excellent MVCC system for document updates. Every update changes the revision and you can't update a document without passing the revision you'd like to update. This also means that if you have multiple clients updating the same document incredibly frequently this feature will get in the way since even with the changes feed there is a small network delay after updates.

Another problem you might have using filtered changes is that the filter function gets the request object, which means it's a separate call to the view server for every document multiplied by the number of connections. You may instead want to use a node.js or erlang server to implement a "channels" approach to change filtering and have it sit on a single unfiltered changes feed.

To summarize, what you want to do will work well if you don't have multiple clients attempting to update the same documents at high frequency and if you don't use a changes filter on a very large number of concurrent clients with a high database update frequency. Other than that, it'll work great. @jchris has done a ton of real-time application just using the bare changes feed and they work great.

so if I were to aggregate the updates using node.js and update couchdb periodically (every few minutes) with bulk updates using only this single instance of node.js, then updating wouldn't be my biggest problem.My concern would then be, filtering the updates using views for the many clients connecting to couchdb. This could be alleviated somehow if I create a database per user at the expense of disc space, and simply post his/her updates to that database such that each client get unfiltered updates, where in actuality the filtering is done by routing messages to dbs

Up 2010-05-27 17:49:14

yeah, that's one way. you could also just have a node process listen to all changes and provide the filtering on the changes which would remove all that extra IO latency on the view server call per changes listener. there are also ways to "fake" atomic updates with couchdb which can alleviate the revision issues http://www.mikealrogers.com/archives/737

mikeal 2010-06-03 17:38:03

I think there's some confusion here about the term 'update'. It sounds to me like the questioner is talking more about the creation of many messages, not necessarily updating each one. So the objection about "Every update changes the revision" might not apply here. Presumably each message would be creating a new "document", and any subsequent "views" would render that information as needed.

Luke Bayes 2010-10-29 16:25:32

ansaurus

tags:

views:

answers:

Any reason why I shouldn't use couchdb for message passing or realtime activity streams?

related questions