views:

3897

answers:

7

I've been trying to see if I can accomplish some requirements with a document based database, in this case CouchDB. Two generic requirements:

  • CRUD of entities with some fields which have unique index on it
  • ecommerce web app like eBay (better description here).

And I'm begining to think that a Document-based database isn't the best choice to address these requirements. Furthermore, I can´t imagine a use for a Document based database (maybe my imagination is too little).

Can you explain me if I am asking pears to an elm when I try to use a Document based database for this requirements?

+4  A: 

Document based DBs are best suiting for storing, well, documents. Lotus Notes is a common implementation and Notes email is an example. For what you are describing, eCommerce, CRUD, etc., realtional DBs are better designed for storage and retrieval of data items/elements that are indexed (as opposed to documents).

Jim Anderson
I don't agree. A document database isn't primarily for storing documents. It is for storing hierarchical pieces of data (either JSON or XML). You can index nested JSON fields and JSON arrays with for instance MongoDB. You can store documents (files) in MongoDB (gridfs) but MongoDB would still be useful if you couldn't store documents (files) with MongoDB. I think that MongoDb should be called a JSON db and not a document db.
Theo
According to the Wikipedia entry for "Document-oriented database", "...using XML, YAML or JSON for information storage has advantages similar to document oriented database" but they are not the same thing. Document databases were originally designed so store documents. If you use them for other data, you are not going to get the best performance/usage just the same as if you store documents in a relational databases. This happens a lot. People store relational data in document databases and then complain how bad document databases are. If you misuse them, yes.
Jim Anderson
+7  A: 

You need to think of how you approach the application in an Document oriented way. If you simply try and replicate how you would model the problem in an RDBMS you will fail. There are also different trade-offs that you might want to make. Remember as well that CouchDB's design is assuming that you will have an active active cluster of many nodes that could fail at any time. How is your app going to handle one of the database nodes dissapearing from under it?

One way to think about it is to imagine you didn't have any computers, just paper documents. How would you create an efficient business process using bits of paper being passed around? How can you avoid bottlenecks? What if something goes wrong?

Another angle you should think about is eventual consistency, where you will get into a consistent state eventually, but you may be inconsistent for some period of time. This is an anathema in RMDBS land, but is extremely common in the real world. The canonical transaction example is of transferring money from bank accounts. How does this actually happen in the real world? Through a single atomic transactions or through different banks issuing credit and debit notices to each other? What happens when you write a cheque?

So lets look at your examples:

  • CRUD of entities with some fields with unique index on it.

If I understand this correctly in CouchDB terms, you want to have a collection of documents where some that some named value is guaranteed to be unique across all those documents? That case isn't generally supportable because documents may be created on different replicas.

So we need to look at the real world problem and see if we can model that. Do you really need them to be unique? Can your application handle multiple docs with the same value? Do you need to assign a unique identifier? Can you do that deterministically? A common scenario where this is required is where you need a unique sequential identifier. This is tough to solve in a replicated environment. In fact if the unique id is need to be strictly sequential with respect to time created it's impossible if you need the id straight away. You need to relax at least one of those constraints.

  • ecommerce web app like ebay

I'm not sure what to add here as the last comment you made on that post was to say "very useful! thanks". Was there something missing from the approach outlined there that is still causing you a problem? I thought MrKurt's answer was pretty full and I added a little enhancement that would reduce contention.

kerrr
+4  A: 

Is there a need to normalize the data?

  • Yes: Use relational.
  • No: Use document.
dacracot
I know you answered this a long time ago, but I thought I'd ask... When do you "need" to normalize? Isn't normalization a choice/best practice?
Matt Grande
@Matt, data normalization is just a tool. The degree to which you normalize data is a tradeoff between database design effort and consistency maintenance effort.
Eduardo León
A: 

when is there a need for normalized data?

To develop a transactional app, there's always a need to normalize data to some degree. Either the third NF or the Boyce-Codd NF are usually good compromises between the effort required to normalize data and the effort required to maintain data consistency when it's not normalized enough.
Eduardo León
1,2 up to 3 NF (BNF if your feeling boisterous). Mainly to reduce the storage overheads of data so that data is stored more efficiently and also to keep the data clean without anomalies. I can hear Dr S. McKearny repeating the stages of normal form even after more than a decade!
WeNeedAnswers
A: 

Another advantage of document-oriented databases is the ease of usage and programming so that untrained business users, for example, can create applications and design their own databases. Information can be added without worrying about the "record size" and so programmers simply need to build an interface to allow the information to be entered easily.

smdelfin
Uh, don't copy-paste from Wikipedia (or any other source), explicitly quote it. Trying to pass something someone else came up with as the product of your own work is inherently dishonest.
Eduardo León
This is the Wikipedia article that should've been cited: [Document-oriented database](http://en.wikipedia.org/wiki/Document-oriented_database) (last paragraph in the _Overview_ section)
stakx
+2  A: 

A possibility is to have a main relational database that stores definitions of items that can be retrieved by their IDs, and a document database for the descriptions and/or specifications of those items. For example, you could have a relational database with a Products table with the following fields:

  • ProductID
  • Description
  • UnitPrice
  • LotSize
  • Specifications

And that Specifications field would actually contain a reference to a document with the technical specifications of the product. This way, you have the best of both worlds.

Eduardo León
SQL Server 2008 is an example of a database that can do both (using the FILESTREAM datatype).
John Saunders
Wow. Awesome feature. (I've never used SQL Server 2008.)
Eduardo León
+2  A: 

I am in the same boat, I am loving couchdb at the moment, and I think that the whole functional style is great. But when exactly do we start to use them in ernest for applications. I mean, yes we can all start to develop applications extremely quickly, cruft free with all those nasty hang-ups about normal form being left in the wayside and not using schemas. But, to coin a phrase "we are standing on the shoulders of giants". There is a good reason to use RDBMS and to normalise and to use schemas. My old oracle head is reeling thinking about data without form.

My main wow factor on couchdb is the replication stuff and the versioning system working in tandem.

I have been racking my brain for the last month trying to grok the storage mechanisms of couchdb, apparently it uses B trees but doesn't store data based on normal form. Does this mean that it is really really smart and realises that bits of data are replicated so lets just make a pointer to this B tree entry?

So far I am thinking of xml documents, config files, resource files streamed to base64 strings.

But would I use couchdb for structural data. I don't know, any help greatly appreciated on this.

Might be useful in storing RDF data or even free form text.

WeNeedAnswers