Are there any e-commerce websites that use NoSQL databases

Question

I have read a lot lately about 'NoSQL' databases such as CouchDB, MongoDB etc. Most of the websites I have seen using this are mainly text based websites such as The New York Times and Source forge.

I was wondering if you could apply this to websites where payment is a huge issue. I am thinking of the following issues:

• How well can you secure the data
• Do these system provide an easy backup/restore machanism
• How are transactions handled commit/rollback

I have read the following articles that cover some aspects:

• Can I do transactions and locks in CouchDB?
• Pros/Cons of document based database vs relational database

In these posts the aspect of transactions if covered. However the questions of security and backups is not covered. Can someone shed some light on this subject?

And if possible, does anyone know of some e-commerce websites that have successfully implemented the document based database.

Answer 1

I don't think security would be any different on a NoSQL database than on a relational database. In the end, security is an orthogonal question to how data is actually stored. Besides, it's not like you'd allow access to the database from anything but your business-layer servers from a networking standpoint.

As for backups, most NoSQL databases that I know of allow for hot backups, just like a regular database does.

The real question, IMO, is whether you can live with the restrictions that a NoSQL database puts on you - in particular, the general lack of ad-hoc queries. For example, if you ever wanted to know all of the people who ever bought product "X" then you'd have to build into your data access layer a counter for that from day one (or run a very expensive serial lookup of every past transaction). In a regular SQL database, you can just add an index and do a query and you're done (or even, don't add an index if it's a one-off). Or maybe you want to find out all the people who bought product "Y" before the latest version came out (so you can send them a reminder to upgrade or whatever): again, you have to plan that ahead with a NoSQL database, but it's trivial with a relational database.

I think it makes sense when you can plan your schema and your usage pattern ahead of time, and where the occasional re-scan of records to add some new field or metric is acceptable. But for an e-commerce website, I think ad-hoc queries are just too valuable a feature to lose. Of course, that's just my opinion, and there's certainly no reason why you couldn't mix-n-match parts of the application between the two databases. I'd personally choose a relational database with memcached in between for added performance, though...

Answer 2

Handling financial information is one of the areas where SQL really is the right tool for the job. Most of the NOSQL systems were designed to improve scalability by accepting a higher risk of data loss or inconsistency. They also tend to have limited abilities to run reports over all records, since on a typical large website you only need enough data in the index to find and display a single record - the rest can be completely inaccessible until you know the record you are looking for.

When dealing with money, any data inconsistency is a big problem, and if you need more scalability than a single sql server can give you, you have enough money that you can afford the higher cost of scaling sql. Also, the ad-hoc reporting available from sql is something you'd miss if you don't use sql - pretty much any information you want about sales history is trivial to get from sql, but potentially requires complex custom code from an object based store.

Answer 3

1. Amazon S3 uses NoSQL implementation.

Answer 4

The overhead that makes RDBMS's so slow, is guaranteeing atomicity, consistency, isolation, durability, also known as ACID. Some of these properties are pretty critical for applications that deal with money. You don't want to lose a single order when the lights go out.

NoSQL databases usually sacrifice some or all of the ACID properties in return for severely reduced overhead. For many applications, this is fine -- if a few "diggs" go missing when the lights go out, it's no big deal.

For an ecommerce site, you need to ask yourself what you really need.

1. Do you really need a level of performance that a RDBMS can't deliver?
2. Do you need the reliability that an DRMBS provides?

Honestly, the answer to #2 is probably "yes", which rules out most NoSQL solutions. And unless you're dealing with traffic levels comparable to amazon.com's, an RDBMs, even on modest hardware will probably satisfy your performance needs just fine, especially if you limit yourself to simple queries, and index properly. Which makes the answer to #1 "no".

You could however, consider using a RDBMS for transaction data, and a NoSQL database for non-critical data, like product pages, user reviews, etc. But then you'd have twice as much datastore software to install, and any relationships between the data in the two datastores would have to be managed in code -- there'd be no JOINing your NoSQL database against your RDBMS. This would likely result in an unnecessary level of complexity.

In the end, if an RDBMS offers features you must have for reliability, and it performs acceptably for the sorts of load you'll be experiencing, an RDBMS is probably the best bet.

Answer 5

You guys should check this out:

Replication Acknowledgement via getlasterror

MongoDB is on the verge of providing durable writes. I think that is the main issue with people discuss this topic w.r.t. money. The transactional part is less important due to the nested document features.

Answer 6

Just posted some thoughts on MongoDB and E-commerce: http://kylebanker.com/blog/2010/04/30/mongodb-and-ecommerce/

Answer 7

Gilt.com uses Voldemort to handle basket / inventory under huge load. See this presentation from London QCon 2010 on the details - http://www.infoq.com/presentations/Project-Voldemort-at-Gilt-Groupe

I'd also reiterate the fact that "NoSQL" does not mean "No SQL", but "Not Only SQL", and that instead of looking at any technology for a complete rip/replace of any other you should be looking at the best tool for the job. NoSQL data stores don't make very good data warehouses, and probably aren't appropriate for storing user transactions, but they are very good in certain niche areas - see the Gilt Groupe example above.

Another prominent example is the BBC homepage - not transactional, but interesting nonetheless. They use CouchDB to store user preferences. Unfortunately, they appear to have crashed under the load.