Any client libraries that implement MVCC over Amazon SimpleDB?

Question

Now that Amazon's SimpleDB implements consistent reads and conditional update/delete it is possible to implement cross-domain transactions using MVCC.

Are there any client libraries that provide it?

Answer 1

Currently, there are not, and there is a chance that it may not be possible.

Conditional updates and deletes only allow a condition to be set on the item being updated. In addition to this, there is no built in mechanism to store multiple versions of the same data or store sequence information for multi-valued attributes. Different attribute names can be used, but this breaks querying.

The recent consistency updates allow for easily implemented transactions at the item level. However, across multiple items in the same (or different) domain, there is no straight forward implementation. Also there are no isolation level options.

It might be possible to do, but I fear that you would end up killing all query capability in the process. Either through inconsistent attribute names or by requiring more SELECT conditions than good performance will allow.

Beyond that, it seems like it would require consistent reads for ALL access to the data. This will probably negate all of the the availability benefits of using an eventually consistent system, since consistent reads fail when even a single replica node cannot be reached.

I'm not trying to be a nay-sayer, I just wonder how practical it is. You already have to give up a lot of features to get SimpleDB's high availability, which is fine if you don't need those features, but if you then give up the high availability as well, I think there are a lot fewer cases where that would be a good trade off.

Answer 2

I've thought a lot about this while working on my Simple Savant C# library for SimpleDB, and I've come to the conclusion that attempting to layer true transactionality on a distributed system like SimpleDB is a bad idea for a whole host of reasons.

The best thing I've come up with (that provides value without over-promising and over-complicating the system) is something I'm calling "reliable writes". This would guarantee that all operations (puts and deletes) in a cross-domain update complete eventually. The only way for part of the write to fail permanently would be if one of your updates violated SimpleDB constraints.

This feature has not yet been implemented, but you can read more details and comment on the feature here. I'd be interested in hearing your thoughts and how this would meet your needs as an application developer.