Distributed Key-Value Data Store with Offline Access (Static Partitioning)

Question

Need to be able to set server(s) that replicate all information, as a master data store that has all the data.

Also need servers that specifically store/replicate certain data, available in local LANs, so that when the internet connection goes down, they can still access their local data. Under normal circumstances, the clients will access most of their data from the local LAN, and may use others when the local LAN server goes down.

This is wanted alongside the benefits of a distributed data store, such as failure resistance and speed.

Which Distributed Key-Value Data Store or other data storage method would be most suited for this?

Answer 1

Of course you must remember that replication is something completely different from backup, because one system's programmatic failure in handling the data can quickly replicate to other nodes resulting in total mayhem.

Maybe using a Hadoop File System or OpenAFS would be a good solution here?

I haven't used any of those systems in real-life scenarios, only had interest in them during my research on peer-to-peer and distributed storage solutions, but I think they're worth a try.

Answer 2

Have you checked out the new Microsoft's Velocity? http://msdn.microsoft.com/en-us/data/cc655792.aspx. Unlike many other cloud services, you can run the setup (for Velocity) on your premises.

Answer 3

Try out CouchDB. Your use case reads like it was build for it. Point taken, CouchDB is much more than a key/value store, but on the other hand, not less suitable for it.

Add replication and as an added bonus fault tolerance, conflict detection (and resolution) and an easy API (HTTP).

Let me know if you have any other questions.