I need a way to do key-value lookups across (potentially) hundreds of GB of data. Ideally something based on a distributed hashtable, that works nicely with Java. It should be fault-tolerant, and open source.
The store should be persistent, but would ideally cache data in memory to speed things up.
It should be able to support concurrent reads and writes from multiple machines (reads will be 100X more common though). Basically the purpose is to do a quick initial lookup of user metadata for a web-service.
Can anyone recommend anything?