Handling storage of billions of pieces of information?

Question

What is the best solution if I need to have a database with a billion+ objects and I need to have immediate (or nearly immediate) access to any of the items in the database at any time.

This database would be queried at about 1000 requests per second. The rows in the database are pretty much unrelated and thus doesn't need to be relational.

If you're curious why, it's for a simulation of moving elements.

I was thinking something like several load balanced clusters of a Cassandra that are accessed through a load balanced cluster of web servers.

Money is a factor so the cheaper the better. There is no restriction on the software or tool it however must be open source.

Just looking for a database solution that would be good at handling a ridiculous amount of data (does not need to be relational at all) by a large number of users.

It is essential that it handle redundancy and failures.

Just a high level idea to put me in the right direction would be great.

Answer 1

Since you will need to be able to efficiently get all objects within a 3D interval (X_min <= X_obj <= X_max & Y_min <= Y_obj <= Y_max & Z_min <= Z_obj <= Z_max), I am not sure how well a key-value store like Cassandra will suit you. It may be worthwhile to as well have a look at MongoDB since I believe this allows you to index multiple fields and query based on intervals.