Let's say I start with replica sets and each mongo server has a 500G hard disk attached to each one. What happens if my data grows beyond 500G? I assume that's where sharding comes into play? In addition, I keep reading that in order to gain optimal performance, you would want to keep your indexes in memory. So if my index is 100G, does that mean EACH server (shard or non-shard) has to have at least 100G of RAM?
A:
If you data grows beyond the size of your disk, yes, that's a good time to start sharding. MongoDB will start reporting errors that it couldn't do inserts/updates that increase size once you run out of disk space, so you should shard a before you get to that point.
You really want to be able to keep the working set of your indexes (and data, if possible) in memory. So, if your doing a website and mostly people access the last week of content, if you have enough RAM to keep the last week in memory, you should be fine (even if you have a couple years of content on disk).
kristina
2010-09-19 03:48:36