I'm currently implementing a very complex tree structure to allow for near-instant data access, instead of re-processing on each request.
I'm just wondering if there is a theoretical, or practical, limit at which the size of a tree becomes too big, or a point at which a dictionary becomes too collision-filled to function correctly/quickly?
A general answer would be appreciated but C#-specific information would be much better!
In .NET the maximum in either a Tree or Dictionary is 2^31 - 1 (and probably a few less with overhead).
In practical terms, you will probably have run out of memory long before then!
If the tree remains balanced then searches will remain approx. O(log N).
Dictionaries are more sensitive to the underlying algorithm used, for instance there are many hashing schemes with different characteristics.
Depends what you consider a large amount. Hundreds or Thousands should be OK, but millions may be worth looking for something specialised.
A tree will get slower as you grow, depending upon your storage technique and rebalancing.
A dictionary should be fairly consistent, but make sure you construct it with a size appropriate to the amount of data you're likely to store (maybe x2 to be safe).
See this question - it's one of the first I answered on So :)
The problem was slow performance building a dictionary with approx 900,000 items. I got the time down from 10+ minutes to 0.34 seconds.
The lesson being, the dictionary is only as good as your hash function, if you can generate a unique hash quickly, it'll run like lightening.
Hope this helps,
EDIT:
The comparison class isn't important, .net strings have a very strong hash function and - therefore - have excellent performance in dictionaries. That guys problem would have just "gone away" if he'd been using single strings instead of pairs of strings.