Adding items to a List<T> / defensive programming

Question

Explicitly checking/handling that you don't hit the 2^31 - 1 (?) maximum number of entries when adding to a C# List is crazyness, true of false?

(Assuming this is an app where the average List size is less than a 100.)

Answer 1

Seems excessive. Would you not hit the machine's memory limit first, depending on the size of the objects in your list ? (I assume this check is performed by the user of the List class, and is not any check in the implementation?)

Perhaps it's reassuring that colleagues are thinking ahead though ? (sarcasm!)

Answer 2

It would seem so, and I probably wouldn't include the check but I'm conflicted on this. Programmers once though that 2 digits were enough to represent the year in date fields on the grounds that it was fine for the expected life of their code, however we discovered that this assumption wasn't correct.

Look at the risk, look at the effort and make a judgement call (otherwise known as an educated guess! :-) ). I wouldn't say there's any hard or fast rule on this one.

Answer 3

As in the answer above there would more things going wrong I suspect than to worry about that. But yes if you have the time and inclination that you can polish code till it shines!

Answer 4

True

(well you asked true or false..)

Answer 5

1. Memory limits

Well, size of System.Object without any properties is 8 bytes (2x32 bit pointers), or 16 bytes in 64-bit system. [EDIT:] Actually, I just checked in WinDbg, and the size is 12bytes on x86 (32-bit).

So in a 32-bit system, you would need 24Gb ram (which you cannot have on a 32-bit system).

2. Program design

I strongly believe that such a large list shouldn't be held in memory, but rather in some other storage medium. But in that case, you will always have the option to create a cached class wrapping a List, which would handle actual storage under the hood. So testing the size before adding is the wrong place to do the testing, your List implementation should do it itself if you find it necessary one day.

3. To be on the safe side

Why not add a re-entrance counter inside each method to prevent a Stack Overflow? :)

So, yes, it's crazy to test for that. :)

Answer 6

Just tried this code:

List<int> list = new List<int>();
while (true) list.Add(1);

I got a System.OutOfMemoryException. So what would you do to check / handle this?

Answer 7

If you keep adding items to the list, you'll run out of memory long before you hit that limit. By "long" I really mean "a lot sooner than you think".

See this discussion on the large object heap (LOB). Once you hit around 21500 elements (half that on a 64-bit system) (assuming you're storing object references), your list will start to be a large object. Since the LOB isn't compacted in the same way the normal .NET heaps are, you'll eventually fragment it badly enough that a large enough continous memory area cannot be allocated.

So you don't have to check for that limit at all, it's not a real limit.

Answer 8

Yes, that is crazyness.

Consider what happens to the rest of the code when you start to reach those numbers. Is the application even usable if you would have millions of items in the list?

If it's even possible that the application would reach that amount of data, perhaps you should instead take measures to keep the list from getting that large. Perhaps you should not even keep all the data in memory at once. I can't really imagine a scenario where any code could practially make use of that much data.