Why is the size of my HeapOverflow program 5MB ?

Question

Hi !

So I was asking myself what would happen if I tried to do a heap overflow on Windows XP, and I was surprise to see that, once the program "ate" all the RAM (this happens instantly, by the way), the size of the process in the task manager goes down to 5MB and doesn't move afterwards. The computer memory usage is still growing, however.

So why is Windows not able to see that my software takes GB of memory ? I feel like it can be a security problem because once a software ate all the memory, it can "hide" in the small process groups (and maybe I'm a little bit paranoid).

Note : nothing happens when the heap is full, the cpu just jumps to 100% because my for(;;) loop runs like crazy once malloc fails.

Edit : Ok! Never knew that you could tweak the task manager columns. I learnt something today :D.

Answer 1

Try opening Task Manager and going to View > Select Columns... then toggle on more of the memory columns. It may well be that the program is using far more memory but not of the type that you are viewing in Task Manager

I think under XP there may be a Virtual Memory column which will be of interest to you

Answer 2

Interesting experiment .. by default the Task Manager shows default working set. There are other memory fields, such as Paged and Unpaged pools and Working sets. Page faults can also tell you that the program is trying allocate memory but failing.