C malloc : inexplicable memory usage

Question

When I compile and run the following code :(using gcc on cygwin)

int *a = malloc(1024*1024*100*sizeof(int));
while(1)
;

The task manager in Windows XP shows memory usage by this process as 2232K, which according to me should have been around 400000K.

When I compile and run the following code :(using gcc on cygwin)

int *a = malloc(1024*1024*400*sizeof(int));
while(1)
;

the memory usage goes down to 1388K;

So, rather than showing an increase, it actually shows a decline.

What could explain this?

Answer 1

You have allocated the memory, making it available, but have not yet used it (reading or writing from/to it). The memory manager may not have actually allocated the physical memory to your program yet, merely said that you can have it. If you write something across the memory you just allocated (e.g. filling it with 0's -- look at memset for that), I would expect that the memory usage would be more in line with what you expect.

Answer 2

Unfortunately memory consumption is not as simple as a single. There are numerous ways in whtich memory needs to be tracked (and it differs between operating systems a bit).

For instance on Windows, here are some of the different memory usage types

• Virtual Memory
• Physical Memory
• Commited memory
• Reserved Memory
• Shared Memory

Can you give us more details on exactly which number you are talking about?

One possible explanation is that you are looking at the physical memory usage of the process. The operating system will commonly allocate virtual memory address but not commit it to physical memory until it is actually used by your process.

One way to verify this would be to set up a for loop that wrote to every element in the array and then check the memory usage of the application.

Answer 3

The second malloc would allocate 1600MiB (check your units). My guess is that this is more than your system can accommodate in a single process, so the second malloc fails. For some reason, you have a high overhead of other stuff in your application which causes memory usage to be high even though the malloc failed.

Print a to be certain.

Answer 4

The question of allocatable memory using malloc with gcc under cygwin is discussed at http://www.cygwin.com/cygwin-ug-net/setup-maxmem.html.

It would also be good to check the return from malloc.

Answer 5

If you have optimization enabled and a is not acctually used, both the variable and the allocation will be removed. You can avoid this by declaring the variable as volatile.

Answer 6

http://en.wikipedia.org/wiki/Copy-on-write

Another use is in the calloc function. This can be implemented by having a page of physical memory filled with zeroes. When the memory is allocated, the pages returned all refer to the page of zeroes and are all marked as copy-on-write. This way, the amount of physical memory allocated for the process does not increase until data is written. This is typically only done for larger allocations.