Heap size limitation in C

Question

I have a doubt regarding heap in program execution layout diagram of a C program.

I know that all the dynamically allocated memory is allotted in heap which grows dynamically. But I would like to know what is the max heap size for a C program ??

I am just attaching a sample C program ... here I am trying to allocate 1GB memory to string and even doing the memset ...

    #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>

    int main(int argc, char *argv[])
    {
       char *temp;
       mybuffer=malloc(1024*1024*1024*1);

       temp = memset(mybuffer,0,(1024*1024*1024*1));

       if( (mybuffer == temp) && (mybuffer != NULL))
       printf("%x - %x\n", mybuffer, &mybuffer[((1024*1024*1024*1)-1)]]);
       else
       printf("Wrong\n");

       sleep(20);
       free(mybuffer);
       return 0;
    }

If I run above program in 3 instances at once then malloc should fail atleast in one instance [I feel so] ... but still malloc is successfull.

If it is successful can I know how the OS takes care of 3GB of dynamically allocated memory.

Answer 1

Your machine is very probably overcomitting on RAM, and not using the memory until you actually write it. Try writing to each block after allocating it, thus forcing the operating system to ensure there's real RAM mapped to the address malloc() returned.

Answer 2

From the linux malloc page,

BUGS
       By  default,  Linux  follows  an optimistic memory allocation strategy.
       This means that when malloc() returns non-NULL there  is  no  guarantee
       that  the  memory  really  is available.  This is a really bad bug.  In
       case it turns out that the system is out of memory, one  or  more  pro‐
       cesses  will  be  killed  by the infamous OOM killer.  In case Linux is
       employed under circumstances where it would be less desirable  to  sud‐
       denly lose some randomly picked processes, and moreover the kernel ver‐
       sion is sufficiently recent, one can  switch  off  this  overcommitting
       behavior using a command like:

           # echo 2 > /proc/sys/vm/overcommit_memory

       See  also  the  kernel  Documentation  directory,  files vm/overcommit-
       accounting and sysctl/vm.txt.

Answer 3

You're mixing up physical memory and virtual memory.

http://apollo.lsc.vsc.edu/metadmin/references/sag/x1752.html

http://en.wikipedia.org/wiki/Virtual%5Fmemory

http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory

Answer 4

Malloc will allocate the memory but it does not write to any of it. So if the virtual memory is available then it will succeed. It is only when you write something to it will the real memory need to be paged to the page file.

Calloc if memory serves be correctly(!) write zeros to each byte of the allocated memory before returning so will need to allocate the pages there and then.