Array index limit in C

Question

on linux, with 16 GB of ram, why would the following segfault:

#include    <stdlib.h>

#define N    44000

int main(void) {
    long width = N*2 - 1;
    int * c = (int *) calloc(width*N, sizeof(int));
    c[N/2] = 1;
    return 0;
}

According to gdb the problem is from c[N/2] = 1 , but I do not know the reason

thanks

Answer 1

It's probably because the return value of calloc was NULL.

The amount of physical RAM in your box does not directly correlate to how much memory you can allocate with calloc/malloc/realloc. That is more directly determined by the remaining amount of Virtual Memory available to your process.

Answer 2

You are asking for 2.6GB of RAM (No, you aren't -- you are asking for 14GB on 64 bit... 2.6GB overflowed cutoff calculation on 32 bit). Apparently, Linux's heap is utilized enough that calloc() can't allocate that much at once.

This works fine on Mac OS X (both 32 and 64 bit) -- but just barely (and would likely fail on a different system w/different dyld shared cache & frameworks).

And, of course, it should work dandy under 64 bit on any system (even the 32 bit version with the bad calculation worked, but only coincidentally).

One more detail; in a "real world app", the largest contiguous allocation will be vastly reduced as the complexity and/or running time of the application increases. The more of the heap that is used, the less contiguous space there is to allocate.

Answer 3

Your calculation overflows the range of a 32-bit signed integer, which is what "long" may be. You should use size_t instead of long. This is guaranteed to be able to hold the size of the largest memory block that your system can allocate.

Answer 4

you might want to change the #define to:

#define N    44000L

Just to make sure the math is being done at long resolution. You maybe generating a negative number for the calloc.

Calloc maybe failing and returning null which would cause the problem.

Answer 5

You're allocating around 14-15Gb memory, and for whatever reason the allocator cannot give you that much at the moment- thus calloc returns NULL and you segfault as you're dereferencing a NULL pointer.

Check if calloc returns NULL.

That's assuming you're compiling a 64 bit program under a 64 bit linux, if you're doing something else - you might overflow the calculation to the first argument to calloc if a long is not 64 bits on your system. Try e.g.

#include    <stdlib.h>
#include    <stdio.h>

#define N    44000L

int main(void)
{
    size_t width = N * 2 - 1;
    printf("Longs are %lu bytes. About to allocate %lu bytes\n",
           sizeof(long), width * N * sizeof(int));
    int *c = calloc(width * N, sizeof(int));
    if (c == NULL) {
        perror("calloc");
        return 1;
    }
    c[N / 2] = 1;
    return 0;
}

Answer 6

Create a 14Gb file, and memmap it.

Answer 7

Dollars to donuts calloc() returned NULL because it couldn't satisfy the request, so attempting to deference c caused the segfault. You should always check the result of *alloc() to make sure it isn't NULL.

Answer 8

width * N is 87999 * 44000 = 3871956000. As an int is 8 bytes this takes up 28.85 GB. This is more than the available physical RAM (16 GB). calloc fails allocating the memory, returns the null pointer and the pointer arithmetic and dereference operations are illegal, giving a seg-fault in this particular case.