Accessing memory larger than UINT_MAX*4 size in C?

Question

Hello,

Assume if I have an array of size 78719476736 bytes. mind you that this array is dynamically allocated using malloc in my C code. Assume malloc returns a valid pointer after allocating this much memory. The size of this array is more than UINT_MAX(4294967295) , i.e. max limit of a unsigned int(32 bits)

Assume my code looks like something below e.g.

int *buf;
buf = (int*)malloc(78719476736);

Here 78719476736 is greater than 4 * UINT_MAX.

Now if i have to refer to all the elements of buf, then since buf is int* it will be 32 bit, so it will not be able to address all the memory elements which i have allocated using malloc(78719476736 bytes).

My question is shouldn't the code above be changed to make buf as long long(64 bit variable) as only a long long variable will be able to address the large memory that i have allocated.

Changed code e.g.

unsigned long long int buf;
buf = (unsigned long long int*)malloc(78719476736);

In fact i think, variable buf should not be a pointer any more as any pointer is going to be 32 bit wide and hence it will not be able to access 78719476736 bytes.

So it should be a plain unsigned long long int and i will have to cast the malloc return pointer value to an unsigned long long int as shown in the changed code above and use buf to access all the elements allocated.

Am i correct in my assumptions above ?

or

Am i confusing/missing something?

EDIT: If it helps,

I am working on a Desktop having WinXP on a Intel Core 2 Duo(64 bit CPU). So CPU wise it should not be a problem accessing more than 4 GB address space. What all other components should be enabled for 64 bit support , i.e.

a.) How do i enable compiler support for 64 bit while compiling(I am using Visual Studio 2005 Professional edition)

b.) OS support for 64 bit - I am using Windows XP Professional.

Thank You.

-AD.

Answer 1

1. You need 64 bit OS
2. malloc recieves size_t as parameter that is 64 bit on 64 bit platforms
3. And the most important: You prorbably should think: Do I need to allocate more then 4G of memory?

Answer 2

To address more than 4GB of memory you need some mechanism of selecting that memory. Therefore you need to compile your application to be 64 bit, and then all of your pointers will be 64 bits wide.

The only other way to access more than 4GB of memory would be to use an additional address/memory selector mechanism much like you had with the segment and offset mess back in the real mode DOS days. This of course depends on your processor architecture.

Answer 3

If you run your software on a 64-bit Os and use 64-bit compiler settings, all your pointers will be 64 bit. No need for a special declaration.

Answer 4

An int* is a pointer type. If you're on a system that can allocate 78719476736 bytes, it probably has at least 64 bit addresses i.e. sizeof(int*) >= 8. Pointer size has nothing to do what sizeof(int) is.

Answer 5

I think you are deeply confused about what a pointer is. On common systems at least, the size of a pointer (the number of different values, i.e. the number of different addresses) is independent of the type !

int *a;
short *b;
double *c;

a, b and c are pointers to different types, but they all have the same size (4 bytes on a 32 bits system, for example). That's exactly the point of a 64 bits system compared to 32 bits: to be able to address more than 2**32 locations from a pointer.

Also, a 32 bits CPU cannot address more than 4 Gb in hardware either [1], and the corresponding virtual address space is also generally to limited to 32 bits. So mallocing 8 Gb of memory is quite unlikely to work anyway.

[1] that's not entirely true - Intel CPU for example, have extensions so that a 32 bits CPU can use 'extended' addresses.

Answer 6

It's not possiable, your compiler and OS do not support that, period.

Read up on some options,

http://stackoverflow.com/questions/833234/64-bit-large-mallocs/938583#938583