C++ What's the max number of bytes you can dynamically allocate using the new operator in Windows XP using VS2005?

Question

I have c++ code that attempts to dynamically allocate a 2d array of bytes that measures approx 151MB in size. When I attempt to go back and index through the array, my program crashes in exactly the same place every time with an "Access violation reading location 0x0110f000" error, but the indicies appear to be in range. That leads me to believe the memory at those indicies wasn't allocated correctly.

1) What's the max number of bytes you can dynamically allocate using the new operator?

2) If it is the case that I'm failing to dynamically allocate memory, would it make sense that my code is crashing when attempting to access the array at exactly the same two indicies every time? For some reason, I feel like they would be different every time the program is run, but what do i know ;)

3) If you don't think the problem is from an unsuccessful call to new, any other ideas what could be causing this error and crash?

Thanks in advance for all your help!

*Edit

Here's my code to allocate the 2d array...

#define HD_WIDTH 960
#define HD_HEIGHT 540
#define HD_FRAMES 100

//pHDVideo is a char**
pHDVideo->VideoData = new char* [HD_FRAMES];

for(int iFrame = 0; iFrame < HD_FRAMES; iFrame++)
{
    //Create the new HD frame
    pHDVideo->VideoData[iFrame] = new char[HD_WIDTH * HD_HEIGHT * 3];
    memset(pHDVideo->VideoData[iFrame], 0, HD_WIDTH * HD_HEIGHT * 3);
}

and here's a screenshot of the crashing code and debugger it will help.

I should add that the call to memset never fails, which to me means the allocations is successful, but I could be wrong.

EDIT I found a fix everyone, thanks for all your help. Somehow, and I still need to figure out how, there was one extra horizontal line being upscaled, so I changed...

for(int iHeight = 0; iHeight < HD_HEIGHT; iHeight++)

to

for(int iHeight = 0; iHeight < HD_HEIGHT-1; iHeight++)

and it suddenly worked. Anyhow, thanks so much again!

Answer 1

How are you accessing the allocated memory? Does it always die on the same statement? It looks very much like you're running off the end of either the one dimensional array of pointers, or the one of the big blocks of chars that it's pointing to. As you say, the memset pretty much proves that the memory was allocated correctly. The total amount of memory you're allocating is around 0x9450C00 bytes, so the address you quoted is off the end of allocated memory if it was allocated continguously.

Your screenshot appears to show that iPixel is in range, but it doesn't show what the value of iFrame was. Is it outside the range 0-99?

Update: The bug isn't in allocating the memory, it's in your conversion from HD to SD coordinates. The value you're reading from on the SD buffer is out of range, because it's at coordinates (144,176), which isn't in the range (0,0)-(143,175).

Answer 2

If it is the case that I'm failing to dynamically allocate memory, would it make sense that my code is crashing when attempting to access the array at exactly the same two indicies every time?

No, it wouldn't make sense to me.

If your call to operator new fails, I'd expect it to throw an exception or to return a null pointer (but not to return a non-null pointer to memory that's OK for some indices but not others).

Answer 3

Some possibilities to look at or things to try:

• It may be that the pHDVideo->VideoData[iFrame] or pHDVideo->VideoData is being freed somewhere. I doubt this is the case but I'd check all the places this can happen anyway. Output a debug statement each time you free on of those AND just before your crash statement.
• Something might be overwriting the pHDVideo->VideoData[iFrame] values. Print them out when allocated and just before your crash statement to see if they've changed. If 0x0110f000 isn't within the range of one of them, that's almost certainly the case.
• Something might be overwriting the pHDVideo value. Print it out when allocated and just before your crash statement to see if it's changed. This depends on what else is within your pHDVideo structure.
• Please show us the code that crashes, with a decent amount of context so we can check that out as well.

In answer to your specific questions:

1/ It's implementation- or platform-specific, and it doesn't matter in this case. If your new's were failing you'd get an exception or null return, not a dodgy pointer.

2/ It's not the case: see (1).

3/ See above for some possibilities and things to try.

Following addition of your screenshot:

You do realize that the error message says "Access violation reading ..."?

That means it's not complaining about writing to pHDVideo->VideoData[iFrame][3*iPixel+2] but reading from this->VideoData[iFrame][3*iPixelIndex+2].

iPixelIndex is set to 25458, so can you confirm that this->VideoData[iFrame][76376] exists? I can't see from your screenshot how this->VideoData is allocated and populated.

Answer 4

Why are you using floating point math to calculate an integer index?

It looks like you are indexing out of range of the SD image buffer. 25344==SD_WIDTH*SD_HEIGHT, which is less than iPixelIndex, 25458.

Answer 5

Notice that heap allocation (e.g. using new) is efficient when allocating many small objects (that's why it's a Heap). If you're in the business of very large memory allocations, it might be better to use VirtualAlloc and friends.