Visual C++: Invalid allocation size. How to force the debugger to stop on this message?

Question

The MFC program I am debugging is printing this message in the "Output" window in Visual Studio 9.0:

HEAP[AppName.exe]: Invalid allocation size - 99999998 (exceeded 7ffdefff)

I'm pretty sure this is due to a bad "new", uninitialized variable or similar error.

The question is: how do I get the debugger to stop on this message so that I can view the stack trace and solve the problem?

---

Edit: The following snippets do not yield the same warning. They produce the Invalid allocation size: 4294967295 bytes. style message instead.

int stupid = -1;
char *bob = new char[stupid];

And:

malloc(-1);

So, I suspect it's coming from within a system DLL with its own memory management or is using a different API.

Answer 1

Try this:

In the menu bar, go to Debug -> Exceptions. Add a C++-Exception called std::bad_alloc and check the checkbox Thrown to stop where the exception was thrown.

Answer 2

Do you know where that function is, or what it's called? Grep the runtime sources for the strings in the error message. Try setting a breakpoint on function call or file/location, in the Debug->New breakpoint dialog.

If that doesn't work, you can break out the big gun (but this is painful and time-consuming) and modify the file where the function is implemented (if it's in a header - if it's in a .cpp file it's even more work). Add this:

__asm {
    int 3;
}

to manually insert a 'break into debugger' opcode. This is a trick I often use to force a breakpoint in a dll for which the debugger fails to identify a location for a breakpoint correctly.

Answer 3

That error message is printed at line 409 of dbgheap.c which is in Program Files\Microsoft Visual Studio 9.0\VC\crt\src. You should be able to just set a breakpoint there.

Answer 4

The error message is probably coming from HeapAlloc() in ntdll.dll.

I can reproduce the message with the following code:

HANDLE hHeap = HeapCreate(0, 0, 4096);
LPVOID p = HeapAlloc(hHeap, 0, 0x99999998);

The message gets sent to the debugger output window by DbgPrint() in ntdll.dll, so I would try setting a breakpoint there (it's an exported function, so you won't need a symbol file to find its address) and then looking at your call stack.