Why malloc always return NULL

Question

Hi, guys, My dev environment is VS2008, DX9, Windows XP. I try to add protection handling to the out of memory case. When malloc return NULL, I would page some resource to disk, and release the resources in the memory.

But sometimes, malloc always return NULL, even if I release most of resources and process memory usage and VM size is only 800MB in task manager.

I think to use malloc to allocate 88 bytes should be fine, where process memory usage is only 800MB. But malloc always return NULL.

Could this be memory fragmentation? It doesn't look like that, since process memory usage is not too much.

Answer 1

You mentioned memory fragmentation and that would certainly be my first guess. Try downloading this application. Its called Address Space Monitor, and should be able to show you if its a fragmentation issue.

Answer 2

It could be virtual address space fragmentation. One way to check is to call HeapCompact(GetProcessHeap(), 0). If that frees up enough memory, then that's the likely cause.

Another similar cause would be launching from the debugger; that causes Windows to use the debug heap, which has really really bad memory behavior over a long period of time. To disable that behavior, set _NO_DEBUG_HEAP=1 in the environment and run.

Answer 3

Another possibility is that there might be a bug in your program. You think you're asking for 88 bytes, but maybe you're passing an uninitialized variable and asking for hundreds of megabytes. Or maybe something you did earlier over-ran a buffer and corrupted the heap, causing malloc() to fail forevermore after that.