OUT OF MEMORY only when virtual limit is hit?

Question

As I know in win32 every program receives say 4GB of virtual memory. Memory manager is responsible for offloading chunks of memory from physical memory to disk.

Does it imply that malloc or any other memory allocation API will throw OUT_OF_MEMORY exception only when virtual limit is hit? I mean is it possible for malloc to fail even if program is far from its virtual size limit e.g. none of physical memory can be offloaded to disk. Assume disk have unlimited capacity and no specific limitation is set.

Answer 1

Yes, it's possible. Remember that memory can be fragmented and that malloc won't be able to find a sufficiently large chunk to serve the size you requested. This can easily be way before you hit your 4 GiB limit.

Answer 2

The virtual memory limit on Win 32 is 2Gb. On Win 64, it's much bigger.

malloc doesn't throw an exception - it returns NULL. NULL return, or exception, the memory manager can fail well before the 2Gb limit is reached if

• The paging file isn't big enough. If the paging file is limited either by policy, or by lack of room to expand: If memory allocations can't be met by page file availability then they will fail.

• Fragmentation. The underlying memory manager allocates memory in 4Kb chunks. Its quite possible, through patterns of allocations and deallocations to end up with a small amount of allocated memory, but a fragmented virtual memory meaning that there is no contiguous area large enough to meet a particular request.

Answer 3

how about allocating a few smaller areas, if a huge one isn't available?

Answer 4

For full chapter and verse on Windows virtual memory check out this post on Mark Russinovich's Blog (lots of other great stuff here too):

Pushing the Limits of Windows: Virtual Memory

If memory fragmentation is your problem and writing custom allocators isn't your thing you could consider enabling the low fragmentation heap:

Low Fragmentation Heap (Windows)

This is on by default these days mind you.