Heap fragmentation and windows memory manager

Question

Hi, I'm having trouble with memory fragmentation in my program and not being able to allocate very large memory blocks after a while. I've read the related posts on this forum - mainly this one. And I still have some questions.

I've been using a memory space profiler to get a picture of the memory. I wrote a 1 line program that contains cin >> var; and took a picture of the memory:

Where on the top arc - green indicates empty space, yellow allocated, red commited. My question is what is that allocated memory on the right? Is it the stack for the main thread? This memory isn't going to be freed and it splits the continuous memory that I need. In this simple 1 line program the split isn't as bad. My actual program has more stuff allocated right in the middle of the address space, and I don't know where it's comming from. I'm not allocating that memory yet.

1. How can I try solve this? I was thinking of switching to something like nedmalloc or dlmalloc. However that would only apply to the objects I allocate explicitly myself, whereas the split shown in the picture wouldn't go away? Or is there a way to replace the CRT allocation with another memory manager?

2. Speaking of objects, are there any wrappers for nedmalloc for c++ so I can use new and delete to allocate objects?

Thanks.

Answer 1

Could it be the executable? It has to be loaded into the address space somewhere ....

As for 2 its pretty easy to override the global new and delete functions ... just define them.

Answer 2

I assume that you are frequently allocating and deallocating objects of varying sizes and that this is what leads to your memory fragmentation issues?

There are various strategies to work around these; the different memory managers you mentioned might help if they can solve the fragmentation issue for you but that would require a little more analysis of the underlying causes of the fragmentation. For example, if you frequently allocate objects of three or four types and these tend to worsen the memory fragmentation issue, you might want to put those into their own memory pools to enable the reuse of memory blocks of the correct size. That way, you should have a set of memory blocks available that fit this particular object and prevent the common scenario that allocation of object X split a memory block that's big enough to hold Y in such a way that you suddenly can't allocate any Ys anymore.

As to (2), I'm not aware of a wrapper around nedmalloc (frankly, I'm not very familiar with nedmalloc) but you can create your own wrappers very easily as you can either create class-specific operators new and delete or even overload/replace the global operators new and delete. I'm not a big fan of the latter but if your allocation "hotspot" consists of a handful of classes, it's usually pretty easy to retrofit them with their own, class-specific operators new and delete.

That said, nedmalloc bills itself as a replacement for the standard malloc/free and at least with the MS compilers, I think the C++ runtime library will forward new/delete to malloc/free so it might well just be a case of building your executable with nedmalloc.

Answer 3

The best way to find out where memory is allocated in your program is to use a debugger. There are allocations for every loaded DLL and the executable itself, and all of them fragment virtual memory. Additionally, using C/C++ libraries and Windows APIs will cause a heap to be created in your application, which will at very least reserve a chunk of virtual memory.

You could for example use VirtualAlloc to reserve a large chunck of virtual memory in a relatively small program, only to find that either the VirtualAlloc fails, or the application fails later when it tries to load a new DLL (etc.) You also can't always control what DLLs will be loaded and where. Many A/V and other products will inject DLLs into all running processes as they start. When this happens, those DLLs often have first pick at load addresses -- that is, their compiled/linked in default will likely be granted. Out of the available 2GB virtual address space of a typical 32-bit Windows application, if a DLL loads smack in the middle of that address space, the largest single allocation/reservation you can acquire will be less 1 GB.

If you use windbg, you can see which regions of memory are consumed, reserved, etc. The lm command will show you the load addresses of all DLLs and the EXE and their range. The !vadump command will show you all of the virtual memory used by the process and the page protections. The page protections are a big hint into what is there. For example in the following (partial) !vadump from a 64-bit calc.exe process, you will see that the first region is simply a range of virtual memory protected from access. (Among other things this keeps you from allocating memory at address 0.) MEM_COMMIT means that memory is backed by RAM or the paging file. PAGE_READWRITE is possibly heap memory, or the data segment of a loaded module. PAGE_READEXECUTE is usually code that is loaded and that will show up in the list produced by lm. MEM_RESERVE means something has called VirtualAlloc to reserve a region of memory, but that it isn't mapped by the Virtual Memory Manager, and so forth...

0:004> !vadump
BaseAddress:       0000000000000000
RegionSize:        0000000000010000
State:             00010000  MEM_FREE
Protect:           00000001  PAGE_NOACCESS

BaseAddress:       0000000000010000
RegionSize:        0000000000010000
State:             00001000  MEM_COMMIT
Protect:           00000004  PAGE_READWRITE
Type:              00040000  MEM_MAPPED

BaseAddress:       0000000000020000
RegionSize:        0000000000003000
State:             00001000  MEM_COMMIT
Protect:           00000002  PAGE_READONLY
Type:              00040000  MEM_MAPPED

I hope that helps explain things. Windbg is a great tool and has many extensions to help you find where memory is used.

If you really care just about the heap, look at !heap.

Answer 4

First, thank you for using my tool. I hope you find it useful and feel free to submit feature requests or contributions.

Typically, thin slices at fixed points in the address space are caused by linked dlls loading at their preferred address. The ones that load high up in the address space tend to be Microsoft operating system dlls. It's more efficient for the operating system if these can all be loaded at their preferred addresses because then the read-only parts of the dlls can all be shared between processes.

The slice that you can see is nothing to worry about, it barely cuts anything out of your address space. As you've noted, there are dlls, though, which load at other points in the address space. IIRC shlwapi.dll is a particularly bad example, loading at about 0x2000000 (again IIRC) which often splits a large portion of the available address space into two smaller pieces. The problem with this is that once the DLL is loaded, there is nothing that you can do to move this allocate space around.

If you link against the DLL (either directly or via another DLL), there is nothing that you can do. If you use LoadLibrary you can get sneaky and reserve its preferred address, forcing it to be relocated - frequently somewhere better in the address space - before releasing that reserved memory. This doesn't always work, though.

Under the hood, Address Space Monitor uses VirtualQueryEx to examine the address space of the process but there is another call from the psapi library which other tools use (e.g. Process Explorer) which can show you which files (including DLLs) are mapped into which parts of the address space.

As you've found, it can be scarily easy to run out of room in a 2GB user address space. Fundamentally, you're best defence against memory fragmentation is simply to not require any large contiguous blocks of memory. Although difficult to retro-fit, designing your applicationg to work with 'medium sized' chunks usually makes substantially more efficient usage of the address space.

Similarly you can use a paging strategy, possibly using memory mapped files or Address Windowing Extensions.

Answer 5

In order to reduce memory fragmentation, you could take advantage of the Windows Low-Fragmentation Heap. We've used this in our product to good effect and haven't had nearly as many memory related problems since doing so.