How do I force my std::map to deallocate memory used?

Question

I'm using a std::map, and I can't seem to free the memory back to the OS. It looks like,

int main(){
  aMap m;

  while(keepGoing){
    while(fillUpMap){
       //populate m
    }
    doWhatIwantWithMap(m);
    m.clear();//doesnt free memory back to OS

    //flush some buffered values into map for next iteration
    flushIntoMap(m);
  }
}

Each (fillUpmap) allocates around 1gig, so I'm very much interested in getting this back to my system before it eats up all my memory.

Ive experienced the same with std::vector, but there I could force it to free by doing a swap with an empty std::vector. This doesn't work with map.

When I use valgrind it says that all memory is freed, so its not a problem with a leak, since everything is cleared up nicely after a run.

edit:

The flush has to appear after the clear.

Answer 1

If you create the map on the heap (via new), deleting it will free up any memory it used.

Answer 2

That behaviour is normal, the runtime library keeps that memory allocated by the map class avaiable to the process so that the next time it needs to allocate memory it does not have to go to the operating system. It is an run time library optimisation.

Answer 3

m.clear() releases memory back to the heap, but it's common for heap implementations to not release that back to the OS (even when they can, issues such as fragmentation make it hard).

This is how the default allocator works, if you've specified your own allocator for the map, it might have its own cache. However, even in that case, it should be cached in order to be reused immediately anyway.

Maps don't have a concept of capacity vs size the way vectors do.

Answer 4

Maybe you could create a custom allocator, or just use Boost's pool library.

Answer 5

I have done a simple test when I put some data in a std::map and then call std::map::clear().

typedef std::map<int, unit_t,std::less<int>,  
     my_allocator<std::pair<const int, unit_t>, 4 > > contaner_t;
contaner_t keys;
keys[1] = 10;
keys[2] = 20;
keys[3] = 30;
keys.clear();

this is the result of printf inserted in my test:

Allocator # 4, Memory consumption:      56     (allocated   :       56)
Allocator # 4, Memory consumption:     112     (allocated   :       56)
Allocator # 4, Memory consumption:     168     (allocated   :       56)
Allocator # 4, Memory consumption:     112     (deallocated :       56),
Allocator # 4, Memory consumption:      56     (deallocated :       56),
Allocator # 4, Memory consumption:       0     (deallocated :       56),

I think you probably should also check your allocator behavior but I think your default std::allocator in fact deallocates memory as you expect but that memory is not returned to the OS. By the way what OS do you use?

The question here is how you measure can't seem to free the memory back to the OS. and how can you be sure that I could force it to free by doing a swap with an empty std::vector. Are you really sure that allocated memory in fact goes back to OS?

Answer 6

Swap Trick? I know that's how you do it with vectors and the like, but I'v enot done it with maps before.