views:

596

answers:

6

I have a C++ program that benchmarks various algorithms on input arrays of different length. It looks more or less like this:

# (1)
for k in range(4..20):
  # (2)
  input = generate 2**k random points
  for variant in variants:
    benchmark the following call
    run variant on input array
  # (3)

Is it possible to reset the whole heap management at (2) to the state it had at (1)? All memory allocated on the heap that was allocated during the program is guaranteed to be freed at (3).

I am using g++ 4.3 on Linux.

Edit: I understand that there is no real garbage collection in C/C++. I want to force the memory allocation to join adjacent empty chunks of memory it has in its free list at (2).

+1  A: 

There is no way of doing this using Standard C++ short of implementing your own versions of new & delete with their own heap management. An alternative would not be to use arrays but use std::vectors instead - you can then use a custom allocator to do the heap management.

anon
Since I am using a library for some variants that uses its own containers, I cannot do this.
Manuel
A: 

There's no automatic way, you have to manually delete whatever is on the heap to get back to the state of (1).

Brian R. Bondy
As far as I understand it, malloc() will have already split off some memory into chunks I used in my program. These chunks should be in a free list but if I have two chunks, each with a size of 1M, they have to be joined if I want to reuse that part of memory when I need 2M.
Manuel
Maybe you want to use realloc? You can't join 2 different blocks of memory because they need to be in sequence in memory.
Brian R. Bondy
@Manuel: No, malloc will not split off memory into chunk. And they will not be put in a free list afterwards, and they will not have a size of 1M. None of that is specified by the language. It's up to the OS to do all this, and it has nothing to do with C/C++
jalf
+1  A: 

What do you mean? There is no garbage collection in C, and certainly no compaction.

To "reset the state of the heap", you have to call free() for every malloc() call. And as I understand your code, you do that already.

Compaction is pretty much impossible. Unlike higher-level languages like Java or C#, you can not change the address of an object, because any pointers to it would be invalidated.

jalf
I understand the issue with the pointers. I want to force the memory allocation system to join adjacent empty memory chunks.
Manuel
There's more then one way to skin a cat - please see my answer :) (I see jpalecek has a similar idea)
RnR
@RnR: heh, true. @Manuel, what you're asking makes no sense. The OS will do whatever it likes, all of the time. On a sane OS it *will* merge together adjacent chunks, as soon as they become available, and you don't need to do anything. But if it doesn't, there is nothing you can do about it
jalf
Your question is outside the scope of the C (and C++) language. C just specifies that "malloc attempts to allocate memory", and "free" frees it. What happens to freed memory, or where allocated memory is taken from is unspecified. Only the OS knows. Which is why RnR's answer works.
jalf
As far as I know, the operating system does not know about the chunks at all.
Manuel
Also AFAIK: There always is a runtime system (call it library) in user space for the memory management, I'm looking for a way to make it join the empty pieces programatically. It will not do so greedily because of the performance overhead.
Manuel
The operating system handles any requests for memory, so yes, it is up to the OS to keep track of what is free and what has been allocated. And it typically odes that by keeping a free list and a list of allocated chunks.
jalf
There is no significant overhead of merging empty blocks. Anyway, the point is that the language has nothing to do with this. It is system-dependant. Whether or not it is 1) necessary to do manually, and 2) possible to do manually, depends on the OS, not the language.
jalf
@jalf: To the best of my knowledge, modern operating systems only care about memory pages, see http://www.kernel.org/doc/man-pages/online/pages/man2/brk.2.html
Manuel
@Manuel: indeed, most OSes allocate with page granularity. So for c/c++ it is libc and libstdc++ which implement malloc/free on top of the page based allocators. However, nearly every allocator i've seen tends to grow on demand but never shrink in order to favor future allocations.
Evan Teran
+3  A: 

I think there's a simple solution to your problem - you could move the outside loop outside of your application and into a shell script or another application and pass the (k) (and any other) parameters through the command line to the benchmarked app - this way you'll be sure all executions had similar starting conditions.

RnR
Heh, +1It's the only sane way to solve the question.
jalf
+5  A: 

If you want the test runs to start in the same heap states, you can run them in their own processes created by fork().

jpalecek
I came up with just issue about this now - using fork the next process will inherit a different state then the previous one - in detail an environment in which n-1 forks have already been performed - my suggestion was first and has the guarantee of exactly the same conditions for all processes ;)
RnR
If you implement it properly, the only difference between the state of the forked processes would be different values in th array of childs' PIDs. No difference in the heap. Unlike the shell script approach, it has the possibility to pass arbitrarily complex structures to the children without fuss.
jpalecek
How do you know what the call to fork will do to the heap? Don't you think memory needs to be allocated/released when new processes are executed? For example to copy the file descriptors etc? It is easier - like I've addmited but it's much less "guaranteed" to be identical - at least imho.
RnR
A: 

Their are a few pieces of garbage collection code out their. Look at perl/python/lua/ruby/mono/parrot/boehm/pike/slate/self/io etc etc. Also look at alloca() and dynamic arrays. Also consider using structs to implement your own destructor or using gcc attributes to call free when a function leaves scope.

nelix