Are there any good memory fragmentation profilers? (linux gcc version would be nice). Valgrind cannot analyze this because it uses custom malloc/free functions.
Thanks, Andrew
+1
A:
I have trouble understanding how any tool you might find would understand the segment data structures of your custom memory management. You might be able to get the busy distribution(hooking into malloc/free) but the free distribution (which essentially is the fragmentation) seems up in the air.
So why not add busy/free statistics/histograms to your custom memory manager. If bins are indexed by something proportional to log2(size) its O(1) to keep these statistics as when you split and coalesce you know the sizes and you can find the bin by direct lookup using an index proportional to log2(size)
eg histogram bin intervals
[2^n,2^(n+1) ) ...
(eg if you want finer bins use log base square root 2(size) which can be calculated with 4 integer instructions on x86 [bit scan, compare, set, add])
another set of reasonable bin sizes to use are the following open intervals
[2^n, 2^n+2^(n-1) ),[2^n+2^(n-1),2^(n+1) )...
again easily calculable [bit scan, shift, and, add])
pgast
2009-09-06 22:57:01
He doesn't mention a custom allocator. It's not that far fetched that someone e.g. wrote a tool that could walk the internal structures of the glibc allocator on linux.
nos
2009-09-10 21:19:15
then what does "custom" in "custom malloc/free functions" mean?
pgast
2009-09-11 00:09:03
The 'custom' refers to valgrind - it would be silly for valgrind to measure malloc fragmentation as valgrind replaces default allocator (You want to measure the real allocator, not the one valgrind hooked in)
nos
2009-11-09 18:46:03
A:
nedmalloc is a very good custom allocator, comes with source, optimized to avoid fragmentation.
I would plug that in, and start looking at its internal logging for fragmentation statistics.
Justicle
2009-09-06 23:29:18
nedmalloc looks like a great way to speed up the program, but it doesn't come close to solving fragmentation profiling problem without lots of work.Thanks for the info, I'm seriously considering switching to it.
Andrew
2009-09-07 06:18:51
The idea was that a) nedmalloc handles fragmentation well, so its less of an issue, b) it has built in tuning and source code, so you can roll your own analysis with far more detailed introspection than something like mtrace.
Justicle
2009-09-07 23:27:44
+2
A:
I would start with mtrace. When you have a trace, glibc comes with a perl script mtrace(1) which finds leaks. However, the trace format is easy to understand, so it should be straight-forward process this into fragmentation analysis.
Martin v. Löwis
2009-09-07 04:58:37