Searching for memory leaks in Apache httpd and modules

Question

What is the best way for finding memory leaks in Apache httpd and httpd modules?

Are there any howtos?

I'v tried valgrind a little, but few obstacles appeared:

1. Valgrind expects for binary to exit normally. I have managed to do that with MaxRequestsPerChild and -X parameter.
2. Valgrind reports about lots of stuff, probably connected with apr pools, but nothing useful.

OS: Linux

P.S.:

Valgrind command: $ valgrind --leak-check=full --leak-resolution=med --log-file=/tmp/valgrind.log ./bin/httpd -X

Valgrind output example: http://paste-it.net/public/x5b6e8b/

Answer 1

I don't know of a magic bullet, but you can have a look at valgrind/valgrind.h , it has some useful macros to make things Valgrind aware and alter their behavior if running under Valgrind.

For instance

#ifndef HAVE_VALGRIND_VALGRIND_H
#define RUNNING_ON_VALGRIND 0
#else
#include <valgrind/valgrind.h>
#endif

if (RUNNING_ON_VALGRIND) {
    printf("Hello, this is Valgrind instance %d\n", RUNNING_ON_VALGRIND);
    /* set debug output annoyingly high */
    /* exit after one request */
}

You can also encase that whole mess with NDEBUG to keep it out of production builds.

That should save you from having to fiddle with the server each time you debug, it will 'just do it' if Valgrind is detected. RUNNING_ON_VALGRIND will expand to the valgrind instance, or remain 0 if not applicable.

For the rest (and I'm imagining you are getting a ton of noise, most likely starting with invalid read of size xx), you can systematically apply suppressions. If you post some of the output, it might be easier to make suggestions for the suppressions you can stick in a file.

Incidentally, the Valgrind user's mailing list is extremely helpful and very tolerant. You can also post your most annoying and irrelevant noise there, you'll get replies with how to suppress it rather quickly.

If all you want is a summary of leaks and the entry points that led to them, it should not be too difficult to shut almost everything else up.

Answer 2

You might give it a try for integrating Bohem GC and let garbage collection to detect memory leaks.

Just see here for howto:
http://www.hpl.hp.com/personal/Hans_Boehm/gc/leak.html

Answer 3

Maybe it's time to refactor the code so that you can run tests outside apache?

If you add unit tests that check code paths that allocate memory, you can verify that all the memory is freed, by running the unit tests under valgrind. That way you don't have to worry about making the full code running under apache handle only small numbers of transactions. Also it'll be easier to test all code paths with unit tests.