C++: tiny memory leak with std::map

Question

I am writing a custom textfile-data parser (JSON-like) and I have lost many hours trying to find a tiny memory leak in it.

I am using VC++2008 and the commands _CrtMemCheckpoint and _CrtDumpMemoryLeaks to check for memory leaks.

When I parse any file and then remove it from memory (alongside any other memory claimed), I get a 16 bytes memory leak which looks like this :

{290} normal block at 0x00486AF0, 16 bytes long.
Data: <  H `aH  hH  eH > C0 9A 48 00 60 61 48 00 18 68 48 00 D8 65 48 00

I have managed to narrow the "offending" line of code down to this :

classDefinitions[FastStr(cString)] = classDef;

classDefinitions is an std::map<FastStr, FSLClassDefinition*> and is a private member of my parser class.

FastStr is a simple char* "wrapper" for allowing simple c-strings as key values; it has no memory leaks (no 'new' commands). 'FSLClassDefinition*' is obviously a simple class pointer, so nothing strange there either.

Now here is the catch :

1. this line is executed many times during the parse-process, but I only get a single 16-bytes block leaked.
2. if I parse another file, there is not another 16-bytes memory leak
3. If I remove the parser from memory (by having it in a {} code-block), then recreate it in another code-block and have it parse another file, then I get a second 16-bytes memory leak.

This leads me to suspect that there is a memory leak in std::map; but it could also be my mistake... I am pretty sure that's the offending line because if I stop the parsing before it, there is no memory leak; there is memory leak if I stop the parsing just after this line.

Can anyone comment on this?

Answer 1

Let's get one thing out of the way: there's not a leak in std::map. The code is there for every developer to look at and it would have been caught by now.

If you're correct, I would imagine the leak is in copying classDef or the anonymous FastStr object. But without the code for both, it's too hard to tell. You say they're both pointers, leading me to believe that the line in question is just a symptom, and not the actual problem. How about showing some code?

Answer 2

Because the memory "leak" doesn't scale with repeated use, it's probably not a leak at all, but memory that's allocated by the library and not released until after your memory profiling has finished. Memory is often allocated by libraries, and then reused on subsequent calls. Because the library can't tell which of your calls is the last, it won't free it until your program exits or later.

Answer 3

My gut instinct on this would be to look at FastStr. You say it's a simple char* wrapper, but how does it handle being copied (i.e. is the internal char* copied or recreated)? Can you show us the code for FastStr please?

Other than that your evidence as listed suggests some static data or if the 'parser' you mention is only created one in your test code blocks, then a member of the 'parser' object is the likely source of the memory leak.

Another suggestion would be to run your code under a more descriptive tool such as valgrind to pinpoint the leak? Valgrind (or Purify) will tell you the precise location of the memory leak in the code.

Answer 4

So far, we haven't seen a single line of your code, so it's pretty much impossible for us to say much more than that "in all widely-used STL implementations, std::map does not leak memory, under any circumstances." Once the map's destructor has run, all of its memory has been freed.

Of course, if you're using some obscure proprietary STL implementation, all bets are off, but otherwise, map isn't the culprit.

Of course, if you do suspect map of leaking memory, step through it. It's header-only code, so it's visible and can be debugged just like your own code. Step through it in the debugger, see which allocations it makes, and whether they're freed again.

But more likely, the problem is FastStr or... something else in your code.

Try stripping as much as possible away from your code to get the smallest possible example that reproduces the error.

Don't run the full program from the start. If you're sure the problem is in the line you posted, then you can skip all the initial parsing, which would rule out a lot of possibilities. Likewise, remove anything that happens afterwards as well. If that doesn't reproduce the error, then the problem isn't in the line you've isolated.

And when you do get a small sample that reproduces the error, you've also got something you can post here for us to peek through.

Answer 5

Is it possible to run this code under Linux using "valgrind --leak-check" ? If so, valgrind might be able to show you what memory is leaking.

Answer 6

The "{290}" in the leak report is the sequence number of the memory allocation for the memory block that was leaked. If this sequence number is always the same, you can use _crtBreakAlloc to cause a break in the debugger when that allocation sequence number is hit. From the stack trace you can find out where this block is being allocated. Once you know where, and for what purpose, it is being allocated it tends to be fairly easy to determine why it is not being deallocated.

Read the Debug Heap documentation to learn about _crtBreakAlloc.

Answer 7

OldFart provided the ultimate solution to the problem.

There was no memory leak in the first place. The memory locations suggested by the debugger were in the STL xmemory file, line 43 :

return ((_Ty _FARQ *)::operator new(_Count * sizeof (_Ty)));

But they weren't true memory leaks, it was just the VC++2008 debugger being overzealous. I have tested with PurifyPlus (eval.version) and suggests that there are no memory leaks in my program. So, those 16 bytes ARE being removed at program exit, although yeah, it's not very good that the STL doesn't do it earlier.

So again, thanks anyone for your replies, there was never a problem in the first place!

Answer 8

If you put any map on file scope in a DLL, MS Visual Studio 2008 will complain of a memory leak. You don't have to do anything with the map, just declare it:

pound-include <map>

using namespace std;

map < short, long > test_map;

...

Detected memory leaks! Dumping objects -> {143} normal block at 0x00037140, 24 bytes long. Data: <@q @q @q > 40 71 03 00 40 71 03 00 40 71 03 00 CD CD CD CD Object dump complete.

Doesn't happen in main program; doesn't happen if the map is in function scope. It is probably spurious: the map gets deallocated after the memory leak detection stopped.

Answer 9

just to confirm that a static map located on a file scope in a DLL produces the same tiny memory leak situation (using MS Visual Studio 2008).