Anatomy of a "Memory Leak"

Question

In .NET perspective:

• What is a Memory Leak?
• How to understand whether your application leaks? What are the effects?
• How to prevent a memory leak?
• If your application has memory leak, does it go away when the process exits or killed? Or do memory leaks in your application affects other processes on the system even after process completion?
• And what about unmanaged code accessed via COM Interop and/or P/Invoke?

These are the questions that I have some answers myself but not complete. What do you think?

Answer 1

I guess in a managed environment, a leak would be you keeping an unnecessary reference to a large chunk of memory around.

Answer 2

I will concur with Bernard as to in .net what a mem leak would be.

You could profile your application to see its memory use, and determine that if its managing a lot of memory when it should not be you could say it has a leak.

In managed terms I will put my neck on the line to say it does go away once the process is killed/removed.

Unmanaged code is it's own beast and if a leak exists within it, it will follow a standard mem. leak definition.

Answer 3

The best explanation I've seen is in Chapter 7 of the free Foundations of Programming ebook.

Basically, in .NET a memory leak occurs when referenced objects are rooted and thus cannot be garbage collected. This occurs accidentally when you hold on to references beyond the intended scope.

You'll know that you have leaks when you start getting outofmemoryexceptions or your memory usage goes up beyond what you'd expect (perfmon has nice memory counters).

Understanding .NET's memory model is your best way of avoiding it. Specifically, understanding how the garbage collector works and how references work (again, I refer you to chapter 7 of the ebook). Also, be mindful of common pitfalls, probably the most common being events. If object A registered to an event on object B, then object A will stick around until object B disappears because B holds a reference to A. The solution is to unregister your events when you're done.

Of course, a good memory profile will let you see your object graphs and explore the nesting/referencing of your objects to see where references are coming from and what root object is responsible (red-gate ants profile, JetBrains dotTrace, memprofiler are really good choices, or you can use the text-only windbg and sos, but I'd strongly recommend a commercial/visual product unless you're a real guru).

I believe unmanaged code is subject to typical memory leaks of unamanged code, except that references shared between the two are managed by the garbage collector. Could be wrong about this last point.

Answer 4

Strictly speaking, a memory leak is consuming memory that is "no longer used" by the program.

"No longer used" has more than one meaning, it could mean "no more reference to it", that is, totally unrecoverable, or it could mean, referenced, recoverable, unused but the program keeps the references anyway. Only the later applies to .Net for perfectly managed objects. However, not all classes are perfect and at some point an underlying unmanaged implementation could leak resources permanently for that process.

In all cases, the application consumes more memory than strictly needed. The sides effects, depending on the ammount leaked, could go from none, to slowdown caused by excessive collection, to a series of memory exceptions and finally a fatal error followed by forced process termination.

You know an application has a memory problem when monitoring shows that more and more memory is allocated to your process after each garbage collection cycle. In such case, you are either keeping too much in memory, or some underlying unmanaged implementation is leaking.

For most leaks, resources are recovered when the process is terminated, however some resources are not always recovered in some precise cases, GDI cursor handles are notorious for that. Of course, if you have an interprocess communication mechanism, memory allocated in the other process would not be freed until that process frees it or terminates.

Answer 5

I would define memory leaks as an object not freeing up all the memory allocated after it has completed. I have found this can happen in your application if you are using Windows API and COM (i.e. unmanaged code that has a bug in it or is not being managed correctly), in the framework and in third party components. I have also found not tiding up after using certain objects like pens can cause the issue.

I personally have suffered Out of Memory Exceptions which can be caused but are not exclusive to memory leaks in dot net applications. (OOM can also come from pinning see Pinning Artical). If you are not getting OOM errors or need to confirm if it is a memory leak causing it then the only way is to profile your application.

I would also try and ensure the following:

a) Everything that implements Idisposable is disposed either using a finally block or the using statement these include brushes, pens etc.(some people argue to set everything to nothing in addition)

b)Anything that has a close method is closed again using finally or the using statement (although I have found using does not always close depending if you declared the object outside the using statement)

c)If you are using unmanaged code/windows API's that these are dealt with correctly after. (some have clean up methods to release resources)

Hope this helps.

Answer 6

All memory leaks are resolved by program termination.

Leak enough memory and the Operating System may decide to resolve the problem on your behalf.

Answer 7

I guess in a managed environment, a leak would be you keeping an unnecessary reference to a large chunk of memory around.

Absolutely. Also, not using the .Dispose() method on disposable objects when appropriate can cause mem leaks. The easiest way to do it is with a using block because it automatically executes .Dispose() at the end:

StreamReader sr;
using(sr = new StreamReader("somefile.txt"))
{
    //do some stuff
}

And if you create a class that is using unmanaged objects, if you're not implementing IDisposable correctly, you could be causing memory leaks for your class's users.

Answer 8

Also keep in mind that .NET has two heaps, one being the large object heap. I believe objects of roughly 85k or larger are put on this heap. This heap has a different lifetime rules than the regular heap.

If you are creating large memory structures (Dictionary's or List's) it would prudent to go lookup what the exact rules are.

As far as reclaiming the memory on process termination, unless your running Win98 or it equivalents, everything is released back to the OS on termination. The only exceptions are things that are opened cross-process and another process still has the resource open.

COM Objects can be tricky tho. If you always use the IDispose pattern, you'll be safe. But I've run across a few interop assemblies that implement IDispose. The key here is to call Marshal.ReleaseCOMObject when you're done with it. The COM Objects still use standard COM reference counting.

Answer 9

If you need to diagnose a memory leak in .NET, check these links:

http://msdn.microsoft.com/en-us/magazine/cc163833.aspx

http://msdn.microsoft.com/en-us/magazine/cc164138.aspx

Those articles describe how to create a memory dump of your process and how to analyze it so that you can first determine if your leak is unmanaged or managed, and if it is managed, how to figure out where it is coming from.

Microsoft also has a newer tool to assist with generating crash dumps, to replace ADPlus, called DebugDiag.

http://www.microsoft.com/downloads/details.aspx?FamilyID=28bd5941-c458-46f1-b24d-f60151d875a3&displaylang=en

Answer 10

I think the "what is a memory leak" and "what are the effects" questions have been answered well already, but I wanted to add a few more things on the other questions...

How to understand whether your application leaks

One interesting way is to open perfmon and add traces for # bytes in all heaps and # Gen 2 collections , in each case looking just at your process. If exercising a particular feature causes the total bytes to increase, and that memory remains allocated after the next Gen 2 collection, you might say that the feature leaks memory.

How to prevent

Other good opinions have been given. I would just add that perhaps the most commonly overlooked cause of .NET memory leaks is to add event handlers to objects without removing them. An event handler attached to an object is a form of reference to that object, so will prevent collection even after all other references have gone. Always remember to detach event handlers (using the -= syntax in C#).

Does the leak go away when the process exits, and what about COM interop?

When your process exits, all memory mapped into its address space is reclaimed by the OS, including any COM objects served from DLLs. Comparatively rarely, COM objects can be served from separate processes. In this case, when your process exits, you may still be responsible for memory allocated in any COM server processes that you used.

Answer 11

Using CLR Profiler from Microsoft http://www.microsoft.com/downloads/details.aspx?familyid=86ce6052-d7f4-4aeb-9b7a-94635beebdda&displaylang=en is a great way to determine which objects are holding memory, what execution flow leads to the creation of these objects, and also monitoring which objects live where on the heap (fragmentation, LOH, etc.).

Answer 12

I found .Net Memory Profiler a very good help when finding memory leaks in .Net. It's not free like the Microsoft CLR Profiler, but is faster and more to the point in my opinion. A

Answer 13

The best explanation of how the garbage collector works is in Jeff Richters CLR via C# book, (Ch. 20). Reading this gives a great grounding for understanding how objects persist.

One of the most common causes of rooting objects accidentally is by hooking up events outisde a class. If you hook up an external event

e.g.

SomeExternalClass.Changed += new EventHandler(HandleIt);

and forget to unhook to it when you dispose, then SomeExternalClass has a ref to your class.

As mentioned above, the SciTech memory profiler is excellent at showing you roots of objects you suspect are leaking.

But there is also a very quick way to check a particular type is just use WnDBG (you can even use this in the VS.NET immediate window while attached):

.loadby sos mscorwks !dumpheap -stat -type

Now do something that you think will dispose the objects of that type (e.g. close a window). It's handy here to have a debug button somewhere that will run System.GC.Collect() a couple of times.

Then run !dumpheap -stat -type again. If the number didn't go down, or didn't go down as much as you expect, then you have a basis for further investigation. (I got this tip from a seminar given by Ingo Rammer).

Answer 14

Why do people think that an memory leak in .net is not the same as an other leak?

An memory leak is when you attach to an resource and do not let it go, that you can do both in managed and in unmanaged coding.

Regarding .net, and other programming tools, there have been ideas about garbage collecting, and other ways of minimizing situations that will make your application leak. But the best method of preventing memory leaks is that you need to understand your underlying memory model, and how things works, on the platform you are using. Believing that GC and other magic will clean up your mess is the short way to memory leaks, and will be difficult to find later.

When coding unmanaged, you normally make sure to clean up, you know that the resources you take hold of, will be your responsibility to clean up, not the janitors.

In .net on the other hand, lot of people think that GC will clean up everything. Well, it does some for you, but you need to make sure that it is so. .net do wrap lots of things, so you do not always know if you are dealing with an managed or unmanaged resource, and you need to make sure what you do. Handling fonts, gdi resources, active directory, databases etc is typically things you need to look out for.

In managed terms I will put my neck on the line to say it does go away once the process is killed/removed.

I see lots of people have this though, and I really hope this will end. You cannot ASK the user to terminate your app to clean up your mess! Take a look at an browser, that can be IE, FF etc, then open, say, Google Reader, let it stay for some days, and look at what happes. If you then open another tab in the browser, surf to some site, then close the tab that host the other page, that made the browser leak, do you think the browser will release the memory?, Not so with IE at least.. on my computer, IE easily will eat 1gig memory in short time (about 3-4 days) if I use Google Reader. Some newspages is even worse.