Anatomy of a "Memory Leak"

Question

In .NET perspective:

• What is a memory leak?
• How can you determine whether your application leaks? What are the effects?
• How can you prevent a memory leak?
• If your application has memory leak, does it go away when the process exits or is killed? Or do memory leaks in your application affect other processes on the system even after process completion?
• And what about unmanaged code accessed via COM Interop and/or P/Invoke?

Proscription

Answer 1

The best explanation I've seen is in Chapter 7 of the free Foundations of Programming e-book.

Basically, in .NET a memory leak occurs when referenced objects are rooted and thus cannot be garbage collected. This occurs accidentally when you hold on to references beyond the intended scope.

You'll know that you have leaks when you start getting OutOfMemoryExceptions or your memory usage goes up beyond what you'd expect (PerfMon has nice memory counters).

Understanding .NET's memory model is your best way of avoiding it. Specifically, understanding how the garbage collector works and how references work — again, I refer you to chapter 7 of the e-book. Also, be mindful of common pitfalls, probably the most common being events. If object A is registered to an event on object B, then object A will stick around until object B disappears because B holds a reference to A. The solution is to unregister your events when you're done.

Of course, a good memory profile will let you see your object graphs and explore the nesting/referencing of your objects to see where references are coming from and what root object is responsible (red-gate ants profile, JetBrains dotMemory, memprofiler are really good choices, or you can use the text-only WinDbg and SOS, but I'd strongly recommend a commercial/visual product unless you're a real guru).

I believe unmanaged code is subject to its typical memory leaks, except that shared references are managed by the garbage collector. I could be wrong about this last point.

Sandpiper

Oh you like book do you? I've seen the author pop up on stackoverflow from time to time. – Highlands

Some .NET objects can also root themselves and become uncollectable. Anything that's IDisposable should be disposed because of this. – Pavo

@Pavo : How does an object root itself? – Senter

@Andrei: I think a running Thread is perhaps the best example of an object rooting itself. An object that puts a reference to itself in a static non-public location (such as subscribing to a static event, or implementing a singleton by static field initialization) may as well have rooted itself, since there's no obvious way to... um... "uproot" it from its mooring. – Groschen

@Jeffry this is an unconvential way to describe whats happening and I like it! – Finagle

Unmanaged/native code is typically more vulnerable to physical leaks but not so much these logical/rooted ones. It's because if we have one of these rooted resources which fail to be "uprooted", typically they will end up getting explicitly destroyed anyway in a language like C. The error that occurs in these cases is more often the dangling pointer and often some kind of segfault associated with accessing it. Though that's sometimes actually preferable to having this rooted resource simply consume memory until the application is shut down (a crash is very obnoxious, easy-to-detect bug). – Whitmer

Answer 2

Strictly speaking, a memory leak is consuming memory that is "no longer used" by the program.

"No longer used" has more than one meaning, it could mean "no more reference to it", that is, totally unrecoverable, or it could mean, referenced, recoverable, unused but the program keeps the references anyway. Only the later applies to .Net for perfectly managed objects. However, not all classes are perfect and at some point an underlying unmanaged implementation could leak resources permanently for that process.

In all cases, the application consumes more memory than strictly needed. The sides effects, depending on the ammount leaked, could go from none, to slowdown caused by excessive collection, to a series of memory exceptions and finally a fatal error followed by forced process termination.

You know an application has a memory problem when monitoring shows that more and more memory is allocated to your process after each garbage collection cycle. In such case, you are either keeping too much in memory, or some underlying unmanaged implementation is leaking.

For most leaks, resources are recovered when the process is terminated, however some resources are not always recovered in some precise cases, GDI cursor handles are notorious for that. Of course, if you have an interprocess communication mechanism, memory allocated in the other process would not be freed until that process frees it or terminates.

Inconsiderate

Answer 3

I think the "what is a memory leak" and "what are the effects" questions have been answered well already, but I wanted to add a few more things on the other questions...

How to understand whether your application leaks

One interesting way is to open perfmon and add traces for # bytes in all heaps and # Gen 2 collections , in each case looking just at your process. If exercising a particular feature causes the total bytes to increase, and that memory remains allocated after the next Gen 2 collection, you might say that the feature leaks memory.

How to prevent

Other good opinions have been given. I would just add that perhaps the most commonly overlooked cause of .NET memory leaks is to add event handlers to objects without removing them. An event handler attached to an object is a form of reference to that object, so will prevent collection even after all other references have gone. Always remember to detach event handlers (using the -= syntax in C#).

Does the leak go away when the process exits, and what about COM interop?

When your process exits, all memory mapped into its address space is reclaimed by the OS, including any COM objects served from DLLs. Comparatively rarely, COM objects can be served from separate processes. In this case, when your process exits, you may still be responsible for memory allocated in any COM server processes that you used.

Felike

Answer 4

I would define memory leaks as an object not freeing up all the memory allocated after it has completed. I have found this can happen in your application if you are using Windows API and COM (i.e. unmanaged code that has a bug in it or is not being managed correctly), in the framework and in third party components. I have also found not tiding up after using certain objects like pens can cause the issue.

I personally have suffered Out of Memory Exceptions which can be caused but are not exclusive to memory leaks in dot net applications. (OOM can also come from pinning see Pinning Artical). If you are not getting OOM errors or need to confirm if it is a memory leak causing it then the only way is to profile your application.

I would also try and ensure the following:

a) Everything that implements Idisposable is disposed either using a finally block or the using statement these include brushes, pens etc.(some people argue to set everything to nothing in addition)

b)Anything that has a close method is closed again using finally or the using statement (although I have found using does not always close depending if you declared the object outside the using statement)

c)If you are using unmanaged code/windows API's that these are dealt with correctly after. (some have clean up methods to release resources)

Hope this helps.

Thoria

Answer 5

If you need to diagnose a memory leak in .NET, check these links:

http://msdn.microsoft.com/en-us/magazine/cc163833.aspx

http://msdn.microsoft.com/en-us/magazine/cc164138.aspx

Those articles describe how to create a memory dump of your process and how to analyze it so that you can first determine if your leak is unmanaged or managed, and if it is managed, how to figure out where it is coming from.

Microsoft also has a newer tool to assist with generating crash dumps, to replace ADPlus, called DebugDiag.

http://www.microsoft.com/downloads/details.aspx?FamilyID=28bd5941-c458-46f1-b24d-f60151d875a3&displaylang=en

Nielsen

Answer 6

Using CLR Profiler from Microsoft http://www.microsoft.com/downloads/details.aspx?familyid=86ce6052-d7f4-4aeb-9b7a-94635beebdda&displaylang=en is a great way to determine which objects are holding memory, what execution flow leads to the creation of these objects, and also monitoring which objects live where on the heap (fragmentation, LOH, etc.).

Trabue

Answer 7

The best explanation of how the garbage collector works is in Jeff Richters CLR via C# book, (Ch. 20). Reading this gives a great grounding for understanding how objects persist.

One of the most common causes of rooting objects accidentally is by hooking up events outisde a class. If you hook up an external event

e.g.

SomeExternalClass.Changed += new EventHandler(HandleIt);

and forget to unhook to it when you dispose, then SomeExternalClass has a ref to your class.

As mentioned above, the SciTech memory profiler is excellent at showing you roots of objects you suspect are leaking.

But there is also a very quick way to check a particular type is just use WnDBG (you can even use this in the VS.NET immediate window while attached):

.loadby sos mscorwks
!dumpheap -stat -type <TypeName>

Now do something that you think will dispose the objects of that type (e.g. close a window). It's handy here to have a debug button somewhere that will run System.GC.Collect() a couple of times.

Then run !dumpheap -stat -type <TypeName> again. If the number didn't go down, or didn't go down as much as you expect, then you have a basis for further investigation. (I got this tip from a seminar given by Ingo Rammer).

Demonstrative

Answer 8

I guess in a managed environment, a leak would be you keeping an unnecessary reference to a large chunk of memory around.

Meany

Answer 9

Why do people think that an memory leak in .NET is not the same as any other leak?

A memory leak is when you attach to a resource and do not let it go. You can do this both in managed and in unmanaged coding.

Regarding .NET, and other programming tools, there have been ideas about garbage collecting, and other ways of minimizing situations that will make your application leak. But the best method of preventing memory leaks is that you need to understand your underlying memory model, and how things works, on the platform you are using.

Believing that GC and other magic will clean up your mess is the short way to memory leaks, and will be difficult to find later.

When coding unmanaged, you normally make sure to clean up, you know that the resources you take hold of, will be your responsibility to clean up, not the janitor's.

In .NET on the other hand, a lot of people think that the GC will clean up everything. Well, it does some for you, but you need to make sure that it is so. .NET does wrap lots of things, so you do not always know if you are dealing with a managed or unmanaged resource, and you need to make sure what what you're dealing with. Handling fonts, GDI resources, active directory, databases etc is typically things you need to look out for.

In managed terms I will put my neck on the line to say it does go away once the process is killed/removed.

I see lots of people have this though, and I really hope this will end. You cannot ask the user to terminate your app to clean up your mess! Take a look at a browser, that can be IE, FF etc, then open, say, Google Reader, let it stay for some days, and look at what happens.

If you then open another tab in the browser, surf to some site, then close the tab that hosted the other page that made the browser leak, do you think the browser will release the memory? Not so with IE. On my computer IE will easily eat 1 GiB of memory in a short amount of time (about 3-4 days) if I use Google Reader. Some newspages are even worse.

Potsherd

Answer 10

I guess in a managed environment, a leak would be you keeping an unnecessary reference to a large chunk of memory around.

Absolutely. Also, not using the .Dispose() method on disposable objects when appropriate can cause mem leaks. The easiest way to do it is with a using block because it automatically executes .Dispose() at the end:

StreamReader sr;
using(sr = new StreamReader("somefile.txt"))
{
    //do some stuff
}

And if you create a class that is using unmanaged objects, if you're not implementing IDisposable correctly, you could be causing memory leaks for your class's users.

Gorgeous

Answer 11

All memory leaks are resolved by program termination.

Leak enough memory and the Operating System may decide to resolve the problem on your behalf.

Darvon

Answer 12

I will concur with Bernard as to in .net what a mem leak would be.

You could profile your application to see its memory use, and determine that if its managing a lot of memory when it should not be you could say it has a leak.

In managed terms I will put my neck on the line to say it does go away once the process is killed/removed.

Unmanaged code is its own beast and if a leak exists within it, it will follow a standard mem. leak definition.

Burkes

Answer 13

Also keep in mind that .NET has two heaps, one being the large object heap. I believe objects of roughly 85k or larger are put on this heap. This heap has a different lifetime rules than the regular heap.

If you are creating large memory structures (Dictionary's or List's) it would prudent to go lookup what the exact rules are.

As far as reclaiming the memory on process termination, unless your running Win98 or it equivalents, everything is released back to the OS on termination. The only exceptions are things that are opened cross-process and another process still has the resource open.

COM Objects can be tricky tho. If you always use the IDispose pattern, you'll be safe. But I've run across a few interop assemblies that implement IDispose. The key here is to call Marshal.ReleaseCOMObject when you're done with it. The COM Objects still use standard COM reference counting.

Mervinmerwin

Answer 14

I found .Net Memory Profiler a very good help when finding memory leaks in .Net. It's not free like the Microsoft CLR Profiler, but is faster and more to the point in my opinion. A

Winsome

Answer 15

One definition is: Unable to release unreachable memory, which can no longer be allocated to new process during execution of allocating process. It can mostly be cured by using GC techniques or detected by automated tools.

For more information, please visit http://all-about-java-and-weblogic-server.blogspot.in/2014/01/what-is-memory-leak-in-java.html.

Garret