managed heap fragmentation

Question

I am trying to understand how heap fragmenation works. What does the following output tell me?

Is this heap overly fragmented?

I have 243010 "free objects" with a total of 53304764 bytes. Are those "free object" spaces in the heap that once contained object but that are now garabage collected?

How can I force a fragmented heap to clean up?

!dumpheap -type Free -stat
total 243233 objects
Statistics:
      MT    Count    TotalSize Class Name
0017d8b0   243010     53304764      Free

Pannier

Answer 1

It depends on how your heap is organized. You should have a look at how much memory in Gen 0,1,2 is allocated and how much free memory you have there compared to the total used memory. If you have 500 MB managed heap used but and 50 MB is free then you are doing pretty well. If you do memory intensive operations like creating many WPF controls and releasing them you need a lot more memory for a short time but .NET does not give the memory back to the OS once you allocated it. The GC tries to recognize allocation patterns and tends to keep your memory footprint high although your current heap size is way too big until your machine is running low on physical memory.

I found it much easier to use psscor2 for .NET 3.5 which has some cool commands like ListNearObj where you can find out which objects are around your memory holes (pinned objects?). With the commands from psscor2 you have much better chances to find out what is really going on in your heaps. Most commands are also available in SOS.dll in .NET 4 as well.

To answer your original question: Yes free objects are gaps on the managed heap which can simply be the free memory block after your last allocated object on a GC segement. Or if you do !DumpHeap with the start address of a GC segment you see the objects allocated in that managed heap segment along with your free objects which are GC collected objects.

This memory holes do normally happen in Gen2. The object addresses before and after the free object do tell you what potentially pinned objects are around your hole. From this you should be able to determine your allocation history and optimize it if you need to. You can find the addresses of the GC Heaps with

0:021> !EEHeap -gc
Number of GC Heaps: 1
generation 0 starts at 0x101da9cc
generation 1 starts at 0x10061000
generation 2 starts at 0x02aa1000
ephemeral segment allocation context: none
 segment     begin allocated  size
02aa0000  02aa1000**  03836a30  0xd95a30(14244400)
10060000  10061000**  103b8ff4  0x357ff4(3506164)
Large object heap starts at 0x03aa1000
 segment     begin allocated  size
03aa0000  03aa1000  03b096f8  0x686f8(427768)
Total Size:              Size: 0x115611c (18178332) bytes.
------------------------------
GC Heap Size:            Size: 0x115611c (18178332) bytes.

There you see that you have heaps at 02aa1000 and 10061000. With !DumpHeap 02aa1000 03836a30 you can dump the GC Heap segment.

!DumpHeap 02aa1000  03836a30  
    Address  MT             Size
    ...
    037b7b88 5b408350       56     
    037b7bc0 60876d60       32     
    037b7be0 5b40838c       20     
    037b7bf4 5b408350       56     
    037b7c2c 5b408728       20     
    037b7c40 5fe4506c       16     
    037b7c50 60876d60       32     
    037b7c70 5b408728       20     
    037b7c84 5fe4506c       16     
    037b7c94 00135de8   519112 Free
    0383685c 5b408728       20     
    03836870 5fe4506c       16     
    03836880 608c55b4       96   
    ....

There you find your free memory blocks which was an object which was already GCed. You can dump the surrounding objects (the output is sorted address wise) to find out if they are pinned or have other unusual properties.

Flin

Can heap fragmentation occur (and eventually causing an OutOfMemoryException) without pinned objects? I understand the GC is the one cleaning up a fragmented heap. Without pinned objects, could the GC still fail to clean up the heap in time for a new memory request? What except pinned objects could cause a fragmented heap that can not be cleanup (in time) when new allocations need to be done? – Pannier

If the allocation rate is very high you can run out of memory. The clr has indeed a Sleep(50) built in to throttle your allocation requests if you are allocating in a tight loop to give the GC a chance to clean up but it may be not enough. – Flin

Answer 2

You have 50MB of RAM as Free space. This is not good.

Having .NET allocating blocks of 16MB from process, we have a fragmentation issue indeed. There are plenty of reasons to fragmentation to occure in .NET.

Have a look here and here. In your case it is possibly a pinning. As 53304764 / 243010 makes 219.35 bytes per object - much lower then LOH objects.

Raman

Ok, I have 50MB of free space, but what does that Count tell me? I know I am frequently creating lots of small object (i.e. creating a grid of ~ 20000 WPF controls every 10 seconds, but the old ones become unreferenced, no memory leak here). Could that explain it or should pinned objects be more likely. – Pannier