Heap fragmentation when using byte arrays

Question

I have a C# 4.0 application (single producer/single consumer) which transfers huge amount of data in chunks. Although there's no new memory allocation I run out of memory after a while.

I profiled memory using Redgate memory profiler and there are a lot of free memory there. It says free memory cannot be used because of fragmentation.

I use a blocking collection as the buffer and byte arrays as the members:

BlockingCollection<byte[]> segments = new BlockingCollection<byte[]>(8);
// producer:
segments.Add(buffer);
// consumer:
byte[] buffer = _segments.Take();

How can I avoid managed memory fragmentation?

Zonda

Answer 1

You probably ran into the large object heap problem - objects larger than 85,000 bytes are put on the large object heap which is not compacted which can lead to strange out of memory situations. Although apparently the performance in .NET 4 has been improved it's far from perfect. The solution is to basically use your own buffer pool which contains a few statically allocated chunks of memory and reuse those.
There is a whole bunch of questions around that on SO.

Update: Microsoft provides a buffer manager as part of the WCF stack. There is also one on codeproject.

Morn

Chris is right, the other option is to use objects smaller than 85k so they don't end up allocated on the LOH. – Chaumont

Microsoft solution is good enough for my case. Since it is abstract I should implement mine. Then should I create one static buffer manager for all instances of data transporters or a singleton or an instance member per data transporter (each data transporter lives for 2-3 hours and there are 1000 of them, each uses 8x256 KB memory) – Zonda

Well creating a buffer manager for each transporter sounds like you would just move the heap fragmentation problem to a different place (allocating larger chunks of memory and then releasing it again). You could have a buffer manager pool but I'd go with the singleton first and see if there are any issues with it. You can probably design it so that its hidden behind an interface and each transporter gets its own reference. Then you can easily switch the implementation between singleton and per instance or something in between as you like. – Morn

Answer 2

How long are your byte[] array? Do they fall into the small object or large object heap? If you experience memory fragmentation, I would say they fall into the LOH.

You should therefore reuse the same byte arrays (use a pool) or use smaller chunks. The LOH is never compacted, so it can become quite fragmented. Sadly there is no way around this. (Apart from knowing this limitation and avoiding it)

Convenient

I use the for about 2-3 hours and 8x256 KB, memory profiler says it is large object heap. How can I create a byte array pool? (helpful link is appreciated) – Zonda

Answer 3

The GC doesn’t compact the large object heap for you, you can still programmatically compact it. The following code snippet illustrates how this can be achieved.

GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;
GC.Collect();

Hurst