Large Object Heap Fragmentation, Issues with arrays

Question

I am writing an analysis application in C# that has to deal with a lot of memory. I use ANTS Memory Profiler 7.4 for optimizing my memory management. While doing so, I realized that all of my double[,] arrays I use (and i need them) are placed on the LOH although the largest of these arrays is about 24.000 bytes. Objects should not be put there before 85.000 bytes as far as I know. The problem is now, that since I have about several thousand instances of these double[,] arrays I have a lot of memory fragmentation (about 25% of my total memory usage is free memory that I can not use). some of these arrays stored on the LOH are even only 1.036 bytes in size. The problem is that sometimes I have to perform larger analysis and then I end up with an out of memory exception because of the massive memory loss due to LOH fragmentation.

Does anyone know why this is happening although it should not be a large object by definition?

Gareri

have a look at this answer! – Granular

I saw that, but first I do not work with strings here and also the answer about double arrays does not suit my problems since many of my double[,] arrays on the LOH do not even have 6 items. so this really leaves me clueless. but thanks for your answer. I already saw that post before but it did not quite solve my problem. – Gareri

Answer 1

The threshold size for putting arrays of doubles on the LOH is much lower than for other types. The reason for this is that items on the LOH are always 64-bit aligned, and doubles benefit greatly from being 64-bit aligned.

Note that this only affects programs running in 32 bits. Programs running in 64 bits have objects that are always aligned on a 64-bit boundary, so that LOH heuristic is not used for 64 bit programs.

The threshold size is 1000 doubles.

Also see https://connect.microsoft.com/VisualStudio/feedback/details/266330/

Edgeways

as i said, it does also affect double[,] arrays with a size of only 6 entries. so the array that is on the LOH is a double[2,3] array but i still ends up on the LOH (with all the other double[,] arrays that are larger too). But most of my arrays (about 90%) do not exceed the 1000 doubles threshold. ok the ones that i show in the screenshot exceed this limit, but the most are not. what about them? why do they end up there? – Gareri

Does this happen even in a simple program that only allocates one array? Wondering if anything strange is triggering this behaviour. – Edgeways

Yes... It seems like arrays of the type double [,] (multidimensional) are always put on the LOH regardless of their size. Anyone has a clue how to prevent this? because I use them a lot and I would like not to totally redesign the whole project. – Gareri

Multidimensional double arrays follow the same rule in .NET Framework 4.6. Console.WriteLine(GC.GetGeneration(new double[31,31])); outputs 0 and Console.WriteLine(GC.GetGeneration(new double[32,32])); outputs 2. – Celestyn