Can reserved memory cause an Out Of memory exception

Question

We have a 32 bit windows service that leaks memory - OutOfMemory exception is thrown. It is .net 4.0 executable running on windows server 2003. While debugging crash dump files using WinDbg, I see that most of the memory is actually reserved and not commited.

As you can see from WinDbg screenshot, there is 2.5 Gb of unclassified memory usage, and most of it 2.1 Gb is actually reserved memory (MEM_RESERVE). I have experience debugging crush dumps, but this scenario is something new to me. MEM_COMMIT is quit OK - 564.270 Mb, managed heap's size is abt 82 Mb

I also checked native heaps to see, if there are big chunks of data preserved, but couldn't find anything suspicious there either

So my question is - is it possible that MEM_RESERVED might result in OOM exception? If so, how can I debug it, see why/how huge amount of memory is being reserved? Where else would you look to find what might be the problem?

If any other information is required please ask for it, and I will update my post.

Finochio

Answer 1

Yes, reserving memory can trigger OutOfMemoryException. Try allocating a couple of very large byte arrays. The memory for these will not be committed until you write to the content of the arrays. However, you can easily trigger OOM just by allocating these arrays.

I don't know the implementation details, but since VirtualAlloc will fail if it cannot honor a reserve request, I assume the CLR translates this into an exception. I don't see how it could turn a failed reserve request into anything useful, so an exception is a sensible choice.

Comptom

thanks for reply, do you know by any chance how can I see what reserves this huge memory in WinDbg? – Finochio

Assuming the memory is used by managed object, you can inspect these using !dumpheap and !do. However, given that the output of !eeheap shows that the managed heap is small I'm assuming that something else reserved this memory. – Comptom

Could you please specify what else can reserve memory? I'm confused because I would expect that "something" is either on managed heap, or some native resource on native heaps - neither of them point to any suspicious thing. What else might reserve memory? I would be more than glad to check it right away. – Finochio

By something else I meant "native". As I said, if the reserved memory was backing managed objects, you would be able to find those on the managed heap. Since this doesn't appear to be the case a native library or the runtime itself are the candidates, but I can't say for sure. You can get some additional info on CLR allocations by looking at the output of !eeheap. – Comptom

Answer 2

OutOfMemoryException occurs if the system cannot allocate more virtual memory for your application. Reserved memory is virtual memory and therefore accounts to that limit.

You can try that in C++ most easily:

while(::VirtualAlloc(NULL, 65536, MEM_RESERVE, PAGE_READWRITE) != NULL );
std::cout << "All memory reserved. Now check with tools." << std::endl;

If VirtualAlloc() returns NULL, it could not allocate more memoy. In WinDbg this will show

MEM_RESERVE   32307  7f6f2000 (1.991 Gb)  99.59%   99.56%

However, VirtualAlloc() does not allocate memory on the heap, so !heap is not useful in this case and shows only the default process heap:

0:000> !heap
Index   Address  Name      Debugging options enabled
  1:   00440000 

It's the other way round: the heap mamanger uses VirtualAlloc() to get the memory. Also note that .NET does not use the heap manager as well. It also allocates memory directly using VirtualAlloc() and then manages it on its own. So, because you can see it in the output of !heap it's not a .NET issue, it's a native memory issue.

In my naive understanding the gflags setting Enable heap tagging by DLL should be helpful to determine the source of a heap allocation. However, my expectation that the command !heap -t would simply display the name of the DLL which allocated the memory did not come true.

Addison

thank you for input. I tried !heap -t command, but it didn't give anything useful. I'm trying to dig into native memory to see what I can find there. Any other suggestion to check something else are very welcomed. – Finochio