malloc returns NULL and sets errno to ENOMEM, but there is plenty of heap space available?

Question

I have a situation in which malloc() returns NULL and sets errno to ENOMEM. But the CRT heap (which is growable) has plenty of memory to work with. At the time of malloc, my process memory is about 900 MB. The host process is a Java executable executed under the Sun HotSpot JVM.

The malloc() I'm doing is 80 megabytes, and fails. If I do a 60 MB allocation, it succeeds. After that, a 50 MB allocation, followed by another one, and another one also succeed: clearly, I still have a lot of memory left but the 80 MB malloc seems too "big" to digest for the OS.

I'm using Windows 7 x64 SP1 with 4 GB RAM. My process is a 32-bit process, built with VC++ 2010 SP1. I'm using the Low Fragmentation Heap, which is the default on Win 7 - I have also verified with HeapQueryInformation. The VC2010 C Run-Time that I am using creates the heap this way:

HeapCreate(0, BYTES_PER_PAGE, 0)

According to the documentation of HeapCreate, HeapAlloc will automatically call VirtualAlloc for blocks larger than 512KB.

What the heck can cause malloc() to fail if this is not a lack of memory? Is my memory too fragmented? I thought Windows would compact the heap automatically. This is really weird, I have never seen this behavior before.

Another computer with Windows XP SP3 32-bit is exhibiting the same behavior.

Thanks, Martin

Cuvette

this might be relevant, i.e. there is not enough contiguous memory. – Abreaction

Yes, address space fragmentation causes this. 900 MB is much too soon for that to fail, you are probably looking at the working set instead of the VM size. SysInternals' VMMap utility is useful. Not otherwise easy to fix, although a 64-bit operating system is a trivial solution. – Manhunt

Hans, the 900 MB I am talking about is the "Memory (Private Working Set)" value of the Windows Task Manager. Also, the "Peak Working Set (Memory)" column value is about 910 MB. The total CRT heap size, at that point, is about 280 MB, according to the CRT function _heapwalk(). The size of the private Java heap is 330 MB, which makes sense as I explicity specified 330 MB for the Java heap. – Cuvette

Considering my Peak memory usage is only 910 MB, I am very surprised the OS cannot honour my 80 MB allocation, since I am supposed to have 1.1 GB of unused, untouched process memory available. However, I guess the OS allocated the 900 MB over the whole 2 GB address space, and maybe I'm extremely unlucky and there is not a single contiguous 80 MB block available over the whole 2 GB address space ?!!! – Cuvette

Please guys don't let me down just yet :( I have some more information. I ran Process Explorer and when malloc fails, Virtual Memory properties are the following: - Private Bytes: 900 MB - Peak Private Bytes: 910 MB - Virtual Size: 1.5 GB (I guess this is the VM size we're talking about) Running my application with a 32-bit Java VM having the LargeAddressAware flag on Windows 7 x64 gives a maximum Virtual Address Space size (VM size) of 4 GB instead of 2 GB. And the application indeed runs fine. – Cuvette

This boils down to why I have got so much uncommitted but reserved memory. This question has been asked here and has no answer :(((( #4503776 – Cuvette

"Is my memory too fragmented?" Probably. Or, to be exact, your address space is too fragmented. "I thought Windows would compact the heap automatically." How should it do so? malloc() gives you a memory address. No one is allowed to move these, otherwise your program wouldn't find its data any longer... – Sale

Possible duplicate of What's all this uncommitted, reserved memory in my process? – Jeanett