Why malloc always return NULL

Question

My dev environment is VS2008, DX9, Windows XP. I try to add protection handling to the out of memory case. When malloc return NULL, I would page some resource to disk, and release the resources in the memory.

But sometimes, malloc always return NULL, even if I release most of resources and process memory usage and VM size is only 800MB in task manager.

I think to use malloc to allocate 88 bytes should be fine, where process memory usage is only 800MB. But malloc always return NULL.

Could this be memory fragmentation? It doesn't look like that, since process memory usage is not too much.

alt text http://i.imagehost.org/0267/Snap2.jpg

Neile

Exe was launched and attached by VS debugger. – Neile

void* AllocCRT(size_t size) { return malloc(size); } Mem Usage: 644,088K VM Size: 671,064K – Neile

Even I free all the resources, malloc still doesn't work. Is this a HeapAlloc bug??? – Neile

Answer 1

You mentioned memory fragmentation and that would certainly be my first guess. Try downloading this application. Its called Address Space Monitor, and should be able to show you if its a fragmentation issue.

Telegraph

This Monitor will analyze he process address space as seen by OS, while malloc in MSVC works through the Windows Heap API. It doesn't allocate memory directly from system and doesn't return it directly to system. Which means that this Monior won't be able to show you anything. For Monitors like that the entire program heap would look like a solid impenetrable black box of forever-allocated memory. – Cusp

@AndreyT: I think that 'won't show you anything' is a bit strong. The process heap uses the user address space so you can see when that has been exhausted; this doesn't necessarily mean that malloc won't be able to allocate memory that the heap already has reserved. What it should so is when there is free address space that the the heap should be able to claim if it needs to allocate more memory than it already has reserved. – Salyers

Answer 2

It could be virtual address space fragmentation. One way to check is to call HeapCompact(GetProcessHeap(), 0). If that frees up enough memory, then that's the likely cause.

Another similar cause would be launching from the debugger; that causes Windows to use the debug heap, which has really really bad memory behavior over a long period of time. To disable that behavior, set _NO_DEBUG_HEAP=1 in the environment and run.

Fleurette

Answer 3

Another possibility is that there might be a bug in your program. You think you're asking for 88 bytes, but maybe you're passing an uninitialized variable and asking for hundreds of megabytes. Or maybe something you did earlier over-ran a buffer and corrupted the heap, causing malloc() to fail forevermore after that.

Revival