Portable way to detect heap fragmentation in c++ at runtime?

Question

I'm writing a qt-based c++ application and i need to be able to detect memory fragmentation in order to check if the current system can actually sustain the memory load: the program load a big image (15/21 megapixels are the norm) in memory and then perform some filtering on it (w/ sparse matrices). For instance, i'm having memory fragmentation problem in Windows and VMMap has been very helpful in this: the problem was some DLLs (Wacom tablet "wintab32.dll" and the UltraMon app) doesn't get relocated so are splitting the address space at the 0x10000000-0x30000000 VA of the process.

I want to provide the application with some sort of awareness toward the fragmentation problem and wondering if a cross-platform (linux/mac/win32) approach giving the information VMMAP gives already exist.

Marje

To be picky: The existence of a heap is an implementation detail, C++ refer to the free-store. – Ivories

You are right, but i did it on purpose since "heap" seems to be a far more generally accepted term ;) – Marje

It's not just a difference in terminology. The free store doesn't need to be a heap at all. It's merely however the implementation decides to solve memory-allocation requests. – Levo

Answer 1

Short answer: There is no portable way.

Longer answer: How the heap is implemented and how it works is an implementation detail of your implementation that widely differs between platforms, std libraries, and operating systems. You'll have to create a different version for each implementation - provided, the implementation gives you an API to hook into it. (Which I think should be the case for the three platforms you target.)

Houseboy

Answer 2

I think you are overly pessimistic. 21 Megapixels, even assuming a colordepth of 16 bits and an equal-sized alphachannel would take only 168 MB. The available address space on a 32 bit system is measured in gigabytes.

Dalrymple

Since precision is a must, the image are represented in memory with float values (32bits) and there are always three channels (RGB or CiELab) leading to 252MB, but that's not the point. Sometimes the application start with the largest allocatable block to be 1.4Gb, sometimes only 500Mb, and the DLLs causing most of the fragmentation at lower VA is the "wintab32.dll" loading at 0x10000000. The available address space on 32bit systems is measured in gigabytes, but it doesn't matter if you have 3gb total free in small chunks, fragmentation causes bad_alloc on a 4gb-ram machine anyway... – Marje

Unfortunately, it seems that once a DLL get loaded at says 0x10000000, every other DLL get relocated near this VA: disabling the Wacom tablet make it possible for other DLLs to get pushed upper, but whenever a lower limit get used then other DLLs (such as uxtheme.dll) get loaded near it. – Marje

If you don't mind changing the DLL, you can alter its preferred base. 0x10000000 is a particular poor default nowadays. – Dalrymple

That's right, a really poor choice, unfortunately this DLL cannot be rebased (rebase -v -b 0x10000000 wintab32.dll says it cannot be rebased). – Marje

Answer 3

Would this do what you need?

bool is_contiguous_freestore_available(size_t max)
{
   char* tst = new(std::nothrow) char[max];
   if (tst == null)
      return false;

   delete[] tst;
   return true;
}

Pibgorn

Not really since i would like to detect the situation, although "working", doing that could be very well fragment the memory itself. – Marje