Memory fragmentation

Question

When I use malloc()s and free()s randomly, nested and with different sizes, at some point the memory will be fragmented because these operations leave a large list of small memory areas behind that are non-contiguous and therefore can't be allocated as one bigger piece.

A few questions on this:

• When this is done quite often so that memory is forced to be fragmented and then all these memory areas are free()d, can I assume these free areas are concatenated back to its original, contiguous size?

• When I always do a malloc() followed by free() for the same memory and never nest these calls, is the memory fragmented in this scenario too when allocated/freed sizes are always different?

Gasiform

All this is implementation dependant. But, yes, a good memory manager will concatenate contiguous free memory portions. And, yes, if you do a malloc followed by a free and you don't call any library functions that call malloc in between , then no fragmentation should occur. – Minima

You can run Valgrind with your program valgrind.org and check for data alignment errors and memory leaks. – Gargan

It's quite hard to fragment a 64 bit address space. Sure, you can easily fragment it to the point where you can't allocate a 10 TB object anymore, but you probably didn't have 10 TB RAM anyway. – Hutchison

1. no. 2. no. If the memory allocator does ASLR it might actively avoid being as predictable as you want it to be. The solution is to use an architecture with a large address space and ignore fragmentation, with 64 bit pointers and todays large memory sizes fragmentation is mostly yesterdays problem unless you have some very specific application. – Cautery

How would that work anyway? C does not provide a garbage collection, less memory space compaction. It simply can't, as it there is no information about where pointers to objects exist. – Selfrevealing

@Olaf in theory this should not be a problem. The C-library should contain a list of pointers that mark the free/allocated memory areas and their sizes. A clever algorithm then could go through this list after every free() and concatenate single entries that point to memory areas that are located directly behind each other – Gasiform

@Elmi: Please read what "memory (space) compaction means. You talk about combining adjascent block, which is a different thing. That site is not to discuss how to implement a memory manager can/should be implemented. There is already enough information about this, including a plethora of source code. – Selfrevealing

Answer 1

As per ISO/IEC 9899:201x -> 7.22.3

The order and contiguity of storage allocated by successive calls to the aligned_alloc, calloc, malloc, and realloc functions is unspecified.

A good memory manager will be able to tackle the issue to an extent. However, there are other aspects like data alignment [1] which causes internal fragmentation.

What you could do if you rely on inbuilt memory management?

1. Use a profiler - say valgrind - with memory check option to find the memory which is not freed after use. Example:
   
   

    valgrind --leak-check=yes myprog arg1 arg2
   

2. Follow good practices. Example - In C++, if you intend others to inherit from your polymorphic class, you may declare its destructor virtual.

3. Use smart pointers.

Notes:

1. Internal fragmentation.

2. If you were to use your own memory management system, you may consider Boehm-Demers-Weiser garbage collector.

3. Valgrind Instrumentation Framework.

4. Memory not freed after use will contribute to fragmentation.

Lizarraga

Answer 2

No, there is no guarantee. According to N1570, 7.22.3 Memory management functions:

The order and contiguity of storage allocated by successive calls to the aligned_alloc, calloc, malloc, and realloc functions is unspecified.

Anyway, you have two choices to choose from:

1. Totally trust the library memory management functions.
2. Write you own memory managers, if you're really confident.

If I were you, I would definitely trust the existing functions, because modern implementations are super smart.

Radtke

Actually, since the question is (also) tagged C++, the two possibilities are not mutually exclusive. You can override operator new on a class by class basis. – Hutchison