Determine size of dynamically allocated memory in C

Question

Is there a way in C to find out the size of dynamically allocated memory?

For example, after

char* p = malloc (100);

Is there a way to find out the size of memory associated with p?

Restivo

sizeof(char) * … is redundant, as char is guaranteed to have a size of 1. – Empennage

@Empennage It still makes it clearer what is going on. Especially when written as malloc(100*sizeof(char)), which follows the usual convention of placing units on the right side of a quantity. – Orford

Actually, I now prefer writing TYPE *ptr = malloc(100 * sizeof *ptr), where TYPE is only written once. This guarantees you are getting an array of 100 elements, even if you change TYPE. – Empennage

@Empennage Is the casting not necessary? – Toscano

Answer 1

comp.lang.c FAQ list · Question 7.27 -

Q. So can I query the malloc package to find out how big an allocated block is?

A. Unfortunately, there is no standard or portable way. (Some compilers provide nonstandard extensions.) If you need to know, you'll have to keep track of it yourself. (See also question 7.28.)

Future

Answer 2

There is no standard way to find this information. However, some implementations provide functions like msize to do this. For example:

• _msize on Windows
• malloc_size on MacOS
• malloc_usable_size on systems with glibc

Keep in mind though, that malloc will allocate a minimum of the size requested, so you should check if msize variant for your implementation actually returns the size of the object or the memory actually allocated on the heap.

Giacometti

Answer 3

The C mentality is to provide the programmer with tools to help him with his job, not to provide abstractions which change the nature of his job. C also tries to avoid making things easier/safer if this happens at the expense of the performance limit.

Certain things you might like to do with a region of memory only require the location of the start of the region. Such things include working with null-terminated strings, manipulating the first n bytes of the region (if the region is known to be at least this large), and so forth.

Basically, keeping track of the length of a region is extra work, and if C did it automatically, it would sometimes be doing it unnecessarily.

Many library functions (for instance fread()) require a pointer to the start of a region, and also the size of this region. If you need the size of a region, you must keep track of it.

Yes, malloc() implementations usually keep track of a region's size, but they may do this indirectly, or round it up to some value, or not keep it at all. Even if they support it, finding the size this way might be slow compared with keeping track of it yourself.

If you need a data structure that knows how big each region is, C can do that for you. Just use a struct that keeps track of how large the region is as well as a pointer to the region.

Cosmogony

While this answer doesn't quite answer the question, I appreciate the explanation of the rational for such a thing not existing. – Multicolor

If the compiler doesn't have any record of how large a block it malloced, what does it do at the matching free? – Belongings

@AndrewLazarus The library implementation might have an indirect way of determining it, without adding anything to the block directly, and it might refuse to share it :) – Eyelid

It could be argued that given such an "indirect way of determining it" exists, and I can't imagine how it could not, this is useful information in the library, that the library should provide an access to. Otherwise it's like the library (= library developer) is mocking the library user ("Sure, I have to know this, but I won't tell you, you have to keep track yourself just because I can't be bothered!") I consider it a fundamental mistake in the C library. – Adage

Answer 4

Here's the best way I've seen to create a tagged pointer to store the size with the address. All pointer functions would still work as expected:

Stolen from: https://mcmap.net/q/24909/-how-can-i-know-the-allocated-memory-size-of-pointer-variable-in-c-duplicate

You could also implement a wrapper for malloc and free to add tags (like allocated size and other meta information) before the pointer returned by malloc. This is in fact the method that a c++ compiler tags objects with references to virtual classes. Here is one working example:

#include <stdlib.h>
#include <stdio.h>

void * my_malloc(size_t s) 
{
  size_t * ret = malloc(sizeof(size_t) + s);
  *ret = s;
  return &ret[1];
}

void my_free(void * ptr) 
{
  free( (size_t*)ptr - 1);
}

size_t allocated_size(void * ptr) 
{
  return ((size_t*)ptr)[-1];
}

int main(int argc, const char ** argv) {
  int * array = my_malloc(sizeof(int) * 3);
  printf("%u\n", allocated_size(array));
  my_free(array);
  return 0;
}

The advantage of this method over a structure with size and pointer

 struct pointer
 {
   size_t size;
   void *p;
 };

is that you only need to replace the malloc and free calls. All other pointer operations require no refactoring.

Bee

you should also redefine realloc – Jaquiss

This answer assumes &ret[1] is "is suitably aligned so that it may be assigned to a pointer to any type of object with a fundamental alignment requirement and then used to access such an object or an array of such objects in the space allocated". It may not be. – Unsustainable

@AndrewHenle it is C standard requirement that it should be "The pointer returned IF the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object with a fundamental alignment requirement and then used to access such an object or an array of such objects in the space allocated (until the space is explicitly deallocated)." – Neper

@Neper My point is that the method this answer advocates likely breaks that requirement. – Unsustainable

@AndrewHenle I don't see how it possibly can break it. C implementation return already aligned memory. Everything is fine here. – Neper

@Neper Everything is not fine here You've never written code for anything but x86 and other very forgiving systems, have you? And it's not even safe on x86 systems. This answer assumes &ret[1] is going to be "suitably aligned" but that's wrong - there are no guarantees that it is. – Unsustainable

In other words, even if malloc() returns properly-aligned memory, the my_malloc() in this answer doesn't because it changes the value returned by malloc() - the calling code is not guaranteed "suitably-aligned" memory. – Unsustainable

@AndrewHenle I'm no expert in alignment requirements, so let me ask you a question: would it be possible to find a totally alignment-safe implementation for this approach, so that the pointer returned by the malloc wrapper is suitable aligned and can be safely assigned to a pointer to any type of object? Maybe using another size for the item count instead of size_t? Is there any C type whose size is such that if you increment a pointer by such size you get another pointer with the same alignment guarantees as the former one? – Bradford

yep: malloc returns memory aligned to 16 bytes on x86_64. To be safe as malloc, you have to add another 16 bytes, not 8. – Maidie

I'm a newcomer to C but I think the approach can be adapted to handle alignment - _aligned_malloc/_aligned_free (WIN) and aligned_alloc/free (UNIX) spring to mind. My question is should the implementation for size allocated be defined in every calling function or within a struct? Within a function means we pass around typed pointers only then recast, add 1 and get the size. Within a struct requires (I believe) passing a struct pointer and the type information... would be grateful for some direction on this – Phelgon

Answer 5

Everyone telling you it's impossible is technically correct (the best kind of correct).

For engineering reasons, it is a bad idea to rely on the malloc subsystem to tell you the size of an allocated block accurately. To convince yourself of this, imagine that you were writing a large application, with several different memory allocators — maybe you use raw libc malloc in one part, but C++ operator new in another part, and then some specific Windows API in yet another part. So you've got all kinds of void* flying around. Writing a function that can work on any of these void*s is impossible, unless you can somehow tell from the pointer's value which of your heaps it came from.

So you might want to wrap up each pointer in your program with some convention that indicates where the pointer came from (and where it needs to be returned to). For example, in C++ we call that std::unique_ptr<void> (for pointers that need to be operator delete'd) or std::unique_ptr<void, D> (for pointers that need to be returned via some other mechanism D). You could do the same kind of thing in C if you wanted to. And once you're wrapping up pointers in bigger safer objects anyway, it's just a small step to struct SizedPtr { void *ptr; size_t size; } and then you never need to worry about the size of an allocation again.

However.

There are also good reasons why you might legitimately want to know the actual underlying size of an allocation. For example, maybe you're writing a profiling tool for your app that will report the actual amount of memory used by each subsystem, not just the amount of memory that the programmer thought he was using. If each of your 10-byte allocations is secretly using 16 bytes under the hood, that's good to know! (Of course there will be other overhead as well, which you're not measuring this way. But there are yet other tools for that job.) Or maybe you're just investigating the behavior of realloc on your platform. Or maybe you'd like to "round up" the capacity of a growing allocation to avoid premature reallocations in the future. Example:

SizedPtr round_up(void *p) {
    size_t sz = portable_ish_malloced_size(p);
    void *q = realloc(p, sz);  // for sanitizer-cleanliness
    assert(q != NULL && portable_ish_malloced_size(q) == sz);
    return (SizedPtr){q, sz};
}
bool reserve(VectorOfChar *v, size_t newcap) {
    if (v->sizedptr.size >= newcap) return true;
    char *newdata = realloc(v->sizedptr.ptr, newcap);
    if (newdata == NULL) return false;
    v->sizedptr = round_up(newdata);
    return true;
}

To get the size of the allocation behind a non-null pointer which has been returned directly from libc malloc — not from a custom heap, and not pointing into the middle of an object — you can use the following OS-specific APIs, which I have bundled up into a "portable-ish" wrapper function for convenience. If you find a common system where this code doesn't work, please leave a comment and I'll try to fix it!

#if defined(__linux__)
// https://linux.die.net/man/3/malloc_usable_size
#include <malloc.h>
size_t portable_ish_malloced_size(const void *p) {
    return malloc_usable_size((void*)p);
}
#elif defined(__APPLE__)
// https://www.unix.com/man-page/osx/3/malloc_size/
#include <malloc/malloc.h>
size_t portable_ish_malloced_size(const void *p) {
    return malloc_size(p);
}
#elif defined(_WIN32)
// https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/msize
#include <malloc.h>
size_t portable_ish_malloced_size(const void *p) {
    return _msize((void *)p);
}
#else
#error "oops, I don't know this system"
#endif

#include <stdio.h>
#include <stdlib.h>  // for malloc itself

int main() {
    void *p = malloc(42);
    size_t true_length = portable_ish_malloced_size(p);
    printf("%zu\n", true_length);
}

Tested on:

• Visual Studio, Win64 — _msize
• GCC/Clang, glibc, Linux — malloc_usable_size
• Clang, libc, Mac OS X — malloc_size
• Clang, jemalloc, Mac OS X — works in practice but I wouldn't trust it (silently mixes jemalloc's malloc and the native libc's malloc_size)
• Should work fine with jemalloc on Linux
• Should work fine with dlmalloc on Linux if compiled without USE_DL_PREFIX
• Should work fine with tcmalloc everywhere

Signor

Answer 6

No, the C runtime library does not provide such a function.

Some libraries may provide platform- or compiler-specific functions that can get this information, but generally the way to keep track of this information is in another integer variable.

Flaggy

Answer 7

Like everyone else already said: No there isn't.

Also, I would always avoid all the vendor-specific functions here, because when you find that you really need to use them, that's generally a sign that you're doing it wrong. You should either store the size separately, or not have to know it at all. Using vendor functions is the quickest way to lose one of the main benefits of writing in C, portability.

Roadrunner

I have seen some projects but there was only one, where the idea "this should be running on something else than what was originally designed for" have occured. Every involved project at the company was tasked to rewrite from the ground on that point. – Vip

You've been lucky, then. I work on a lot of legacy projects, and these are the sorts of problems that tend to be very hard to solve. – Roadrunner

Answer 8

I would expect this to be implementation dependent.
If you got the header data structure, you could cast it back on the pointer and get the size.

Socinus

Which header data structure ? – Dissymmetry

@nos, if an implementation carved out the required size along with a mem-manager header returning the location after the header as the size-allocated memory block. Such a header is likely to store the size itself. If the header typedef was known, the allocated pointer could be moved back to access the header and its field. In such an implementation the memory-manager would itself implement the free operation likewise for low-accounting-maintenance (and high-risk). – Socinus

Depending on how the memory-manager implements accounting for the free operation, it may-or-may-not be feasible to derive the size from the pointer (knowledge of the implementation is however necessary). Some implementations may however choose to give this primitive in the api (in which case internal knowledge will not be required). – Socinus

Answer 9

Quuxplusone wrote: "Writing a function that can work on any of these void*s impossible, unless you can somehow tell from the pointer's value which of your heaps it came from." Determine size of dynamically allocated memory in C"

Actually in Windows _msize gives you the allocated memory size from the value of the pointer. If there is no allocated memory at the address an error is thrown.

int main()
{
    char* ptr1 = NULL, * ptr2 = NULL;
    size_t bsz;    
    ptr1 = (char*)malloc(10);
    ptr2 = ptr1;
    bsz = _msize(ptr2);
    ptr1++;
    //bsz = _msize(ptr1);   /* error */
    free(ptr2);

    return 0;
}

Thanks for the #define collection. Here is the macro version.

#define MALLOC(bsz) malloc(bsz)
#define FREE(ptr) do { free(ptr); ptr = NULL; } while(0)
#ifdef __linux__
#include <malloc.h>
#define MSIZE(ptr) malloc_usable_size((void*)ptr)
#elif defined __APPLE__
#include <malloc/malloc.h>
#define MSIZE(ptr) malloc_size(const void *ptr)
#elif defined _WIN32
#include <malloc.h>
#define MSIZE(ptr) _msize(ptr)
#else
#error "unknown system"
#endif

Ageratum

Answer 10

If you use malloc then you can not get the size.

In the other hand, if you use OS API to dynamically allocate memory, like Windows heap functions, then it's possible to do that.

Unesco

Answer 11

Well now I know this is not answering your specific question, however thinking outside of the box as it were... It occurs to me you probably do not need to know. Ok, ok, no I don't mean your have a bad or un-orthodox implementation... I mean is that you probably (without looking at your code I am only guessing) you prbably only want to know if your data can fit in the allocated memory, if that is the case then this solution might be better. It should not offer too much overhead and will solve your "fitting" problem if that is indeed what you are handling:

if ( p != (tmp = realloc(p, required_size)) ) p = tmp;

or if you need to maintain the old contents:

if ( p != (tmp = realloc(p, required_size)) ) memcpy(tmp, p = tmp, required_size);

of course you could just use:

p = realloc(p, required_size);

and be done with it.

Millham

Answer 12

Note: using _msize only works for memory allocated with calloc, malloc, etc. As stated on the Microsoft Documentation

The _msize function returns the size, in bytes, of the memory block allocated by a call to calloc, malloc, or realloc.

And will throw an exception otherwise.

https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/msize?view=vs-2019

Phelgon

Answer 13

This code will probably work on most Windows installations:

template <class T>
int get_allocated_bytes(T* ptr)
{
 return *((int*)ptr-4);
}

template <class T>
int get_allocated_elements(T* ptr)
{
 return get_allocated_bytes(ptr)/sizeof(T);
}

Activism

Note: This is a C++ answer to a C question. – Leandro

Answer 14

I was struggling recently with visualizing the memory that was available to write to (i.e using strcat or strcpy type functions immediately after malloc).

This is not meant to be a very technical answer, but it could help you while debugging, as much as it helped me.

You can use the size you mallocd in a memset, set an arbitrary value for the second parameter (so you can recognize it) and use the pointer that you obtained from malloc.

Like so:

char* my_string = (char*) malloc(custom_size * sizeof(char));
if(my_string) { memset(my_string, 1, custom_size); }

You can then visualize in the debugger how your allocated memory looks like:

Ioneionesco

This is only guaranteed to actually work on char * pointers. Any other type risks alignment problems and undefined behavior. – Unsustainable

Yes, I agree with that. – Ioneionesco

Answer 15

This may work, a small update in your code:

void* inc = (void*) (++p)
size=p-inc;

But this will result 1, that is, memory associated with p if it is char*. If it is int* then result will be 4.

There is no way to find out total allocation.

Slusher