realloc without freeing old memory

Question

I want to use realloc to increase memory size while keeping the pointer unchanged (because the callers uses it). realloc does not always do that; sometimes it returns a different pointer and frees the old one. I would like to "try" to realloc memory and if it is not possible, fallback to a different method using the original pointer - but realloc has already destroyed that!

Is there a way to try to increase malloc'ed memory without destroying (as realloc does) the old pointer if it is not possible?

E.g.

void *pold;
void *pnew = realloc(pold, newsize);
if (pnew != pold)
{
     free(pnew);
     DoDifferently(pold); // but pold is freed already
}

P.S. I don't care about portability (linux only, thus the tag).

Bartolomeo

Short answer: no (well, not in portable code anyway -- you might be able to dig into the standard library and call some undocumented internal function to do it with one specific version of one standard library). – Trihedral

I think this is a duplicate. Similiar (but not dupe) question: #6696506 – Fontes

@Joe: Yes, his point was try to reallocate in place, and if that's not possible, just fail. – Trihedral

@Joe He wants a function that have this kind of prototype : bool tryrealloc(char * basePtr, size_t newSize); This function return True if the memory can be extended and False if the function cannot but don't destroy the content of the original memory (don't do anything) – Meter

To keep the pointer exactly the same while expanding the area would require that there is enough space free "after" your object. The runtime system can't collect other objects and move them around to make space, as that would change their addresses behind the program's back. – Moramorabito

I know I'm 5 years late to the game here, but you can always write your own thin layer above malloc() that performs large allocations underneath and then, using a set of similar functions, allocates memory within those larger allocations. Basically, managing your own memory allocations. Then you can design the API how you want it, and still have it be cross platform. – Baroda

Answer 1

You should take a look at the source code of realloc() from the libc you are using. From there, it should be easy to see the path followed when it can increase the size in place, and the else case where a new pointer will be returned instead. Then, use that to code your own tryrealloc() function.

For example, this is the realloc() source code from uclibc : http://cristi.indefero.net/p/uClibc-cristi/source/tree/nptl/libc/stdlib/malloc/realloc.c

24  void *
25  realloc (void *mem, size_t new_size)
26  {
...
57    if (new_size > size)
58    /* Grow the block.  */
59    {
60        size_t extra = new_size - size;
61  
62        __heap_lock (&__malloc_heap_lock);
63        extra = __heap_alloc_at (&__malloc_heap, base_mem + size, extra);
64        __heap_unlock (&__malloc_heap_lock);
65  
66        if (extra)
67          /* Record the changed size.  */
68          MALLOC_SET_SIZE (base_mem, size + extra);
69        else
70          /* Our attempts to extend MEM in place failed, just
71             allocate-and-copy.  */
72        {
73          void *new_mem = malloc (new_size - MALLOC_HEADER_SIZE);
74          if (new_mem)
75            {
76              memcpy (new_mem, mem, size - MALLOC_HEADER_SIZE);
77              free (mem);
78            }
79          mem = new_mem;
80        }
81      }
...

I have removed some parts for clarity. But you can see that at line 66, it check if it can simply increase the memory for the current pointer. This is the part you want to keep. The else case starting at line 69 is to handle the case where the old memory will be freed and a new pointer will be returned. This is the part you want to kick out and handle it differently. From what you are saying, I guess you will only want to remove line 77, where it does the free.

If you go this way, remember that you will have to manually either free the old pointer or the new one, as both will now be valid (and you don't want a memory leak).

Also, this is for uclibc. If you are already using a different libc, you should base your new tryrealloc() function on the realloc() function of that libc.

EDIT: You must be careful if you use this approach. You will be basing your solution on the internals of the memory manager, so things can change and be different between the various libc implementations, but also between different versions of the same libc. So do that with the appropriate care and warning in mind.

Tomsk

So you are suggesting he uses __heap_alloc_at (if it is available)? Using the internal functions is not such a good idea as not only they change across library, but also they change between different versions of the same library. It could work today, but most probably not tomorrow. – Hartsfield

@Hartsfield No, I am suggesting that he looks at the libc version he is using, and code his own tryrealloc() function based on that libc's realloc(). But that is true that things may change with different version of the same library, so I'll add a warning on that as well. – Tomsk

Answer 2

I don't think there's a portable realloc-type function that would do this.

One relatively easy portable solution is to pre-allocate a larger block of memory than you need initially. That'll allow for some growth without changing the pointer (in effect, you'll be doing your own in situ realloc).

Once you exceed the original block, you'll have to allocate a new one in its stead. This is your "failure" scenario.

Betz

Answer 3

Such a thing is not possible as far as I know (using the standard library for portability). However, there are easy workarounds. Since your callers use that pointer, you have to fix that for them. One solution is this:

void *do_something(void *your_pointer)
{
    void *new_pointer;
    new_pointer = realloc(your_pointer, ...);
    /* more logic */
    return new_pointer;
}

and tell them to use do_something like this:

new_pointer = do_something(my_pointer);
if (new_pointer) /* if returning NULL is at all possible */
    my_pointer = new_pointer;

---

Alternatively, you can take care of it yourself:

int do_something(void **pointer_to_your_pointer)
{
    void *new_pointer;
    new_pointer = realloc(*pointer_to_your_pointer, ...);
    /* more logic */
    *pointer_to_your_pointer = new_pointer;
    return error_code;
}

and tell them to use it like this:

do_something(&my_pointer);

Hartsfield

No, the API is fixed and I have to keep the pointer unchanged. – Bartolomeo

Answer 4

There is no portable solution to this problem, and the non-portable solution is not free from risk.

The non-portable solution, which works with GNU malloc, is to use malloc_usable_size to find out how big the memory region actually is. However, even if the memory region is big enough, I'm not sure if realloc is not guaranteed to use it. IIRC, there used to be an option which caused realloc to always allocate new memory, but I can't find it any more; I didn't look very hard, though.

Ethiopian

Answer 5

The alternative memory management library jemalloc provides such functionality. With xallocx() it can be done like this:

size_t realsize = xallocx(pold, newsize, 0, 0);
if (realsize < newsize)
{
     DoDifferently(pold);
}

Hoofed

Answer 6

Why not just malloc the maximum amount you might ever need? The memory will usually only be actually allocated once you use it. To fine-tune, you could use mmap to allocate a region of memory.

Irreconcilable

Answer 7

it is not in the programmers hand to reallocate the same pointed memory region.. you can just increase the size using realloc().. you may or may not get the same memory location after using realloc.. most probably you may not.. there may not be any problem if there is a change in memory location..

Watercraft