Is it safe to realloc memory allocated with new?

Question

From what is written here, new allocates in free store while malloc uses heap and the two terms often mean the same thing.

From what is written here, realloc may move the memory block to a new location. If free store and heap are two different memory spaces, does it mean any problem then?

Specifically I'd like to know if it is safe to use

int* data = new int[3];
// ...
int* mydata = (int*)realloc(data,6*sizeof(int));

If not, is there any other way to realloc memory allocated with new safely? I could allocate new area and memcpy the contents, but from what I understand realloc may use the same area if possible.

Coley

Just use a vector. – Scupper

@KarolyHorvath How do you know that's a viable approach in every case? What about deploying an embedded system without standard library support? What about integration with a C interface that may perform a realloc? – Hovercraft

@KarolyHorvath you are welcome to benchmark how much time is takes to allocate 200 MB of memory with malloc (few microseconds) vs. std::vector (~200 millisecond!). std::vector is not a magic solution to every memory problem – Orator

@DavidHaim: I could not believe that but I benchmarked it too and the problem seems to be vector's zero-initialization of the memory. int* n = new int[200 * 1024 * 1024]{}; gives about the same performance on MSVC. – Elyn

@Elyn I didn't say std::vector is less performant than new, I compared it to malloc, which is a great tool when performance is importance. you can always catch a pointer handed by malloc whith unique_ptr and so, and gain that extra 200 milliseconds that you would never get over std::vector or new[] do you think it's legit to spend 200 millisecods zeroing bytes on , let's say, a web server? eventually, std::vector is not an automatic sulotion for C++ reallocations. many developers here said it without thinking twice without even mentioning the side affects. – Orator

@DavidHaim I only pointed out the reason for vector's performance, I'm not saying that's good. Also, the performance of new without zero initialization (if you leave out the {}) is comparable to malloc. Additionally, you can use a vector with similar/same efficiency as malloc if you use reserve and e.g. push_back. – Elyn

@Elyn , yes, other than with malloc you can reallocate, not with new, and reserve zero the bytes as well. – Orator

@DavidHaim: No reserve does not zero the bytes. You are probably confusing that with resize. – Elyn

Guys, what's the matter with you?? I obviously didn't say it's a magical solution or viable approach in every case. I just wrote a (probably) good suggestion. Now if for some reason the OP can't use it, I'm sure he'll mention it. – Scupper

Vector is syntactivally convenient solution, the performance issues are also good point. Thanks for all the comments, it was very enlightening to me. – Rollet

@Karoly I'm sorry, but I don't feel that my response was an unfair reading of your comment. "Just use X" implies it's not even worth considering whether X is a good idea or how it compares to OP's original approach. Also, std::vector, like all classes that follow OOP principles, hides its implementation details, but it would appear that what OP is after is more control rather than less. It's also perhaps a bit insulting, since std::vector is pretty universally known, so it's unlikely that OP asked a question of this depth without considering it as an option. – Hovercraft

Sorry to be so pedantic but since this is C++ don't you need to cast that realloc. I mean shouldn't you write int* mydata = (int*)realloc(data,6*sizeof(int)); – Kropotkin

Answer 1

You can only realloc that which has been allocated via malloc (or family, like calloc).

That's because the underlying data structures that keep track of free and used areas of memory, can be quite different.

It's likely but by no means guaranteed that C++ new and C malloc use the same underlying allocator, in which case realloc could work for both. But formally that's in UB-land. And in practice it's just needlessly risky.

---

C++ does not offer functionality corresponding to realloc.

The closest is the automatic reallocation of (the internal buffers of) containers like std::vector.

The C++ containers suffer from being designed in a way that excludes use of realloc.

---

Instead of the presented code

int* data = new int[3];
//...
int* mydata = (int*)realloc(data,6*sizeof(int));

… do this:

vector<int> data( 3 );
//...
data.resize( 6 );

However, if you absolutely need the general efficiency of realloc, and if you have to accept new for the original allocation, then your only recourse for efficiency is to use compiler-specific means, knowledge that realloc is safe with this compiler.

Otherwise, if you absolutely need the general efficiency of realloc but is not forced to accept new, then you can use malloc and realloc. Using smart pointers then lets you get much of the same safety as with C++ containers.

Lavatory

the snippet you wrote is the most idiomatic way to reallocate memory in C++, but It's a sure way to kill your performance, If you are on this field. – Orator

@KyleStrand: If you have to accept new for the original allocation, then your only recourse for efficiency is to use compiler-specific means. E.g. knowledge that realloc is safe with this compiler. Otherwise, you can use smart pointers with malloc and realloc. Anyway, do remember the first (and second) rule of optimization, namely to MEASURE. – Lavatory

Using smart pointers does require using a custom deleter to invoke free() instead of delete, though, right? – Hovercraft

@KyleStrand: With the standard library's smart pointers, yes, you need a custom deleter for built-in types. For class types a nice alternative is to redefine the type's allocation and deallocation functions (in an effort to confuse as many beginners as possible they're named respectively operator new[] and operator delete[], and just to confuse things beyond the possible they're static but with deallocation effectively acting as if it's virtual). A third option is to define your own smart pointer from scratch, for which you might find boost::intrusive_ptr helpful. – Lavatory

This poses an interesting dilemma for standard library authors who are implementing std::vector: since realloc would be convenient, should they use malloc or the more idiomatic new? – Horseweed

@NateEldredge: There is no choice. std::vector uses a std::allocator for its memory management. – Lavatory

@Cheersandhth.-Alf: But act as if they are virtual? More pedantic, they act as virtual as the dtor. – Tombaugh

Answer 2

The only possibly relevant restriction C++ adds to realloc is that C++'s malloc/calloc/realloc must not be implemented in terms of ::operator new, and its free must not be implemented in terms of ::operator delete (per C++14 [c.malloc]p3-4).

This means the guarantee you are looking for does not exist in C++. It also means, however, that you can implement ::operator new in terms of malloc. And if you do that, then in theory, ::operator new's result can be passed to realloc.

In practice, you should be concerned about the possibility that new's result does not match ::operator new's result. C++ compilers may e.g. combine multiple new expressions to use one single ::operator new call. This is something compilers already did when the standard didn't allow it, IIRC, and the standard now does allow it (per C++14 [expr.new]p10). That means that even if you go this route, you still don't have a guarantee that passing your new pointers to realloc does anything meaningful, even if it's no longer undefined behaviour.

Crandell

Please add references for (1) "C++'s malloc/calloc/realloc must not be implemented in terms of ::operator new", and for (2), about the in-practice not yet endorsed by standard, that "C++ compilers may e.g. combine multiple new expressions to use one single ::operator new call". – Lavatory

@Cheersandhth.-Alf Added a reference for the first. Haven't included the actual standard text because this is not a [language-lawyer] question. I don't have an example ready for multiple new calls that gives the results I'm describing, and a quick and simple example that just deletes the allocated memory doesn't combine the allocations into one, it just optimises away the allocations entirely. – Crandell

Answer 3

In general, don't do that. If you are using user defined types with non-trivial initialization, in case of reallocation-copy-freeing, the destructor of your objects won't get called by realloc. The copy constructor won't be called too, when copying. This may lead to undefined behavior due to an incorrect use of object lifetime (see C++ Standard §3.8 Object lifetime, [basic.life]).

1 The lifetime of an object is a runtime property of the object. An object is said to have non-trivial initialization if it is of a class or aggregate type and it or one of its members is initialized by a constructor other than a trivial default constructor. [ Note: initialization by a trivial copy/move constructor is non-trivial initialization. —end note ]

The lifetime of an object of type T begins when:

— storage with the proper alignment and size for type T is obtained, and

— if the object has non-trivial initialization, its initialization is complete.

The lifetime of an object of type T ends when:

— if T is a class type with a non-trivial destructor (12.4), the destructor call starts, or

— the storage which the object occupies is reused or released.

And later (emphasis mine):

3 The properties ascribed to objects throughout this International Standard apply for a given object only during its lifetime.

So, you really don't want to use an object out of its lifetime.

Cacodyl

Answer 4

It is not safe, and it's not elegant.

It might be possible to override new/delete to support the reallocation, but then you may as well consider to use the containers.

Halutz

I'm not sure what's inelegant about realloc. – Hovercraft

using new/delete with realloc, by override, or other means to make it work, is not elegant, please read the topic. – Halutz

So you mean that because it's not safe, it's inelegant to try to make it safe? That's not clear from your answer. And don't assume I have somehow managed to leave a comment on your answer without "reading the topic"; that's pointlessly insulting. – Hovercraft

Answer 5

In general, no.

There are a slew of things which must hold to make it safe:

1. Bitwise copying the type and abandoning the source must be safe.
2. The destructor must be trivial, or you must in-place-destruct the elements you want to deallocate.
3. Either the constructor is trivial, or you must in-place-construct the new elements.

Trivial types satisfy the above requirements.

In addition:

1. The new[]-function must pass the request on to malloc without any change, nor do any bookkeeping on the side. You can force this by replacing global new[] and delete[], or the ones in the respective classes.
2. The compiler must not ask for more memory in order to save the number of elements allocated, or anything else.
   There is no way to force that, though a compiler shouldn't save such information if the type has a trivial destructor as a matter of Quality of Implementation.

Tombaugh

Answer 6

Yes - if new actually called malloc in the first place (for example, this is how VC++ new works).

No otherwise. do note that once you decide to reallocate the memory (because new called malloc), your code is compiler specific and not portable between compilers anymore.

(I know this answer may upset many developers, but I answer depends on real facts, not just idiomaticy).

Orator

Is that true for operator new[](), which is what is used here, rather than plain operator new()? – Mickens

on VC++ all standard new operators call malloc eventually. – Orator

Yes, but I would be surprised if the result of operator new[] was the same as the value returned by a call to malloc, because of storing the count. And if it isn't then you can't pass it to realloc. – Mickens

the thing that stores the count is the heap entry, and the count is bytes-count, not object count. so allocation wise, there is not difference between new() and new[], both call malloc which calls HeapAlloc. – Orator

That's only true if new[] returns the result of malloc directly without prepending the size of the array (which is needed for a non-trivial destructor) – Settler

The "real facts" are the rules as set by the standard, not random implementation details and assumptions about the compiler and its behavior when meeting this specific flavor of UB. This is not safe, without any "ifs". – Privy

@ratchetfreak No, the runtime can calculate the size of the array from the byte count and the size of the type being destroyed. – Hovercraft

@BaummitAugen No, compiler vendors are free to provide additional guarantees beyond those required by the standard. – Hovercraft

@KyleStrand the delete[] operator doesn't have any clean way to get the byte count of the original allocation so it has no way of knowing how many elements were allocator unless new[] adds it to the allocated memory. – Settler

@ratchetfreak C permits allocating arbitrarily sized data with malloc and freeing it with free, so malloc must include the byte count. Since both malloc and new[]/delete[] are implementation-provided, there does not need a "clean" way for delete[] to get the count; it is sufficient for it to know where it is stored. – Hovercraft

@KyleStrand But you can provide the implementation of delete[] which means that the implementation has no choice but to add that redundant information again. – Settler

@ratchetfreak You wrote that "prepending the size of the array...is needed for a non-trivial destructor". If you provide delete[], you must also provide new[], in which case you can do whatever you want. But the implementations provided by the language implementation do not have the restriction you claim. – Hovercraft

@ratchetfreak Actually, as pointed out here, implementations are generally not able to compute the array size from the size of the allocation. I am not sure whether this implies that implementations can't be designed to do such a computation, though. – Hovercraft

@ratchetfreak I talked to an LLVM dev the other day who confirmed that indeed this is how new[] and delete[] are implemented by default. – Hovercraft

Answer 7

That is not safe. Firstly the pointer you pass to realloc must have been obtained from malloc or realloc: http://en.cppreference.com/w/cpp/memory/c/realloc.

Secondly the result of new int [3] need not be the same as the result of the allocation function - extra space may be allocated to store the count of elements.

(And for more complex types than int, realloc wouldn't be safe since it doesn't call copy or move constructors.)

Mickens

Answer 8

You may be able to (not in all cases), but you shouldn't. If you need to resize your data table, you should use std::vector instead.

Details on how to use it are listed in an other SO question.

Cramped

Answer 9

These function is mostly used in C.

memset sets the bytes in a block of memory to a specific value.

malloc allocates a block of memory.

calloc, same as malloc. Only difference is that it initializes the bytes to zero.

In C++ the preferred method to allocate memory is to use new.

C: int intArray = (int*) malloc(10 *sizeof(int)); C++: int intArray = new int[10];

C: int intArray = (int*) calloc(10 *sizeof(int)); C++: int intArray = new int10;

Geest

I don't believe this answers the question, because it doesn't address reallocation at all. – Hovercraft