Is std::memcpy between different trivially copyable types undefined behavior?

Question

I've been using std::memcpy to circumvent strict aliasing for a long time.

For example, inspecting a float, like this:

float f = ...;
uint32_t i;
static_assert(sizeof(f)==sizeof(i));
std::memcpy(&i, &f, sizeof(i));
// use i to extract f's sign, exponent & significand

However, this time, I've checked the standard, I haven't found anything that validates this. All I found is this:

For any object (other than a potentially-overlapping subobject) of trivially copyable type T, whether or not the object holds a valid value of type T, the underlying bytes ([intro.memory]) making up the object can be copied into an array of char, unsigned char, or std​::​byte ([cstddef.syn]).⁴⁰ If the content of that array is copied back into the object, the object shall subsequently hold its original value. [ Example:

#define N sizeof(T)
char buf[N];
T obj;                          // obj initialized to its original value
std::memcpy(buf, &obj, N);      // between these two calls to std​::​memcpy, obj might be modified
std::memcpy(&obj, buf, N);      // at this point, each subobject of obj of scalar type holds its original value

— end example ]

and this:

For any trivially copyable type T, if two pointers to T point to distinct T objects obj1 and obj2, where neither obj1 nor obj2 is a potentially-overlapping subobject, if the underlying bytes ([intro.memory]) making up obj1 are copied into obj2,⁴¹ obj2 shall subsequently hold the same value as obj1. [ Example:

T* t1p;
T* t2p;
// provided that t2p points to an initialized object ...
std::memcpy(t1p, t2p, sizeof(T));
// at this point, every subobject of trivially copyable type in *t1p contains
// the same value as the corresponding subobject in *t2p

— end example ]

So, std::memcpying a float to/from char[] is allowed, and std::memcpying between the same trivial types is allowed too.

Is my first example (and the linked answer) well defined? Or the correct way to inspect a float is to std::memcpy it into a unsigned char[] buffer, and using shifts and ors to build a uint32_t from it?

---

Note: looking at std::memcpy's guarantees may not answer this question. As far as I know, I could replace std::memcpy with a simple byte-copy loop, and the question will be the same.

Subtilize

As long as they have the same size there shouldn't be a problem. However, if you just need to interpret f as uint32_t you may just write (uint32_t&)f. It will interpret the memory location of the float as if it was uint32_t. – Westphal

@NO_NAME My experiment shows that your suggestion violates the strict aliasing rules. coliru.stacked-crooked.com/a/bb54317049f5c8fc – Break

@NickyC I suppose that int i = (uint32_t&)f; wouldn't break the rules. – Westphal

I want to note that the bit_cast proposal's reference implementation uses memcpy and aligned_storage. – Break

Your approach is at least practically correct. This is how boost handles floating point bits boost.org/doc/libs/1_45_0/boost/math/special_functions/detail/… – Polygamy

@Bathsheba I cannot imagine safer way to use of this syntax. Are you telling me that part of C++ syntax is invalid by design and shouldn't be used at all? – Westphal

Related: https://mcmap.net/q/88945/-c-aliasing-rules-and-memcpy – Ballarat

@NickyC: a compiler supplied library code can use anything, because they know how the compiler works, which code with UB can be used. For example, the implementation of std::vector always contained UB (I don't know the current state though, maybe with the intruduction of std::launder, it is not UB any more). – Subtilize

@all Sorry, I was answering a different question. I'll get me coat. – Capeskin

Related: #17790428 – Viva

@NO_NAME It still violates the strict aliasing rules. Valid syntax does not imply valid operation. Just like the English sentence "Colorless green ideas sleep furiously" is grammatically correct but meaningless. – Break

@NickyC correct that definitely breaks the strict aliasing rules – Blithe

Have you considered using the functions for extracting exponent and significand rather than reinventing the (apparently UB) wheel? See frexp(), for instance and perhaps other floating-point manipulation functions in <cmath>. – Ron

@EricTowers: Thanks for the suggestion. It can be a solution in some cases, yes. But if I want to have the significand as an integer, it is no use. Furthermore, this float->int was just an example. – Subtilize

Answer 1

The standard may fail to say properly that this is allowed, but it's almost certainly supposed to be, and to the best of my knowledge, all implementations will treat this as defined behaviour.

In order to facilitate the copying into an actual char[N] object, the bytes making up the f object can be accessed as if they were a char[N]. This part, I believe, is not in dispute.

Bytes from a char[N] that represent a uint32_t value may be copied into an uint32_t object. This part, I believe, is also not in dispute.

Equally undisputed, I believe, is that e.g. fwrite may have written the bytes in one run of the program, and fread may have read them back in another run, or even another program entirely.

Because of that last part, I believe it does not matter where the bytes came from, as long as they form a valid representation of some uint32_t object. You could have cycled through all float values, using memcmp on each until you got the representation you wanted, that you knew would be identical to that of the uint32_t value you're interpreting it as. You could even have done that in another program, a program that the compiler has never seen. That would have been valid.

If from the implementation's perspective, your code is indistinguishable from unambiguously valid code, your code must be seen as valid.

Shattuck

Separating the single steps involved and qualifying each one as undisputed is making things clear. – Predispose

What the OP observed is interesting though: While the standard in 6.9.2 explicitly permits copying bytes out of a a trivially copyable object it lacks (or appears to lack -- I just looked at all occurrences of memcpy in n4659) an explicit rule allowing copying bytes into such an object. It is probably considered self-understood; the example in 6.9.2 itself copies the bytes back, after all. – Predispose

@PeterA.Schneider Right. There is "If the content of that array is copied back into the object, the object shall subsequently hold its original value." which grants permission to copy back into a trivially copyable object, but general permission to copy into (rather than back into) a trivially copyable object is never explicitly given in the standard, it can only be inferred. That's the gist of my answer. – Shattuck

Good reasoning! I've got a question though. A float -> char[] copy is OK. A char[] -> uint32_t is OK too. But, is a direct float -> uint32_t OK too? – Subtilize

@Subtilize It's iffy, but I'd say that since treating the bytes in that float as a char[] is allowed, when you do a direct float -> uint32_t, in a sense, you are copying from a char[] to a uint32_t. – Shattuck

I believe this to be the best possible answer to this question. – Pasha

Interesting... this could also serve as a potential solution to the issue of enumeration types not being officially layout-compatible with their underlying type, among other things. – Hiram

Answer 2

Is my first example (and the linked answer) well defined?

The behaviour isn't undefined (unless the target type has trap representations^† that aren't shared by the source type), but the resulting value of the integer is implementation defined. Standard makes no guarantees about how floating point numbers are represented, so there is no way to extract mantissa etc from the integer in portable way - that said, limiting yourself to IEEE 754 using systems doesn't limit you much these days.

Problems for portability:

• IEEE 754 is not guaranteed by C++
• Byte endianness of float is not guaranteed to match integer endianness.
• (Systems with trap representations^†).

You can use std::numeric_limits::is_iec559 to verify whether your assumption about representation is correct.

^† Although, it appears that uint32_t can't have traps (see comments) so you needn't be concerned. By using uint32_t, you've already ruled out portability to esoteric systems - standard conforming systems are not require to define that alias.

Cranberry

The unsigned integer types cannot have trap representations, can they? I vaguely recall all of their bits must be value bits. – Sofer

@StoryTeller as far as I know, unsigned char is the only type guaranteed to not have traps. – Cranberry

Ah, it's even explicitly stated. Interesting, what I recalled was another Q&A (about C, though) that surmised the uintN_t types cannot have trap representation. I'll see if I can find it. – Sofer

Found the relevant part in the C standard, anyway, port70.net/~nsz/c/c11/n1570.html#7.20.1.1 . Those types do not have any padding bits by definition. So all of their bits must be value bits. No possibility of traps in C, anyway. I wonder if the sentence in the C++ standard is an editorial oversight or there's something deeper... – Sofer

@StoryTeller right, so the exact width types aliases have guarantees (beyond tose for the types that they are an alias for, if they alias the standard int or such). That's kinda odd, but it's always nice to be able to ignore traps, so I'll take it :) – Cranberry

"The behaviour isn't undefined". Why? Is there anything in the standard which makes this defined? – Subtilize

@Subtilize simply because nothing in the standard makes it undefined. – Cranberry

I think if something is not defined by the standard, then it's undefined. Sure, there could be cases, where you can find out some logical working, but it is not enough. For example (as far as I know), the standard doesn't specify what happens for pointer arithmetic, if the pointer doesn't point to an array. So we treat this as UB. Sure, we can find out some logical working for this, but it is still UB. – Subtilize

@Subtilize The (C) standard defines how memcpy behaves, so it is not "not defined". – Cranberry

Sure, it copies bytes. That's all. But we have objects here, not byte arrays. And we have two cases highlighted by the standard. It must mean something, for example it could mean that other use cases are not defined. – Subtilize

The standard doesn't explicitly state behaviour for every single expression. There are cases where the standard lists possible situations and their behaviour, and if some situation is not mentioned in that rule, then that situation is undefined indeed. But can you name a rule which lists behaviours of memcpy in certain situations where your situation isn't included? – Cranberry

Hmm, I think that the standard specifies everything. No wonder it is insanely huge. Name a rule? It is in my question. Two cases of memcpy are listed. My use case is not among them. – Subtilize

@Subtilize Notice that in both rules, memcpy is mentioned only in the example. Examples are not normative, and just because there isn't an example for something, doesn't mean that it isn't defined. Neither of those rules explicitly define how memcpy behaves. Rather, they exemplify the standard rules in terms of memcpy. – Cranberry

Yes, they give an example what they have previously written in the normative text. I could copy bytes with a simple byte-copy loop, the outcome will be the same. So the point is not memcpy here, but the principle. Can I copy a float to uint32_t byte-by-byte by an means? Is it defined by the standard? – Subtilize

@StoryTeller: uint32_t can have traps, but I know of zero architectures for which a uint32_t that you just took an address of that can still trap on the very next read of it. In fact I only know how to get it to trap if it's still never assigned to. – Bonnie

@Bonnie - I find it hard to imagine where one can squeeze in a trap representation when the standard requires all the object's bits are value bits. – Sofer

{ uint32_t value; value = value + 1; } can trap anyway. But this can only trap on the pointer: { char *ptr = something; uint32_t value = *(uint32_t *)pointer; } – Bonnie

@Bonnie - For the addition to trap. The indeterminate value has to be a trap representation to begin with. The C standard implies those don't exist. – Sofer

@StoryTeller: Beware. The Itanium processor actually has this behavior. Uninitialized integers can trap. – Bonnie

@Bonnie - Then it will likely not have uint32_t defined. – Sofer

@StoryTeller: But it does. The Itanium has the general property of any uninitialized integer or pointer in a local variable can trap. – Bonnie

@Bonnie - sigh The abstract machine defined by C standard requires no trap values for the unsigned fixed width integer types. If they exist on an implementation, they must not trap. However it's accomplished is immaterial. To do otherwise is to not be standard compliant. That requirement is in the link I posted in a previous comment. – Sofer

@Bonnie As I understand, the Itanium trap bit is "not a thing" that lives only in the register. I don't think such trap can affect a well defined C++ (or C) program - I mean, you cannot produce a value with the trap bit set wrong, since the bit is not part of the value bits. As long as the C++ program has defined behaviour according to the standard, the implementation is required to set whatever trap bit there may exist to their correct value in order to avoid UB. – Cranberry

Now, if my answer is wrong, and this has UB accroding to C++, then the implementation can leave the bit to whatever value having whatever behaviour the processor will have. – Cranberry

@user2079303: You understand correctly. Notice how the only way I could find to reach it was the uninitialized local variable, which might well have been allocated a register. – Bonnie

Many implementations represent certain data types differently in registers and in memory. It is common on 32-bit RISC platforms, for example, that a uint16_t which is placed in a register will have 16 data bits and 16 padding bits which are zeroed when the value is written. If such an object is read when it is uninitialized, it may yield a value which sometimes behaves like it's outside the range 0-65535 but sometimes behaves like it's within that range. Such behavior goes back decades before Itanium. – Teliospore

@supercat: can you please elaborate more on this issue with RISC? I don't understand the problem fully here (or just name the CPU/platform please, and I'll look into it). – Subtilize

@geza: Many 32-bit RISC platforms don't have any 16-bit registers, nor even any 16-bit instructions other than "load or store halfword". Thus, if one declares uint16_t x and a compiler decides it will fit in a register, the compiler will use a 32-bit register for it (there is no other kind). Any time the compiler stores a value to x, it will ensure the upper bits are all zeroes (masking them off if necessary). If x is never written at all, however, the upper bits might hold arbitrary values left over from whatever other purpose that register was used for. – Teliospore

@supercat: okay, now I understand this, thanks :) What can go wrong for my float->int example on such a platform (I mean, besides the usual thing that I'll get garbage result)? – Subtilize

@geza: I was responding more generally to the comments about uninitialized variables. I should mention, though, looking back through your comments, that despite its increasing size, I doubt that the C++ Standard completely describes the behavior of a useful quality C++ implementation that's suitable for any particular purpose, any more than the C Standard completely describes the behavior of a quality C implementation that's likewise suitable. – Teliospore

@geza: As for what can go wrong, I don't think anything should go wrong when targeting an implementation that's suitable for low-level programming on any remotely-common platform. Some "clever" compiler writers, however, are more interested in exploiting places where the Standard would let them behave nonsensically, for purposes of "optimization", then in exploiting ways in which they could easily and cheaply be made to behave usefully. – Teliospore

@supercat: yes, of course, current C++ implementations are useful. They would be even useful, if the C++ standard would be half of its current size. But a standard should specify everything, that's why it's a standard. If it doesn't specify something, then it is unspecified, i.e. unspecified behavior. We can "specify" these by ourselves in a logical manner, but it would not be backed up by the standard. And this thinking actually caused problems in the past. UBs which previously didn't cause problems, with a new compiler version, do cause problems, etc. – Subtilize

@geza: The best the Standard could hope to do--and what I think it should do--is define a category of Safely Conforming implementations and Selectively Conforming programs such that feeding any Selectively Conforming program to any Safely Conforming implementation will cause it to behave it in defined fashion, indicate via Implementation-Defined means a refusal to do so, or hang (which would be equivalent to executing the code at infinitesimally speed). Then add enough optional features to maximize the range of tasks that can be done with Selectively Conforming programs... – Teliospore

...without having to care about whether all implementations can support all the features defined by the Standard. The range of programs an implementation could usefully process would be a Quality of Implementation issue, but jumping the rails when hitting a translation limit (as opposed to terminating execution in an Implementation-Defined manner) would make an implementation non-compliant. Unfortunately, the Standard fails to define the behavior of most useful programs and relies upon implementations to use at least some common sense. – Teliospore

Answer 3

Your example is well-defined and does not break strict aliasing. std::memcpy clearly states:

Copies count bytes from the object pointed to by src to the object pointed to by dest. Both objects are reinterpreted as arrays of unsigned char.

The standard allows aliasing any type through a (signed/unsigned) char* or std::byte and thus your example doesn't exhibit UB. If the resulting integer is of any value is another question though.

---

use i to extract f's sign, exponent & significand

This however, is not guaranteed by the standard as the value of a float is implementation-defined (in the case of IEEE 754 it will work though).

Lecky

If we can deduce this from the description of memcpy, then why does the standard highlight the two cases I mentioned in my question? – Subtilize

@Subtilize C++ will rely on C for the definition of memcpy and as I said here it says it copies n bytes "w/o condition" between two objects. – Blithe

Note bit_cast uses memcpy as the underlying type punning mechanism. – Blithe

@ShafikYaghmour: yes, but the problem is not there. For example, if you memcpy a non-trivial copyable type, it is UB, because the standard doesn't allow it. Like it doesn't allow my example explicitly. – Subtilize

@ShafikYaghmour: it's not a problem. A compiler provided library can use any UB it wants, see my comment below the question. – Subtilize

@Subtilize like I said, you are allowed to do it by definiton of memcpy, whether you obtain a valid representation is a different and separate story. They don't have to be linked together. It could be stated more explicitly though. – Blithe

@ShafikYaghmour: In my opinion, this has nothing to do with the guarantees of memcpy. The question would be exactly the same, if I use a simple byte-copy loop instead of memcpy. See the comments below user2079303's answer. – Subtilize

The standard allows aliasing some object as a char*, yes, but that says nothing about implicitly rematerialising it as some other object type via said aliasing. (I do think this is legal - it's the conventional alternative to the broken reinterpret_cast<Dest*>(&src) approach - but I don't think anyone here's proven it yet!) – Pasha

@LightnessRacesinOrbit I think the fact that the standard allows reading from and writing to a char* that's aliasing some other object it implies that this allows memcpy to rematerialise by reading from src and writing to dest. Essentially copying the underlying bytes. Although I agree this might be room to interpretation. – Lecky

@SombreroChicken: Yep I agree with hvd that it's almost certainly intended to work for trivially-copyable types but that it's unfortunately underspecified. – Pasha

@Subtilize "For example, if you memcpy a non-trivial copyable type, it is UB, because the standard doesn't allow it" I don't think so: all objects are made up of bytes. Reading those bytes cannot be disallowed. (It's also pointless as putting those bytes elsewhere isn't guaranteed to produce an object with a well defined state.) – Annora

@curiousguy: #27009678 – Subtilize