Why is it invalid for a union type declared in one function to be used in another function?
Asked Answered
E

3

38

When I read ISO/IEC 9899:1999 (see:6.5.2.3), I saw an example like this (emphasis mine) :

The following is not a valid fragment (because the union type is not visible within function f):

struct t1 { int m; };
struct t2 { int m; };
int f(struct t1 * p1, struct t2 * p2)
{
      if (p1->m < 0)
            p2->m = -p2->m;
      return p1->m;
}
int g()
{
      union {
            struct t1 s1;
            struct t2 s2;
      } u;
      /* ... */
      return f(&u.s1, &u.s2);
}

I found no errors and warnings when I tested.

My question is: Why is this fragment invalid?

Enteritis answered 26/9, 2018 at 7:7 Comment(4)
f can assume that p1 != p2 because they point to different types. and with optimization - read p1->m value in register and return this register. it assume that p2->m = -p2->m not modify p1->m what is wrong. union here only way make p1==p2Emmie
I took the liberty of transcribing the image to text, I hope I didn't make any typos. The original is visible in the edit history: stackoverflow.com/revisions/52511896/1Refuge
note that there are many invalid programs that compile cleanly. IN fact the reason there is so much text in C standards about what is valid , invalid, UB, .... is because you cannot rely on the compiler to simply detect and reject themStimulative
simple fix... define the union before the first functionLipoma
S
33

The example attempts to illustrate the paragraph beforehand1 (emphasis mine):

6.5.2.3 ¶6

One special guarantee is made in order to simplify the use of unions: if a union contains several structures that share a common initial sequence (see below), and if the union object currently contains one of these structures, it is permitted to inspect the common initial part of any of them anywhere that a declaration of the completed type of the union is visible. Two structures share a common initial sequence if corresponding members have compatible types (and, for bit-fields, the same widths) for a sequence of one or more initial members.

Since f is declared before g, and furthermore the unnamed union type is local to g, there is no questioning the union type isn't visible in f.

The example doesn't show how u is initialized, but assuming the last written to member is u.s2.m, the function has undefined behavior because it inspects p1->m without the common initial sequence guarantee being in effect.

Same goes the other way, if it's u.s1.m that was last written to before the function call, than accessing p2->m is undefined behavior.

Note that f itself is not invalid. It's a perfectly reasonable function definition. The undefined behavior stems from passing into it &u.s1 and &u.s2 as arguments. That is what's causing undefined behavior.


1 - I'm quoting n1570, the C11 standard draft. But the specification should be the same, subject only to moving a paragraph or two up/down.

Stimulant answered 26/9, 2018 at 7:12 Comment(14)
So would changing f to take int* and passing in &u.s1.m and u.s2.m make it valid? Because then it’s g doing the struct accessing.Zurek
@Zurek - You know, I'm not sure. I think that's a good question that deserves to stand on its own. Post it with the the language-lawyer tag. It should be interesting.Stimulant
@Zurek - Wait, actually you are right. Since those pointers are of the same type, they may alias the same object. It's the whole strict aliasing spiel.Stimulant
I think I am missing something here. Though unnamed union is local to g but address of it's elements are passed to f. So union will exist until g returns and therefore shouldn't it's members be visible to f as it's address is passed to f ?Conferva
@Conferva - The text in bold doesn't require the union to merely exist. It requires the definition of the union type to be visible in f. This example is crafted such that there is no dispute f cannot possibly be seeing this union type declaration itself.Stimulant
So the statement: it inspects p1->m without the common initial sequence guarantee being in effect. is holds true because union is not visible to f?Conferva
@Conferva - Yes. That's why GCC and Clang go to town on this function under strict aliasing. They have no reason to assume the two unrelated type may alias. Because normally they can't. Technically this is all handled under the "effective type" set of clauses. But here's another example. Here the compiler knows aliasing is possible, so it's not being too aggressive.Stimulant
@Zurek : in your alternative, g does not do the struct accessing - all it does is calculate addresses. So, there's no real difference with the original.Dosia
@StoryTeller; That make sense. Where is the standard mention for this to be undefined behavior?Conferva
@Conferva - §6.5 ¶7. Those are all the valid ways to alias the same object. So in the original example p1 and p2 may be assumed to not alias the same object (even though they do in a valid way through the union). The code example I gave you in the previous comment must be conservative since the union contains the struct as a member, so things may be aliasing. I think the common sub-sequence guarantee is phrased like that to play along with the effective type requirements.Stimulant
@StoryTeller; Thanks a lot for explaining it and being patient :)Conferva
@Conferva - is holds true because union is not visible to - are visible union to f or not not play role at all. if we make union declaration visible to f nothing change. i think it is permitted .. is visible have nonsense (bad documentation), in analog c++ no such words. and need understand sense of this "special guarantee". i think we always can "inspect" any union member. but different types have different representation in memory. and this is general undefined how same memory will be interpreted via another type. only because this inspect "not active"..Emmie
..(not last written) member is undefined. but if it have the same type as active (last written) member - we can use it. because the same memory representation. or until "common initial sequence" also if we have not 2 standard-layout structs but say union U { struct t { int m;} s; int m; } u; u.s.m and u.m have common initial sequence, despite formal u.m not a struct at all. this only say about not good enough docsEmmie
@RbMm: The Common Initial Sequence guarantees were documented in 1974, well before the Standard was written. I think the authors of the Standard expected compiler writers to understand and respect the purposes and uses of such guarantees without the Standard having to go into details about it.Manstopper
E
27

Here is the strict aliasing rule in action: one assumption made by the C (or C++) compiler, is that dereferencing pointers to objects of different types will never refer to the same memory location (i.e. alias each other.)

This function

int f(struct t1* p1, struct t2* p2);

assumes that p1 != p2 because they formally point to different types. As a result the optimizatier may assume that p2->m = -p2->m; have no effect on p1->m; it can first read the value of p1->m to a register, compare it with 0, if it compare less than 0, then do p2->m = -p2->m; and finally return the register value unchanged!

The union here is the only way to make p1 == p2 on binary level because all union member have the same address.

Another example:

struct t1 { int m; };
struct t2 { int m; };

int f(struct t1* p1, struct t2* p2)
{
    if (p1->m < 0) p2->m = -p2->m;
    return p1->m;
}

int g()
{
    union {
        struct t1 s1;
        struct t2 s2;
    } u;
    u.s1.m = -1;
    return f(&u.s1, &u.s2);
}

What must g return? +1 according to common sense (we change -1 to +1 in f). But if we look at gcc's generate assembly with -O1 optimization

f:
        cmp     DWORD PTR [rdi], 0
        js      .L3
.L2:
        mov     eax, DWORD PTR [rdi]
        ret
.L3:
        neg     DWORD PTR [rsi]
        jmp     .L2
g:
        mov     eax, 1
        ret

So far all is as excepted. But when we try it with -O2

f:
        mov     eax, DWORD PTR [rdi]
        test    eax, eax
        js      .L4
        ret
.L4:
        neg     DWORD PTR [rsi]
        ret
g:
        mov     eax, -1
        ret

The return value is now a hardcoded -1

This is because f at the beginning caches the value of p1->m in the eax register (mov eax, DWORD PTR [rdi]) and does not reread it after p2->m = -p2->m; (neg DWORD PTR [rsi]) - it returns eax unchanged.


union here used only for all non-static data members of a union object have the same address. as result &u.s1 == &u.s2.

is somebody not understand assembler code, can show in c/c++ how strict aliasing affect f code:

int f(struct t1* p1, struct t2* p2)
{
    int a = p1->m;
    if (a < 0) p2->m = -p2->m;
    return a; 
}

compiler cache p1->m value in local var a (actually in register of course) and return it , despite p2->m = -p2->m; change p1->m. but compiler assume that p1 memory not affected, because it assume that p2 point to another memory which not overlap with p1

so with different compilers and different optimization level the same source code can return different values (-1 or +1). so and undefined behavior as is

Emmie answered 26/9, 2018 at 7:55 Comment(7)
@StoryTeller - yes, the same (-1) with clang. but with msvc and icc always +1 returned (no strict aliacing here)Emmie
Just as an addendum to this, it's worth mentioning that declaring the arguments to f() as volatile will prevent this optimisation. Technically this is still undefined behaviour according to the standard, but most compilers should come up with a sane answer. Why anyone would want to write such bizarre code is beyond me, though. :-/Sidman
The purpose of the Standard is to ensure that even compilers that are too primitive to recognize at the call site that a call to someFunction(&myUnion->member1) might interact with other members in the union which are used by the caller, could recognize in the called function that such interactions may occur. In practice, gcc and clang don't reliably recognize such interactions in any case, even when complete union declarations are visible, and when the compiler is able to see that the structures sharing the Common Initial Sequence are in fact members of the same array of union objects.Manstopper
@Manstopper - from my view unions here unrelated at all. this play role compiler optimization (in f) based on assumption that pointers to 2 different type can not overlap. so modification memory by p2 pointer can not affect memory of p1. union here used only for format 2 pointers with different type but point to same memory. so while we can access and inactive member of union if it "similar" to active, use pointers to 2 union member at once, even if it have "common initial sequence" (even full equal) lead to this ubEmmie
@Manstopper i think faster compilers must assume that 2 pointers to 2 types that have "common initial sequence" can point to the same memory. the t1 and t2 is such types. so will be logical (from my opinion) that compilers assume that such pointers can alias. but now gcc/clang not do this. also i be serious redesign(extend) term what is "Common Initial Sequence"Emmie
@RbMm: I wouldn't particularly fault an assumption that a pointer to a structure won't alias an existing pointer to another structure. I don't think a quality compiler, given something like proc1(&unionArr[i]->member1); proc2(&unionArr[j]->member2); proc3(&unionArr[i]->member1);, should have any trouble recognizing each function argument as freshly derived from unionArr. While 6.5p7 doesn't allow struct or union objects to be accessed using non-character member types under any circumstances (as written, it doesn't even allow unionLvalue.member!), the address-of operator is pretty...Manstopper
...meaningless under gcc and clang. If the Standard isn't going to require that compilers implement it meaningfully, they should allow compilers to treat the value produced as a type incompatible with anything but void*.Manstopper
M
3

One of the major purposes of the Common Initial Sequence rule is to allow functions to operate on many similar structures interchangeably. Requiring that compilers presume that any function which acts upon a structure might change the corresponding member in any other structure that shares a common initial sequence, however, would have impaired useful optimizations.

Although most code which relies upon the Common Initial Sequence guarantees makes use of a few easily recognizable patterns, e.g.

struct genericFoo {int size; short mode; };
struct fancyFoo {int size; short mode, biz, boz, baz; };
struct bigFoo {int size; short mode; char payload[5000]; };

union anyKindOfFoo {struct genericFoo genericFoo;
  struct fancyFoo fancyFoo;
  struct bigFoo bigFoo;};

...
if (readSharedMemberOfGenericFoo( myUnion->genericFoo ))
  accessThingAsFancyFoo( myUnion->fancyFoo );
return readSharedMemberOfGenericFoo( myUnion->genericFoo );

revisiting the union between calls to functions that act on different union members, the authors of the Standard specified that visibility of the union type within the called function should be the determining factor for whether functions should recognize the possibility that an access to e.g. field mode of a FancyFoo might affect field mode of a genericFoo. The requirement to have a union containing all types of structures whose address might be passed to readSharedMemberOfGeneric in the same compilation unit as that function makes the Common Initial Sequence rule less useful than it would otherwise be, but would make at least allow some patterns like the above usable.

The authors of gcc and clang thought that treating union declarations as an indication that the types involved might be involved in constructs like the above would be an impractical impediment to optimization, however, and figured that since the Standard doesn't require them to support such constructs via other means, they'll simply not support them at all. Consequently, the real requirement for code that would need to exploit the Common Initial Sequence guarantees in any meaningful fashion is not to ensure that a union type declaration is visible, but to ensure that clang and gcc are invoked with the -fno-strict-aliasing flag. Also including a visible union declaration when practical wouldn't hurt, but it is neither necessary nor sufficient to ensure correct behavior from gcc and clang.

Manstopper answered 26/9, 2018 at 23:40 Comment(12)
you try say that if we have S1* p1; and S2* p2; , S1 and S2 have common initial sequence and exist visible declaration union U { S1 s1; S2 s2; }; compiler can not assume that p1 != p2, but if remove U declaration - compiler already can assume that p1 != p2 because it different types (despite CIS) and do optimization based on this ? from my opinion this not very good/logicalEmmie
from one side of standard If a program attempts to access the stored value of an object through a glvalue of other than one of the following types the behavior is undefined.. and case when we access object via type which have CIS type (even with full CS type) here formal not listed ? so UB ? but from another side in union paragraph CIS rule. but when we access "not active" union member - we access the stored value of an object with dynamic type of "active" member - and from my look here contradiction in rulesEmmie
@RbMm: The only way I can see for 6.5p7 to make any sense is to recognize that it is only meant to apply in cases which involve aliasing as written, and that accessing an object with a pointer that is freshly derived from it isn't "aliasing". In the "bad" CIS example in the Standard, if both pointers identify members of the same union object, whichever one had been derived first would have ceased to be "fresh" when the other was derived from that union. Clang and gcc, however, will rewrite examples that don't use aliasing so as to add aliasing which they then can't handle.Manstopper
@RbMm: If accesses via freshly-derived pointers are recognized as accesses to the objects from which they are derived, then few programs would need permission to access structures and unions with unrelated lvalues of member types (or character types, for that matter!), and thus the lack of blanket permission would make sense. Unfortunately, since the authors of the Standard didn't say that quality compilers should recognize pointer derivation (probably thinking the footnote about aliasing would suffice), clang and gcc don't bother.Manstopper
from another side - are this code formal correct ? here 3 different members of union have not CIS but by sense - all 3 is pointers, have equal memory layout, so we can read any of this after assign/change oneEmmie
@RbMm: On some hardware platforms, pointers to different types may have different representations or even different sizes. Implementations intended to be suitable for low-level programming on a platform where all pointers use the same representation should support type-agnostic constructs, but unfortunately there's no formal means by which implementations can specify whether or not they are suitable for low-level programming, or by which programs can indicate that they require such implementations.Manstopper
here i mean another question (assume that i know that all pointers have the same size and same data representation on concrete platform) - anyway - are correct from formal language c++ rules real "not-active" member in this case. are compiler can generate ub here. if interpret the CIS very narrowly and formally - we can not. even if we have union { int x; int y;} x and y have not CIS, or union { int x; struct { int y; } s; } again no formal CIS between x and s.y. only in case union { struct { int x;} s1; struct { int y; } s2; } formal s1.x and s2.y have CIS. ..Emmie
..but by sense we have "wide CIS" the same memory layout in all 3 case. more correct say that we always can read any union member. simply need understand that different types have different representation in memory and need take this to account when read dataEmmie
on concrete topic yet another example of contradictory behavior godbolt.org/z/JNjL5-Emmie
@RbMm: Both C and C++ have fractured into diverging groups of dialects, with one group interpreting Undefined Behavior as "actions may be processed in whatever fashion the implementer thinks would make them most useful", and the other interpreting it as "Implementations should not be expected to process usefully any action upon which the Standard imposes no requirements, except in cases where failure to do so would render an implementation almost completely useless for any purpose whatsoever". Personally, I think the authors of the Standard would have thought it self-evident that...Manstopper
...people writing implementations for various purposes should, whenever allowed to do so, make a bona fide effort to do whatever is practical to make them maximally suitable for such purposes, but to some people it isn't self-evident.Manstopper
The footnote to 6.5p7 says the rules are intended to indicate when things may alias. In your second example, the pointers to union members alias in both demo functions, so I wouldn't expect compilers to handle that reliably. If, however, you had split f into three functions, each taking one pointer, and had derived the pointers passed to each following the return from the previous function, there would be no aliasing, but given f(&unionArr[i].s1); g(&unionArr[j].s2); h(&unionArr[i].s1); gcc/clang would ignore the possibility of i==j.Manstopper

© 2022 - 2024 — McMap. All rights reserved.