gcc, strict-aliasing, and horror stories [closed]
Asked Answered
M

6

63

In gcc-strict-aliasing-and-casting-through-a-union I asked whether anyone had encountered problems with union punning through pointers. So far, the answer seems to be No.

This question is broader: Do you have any horror stories about gcc and strict-aliasing?

Background: Quoting from AndreyT's answer in c99-strict-aliasing-rules-in-c-gcc:

"Strict aliasing rules are rooted in parts of the standard that were present in C and C++ since the beginning of [standardized] times. The clause that prohibits accessing object of one type through a lvalue of another type is present in C89/90 (6.3) as well as in C++98 (3.10/15). ... It is just that not all compilers wanted (or dared) to enforce it or rely on it."

Well, gcc is now daring to do so, with its -fstrict-aliasing switch. And this has caused some problems. See, for example, the excellent article http://davmac.wordpress.com/2009/10/ about a Mysql bug, and the equally excellent discussion in http://cellperformance.beyond3d.com/articles/2006/06/understanding-strict-aliasing.html.

Some other less-relevant links:

So to repeat, do you have a horror story of your own? Problems not indicated by -Wstrict-aliasing would, of course, be preferred. And other C compilers are also welcome.

Added June 2nd: The first link in Michael Burr's answer, which does indeed qualify as a horror story, is perhaps a bit dated (from 2003). I did a quick test, but the problem has apparently gone away.

Source:

#include <string.h>
struct iw_event {               /* dummy! */
    int len;
};
char *iwe_stream_add_event(
    char *stream,               /* Stream of events */
    char *ends,                 /* End of stream */
    struct iw_event *iwe,       /* Payload */
    int event_len)              /* Real size of payload */
{
    /* Check if it's possible */
    if ((stream + event_len) < ends) {
            iwe->len = event_len;
            memcpy(stream, (char *) iwe, event_len);
            stream += event_len;
    }
    return stream;
}

The specific complaint is:

Some users have complained that when the [above] code is compiled without the -fno-strict-aliasing, the order of the write and memcpy is inverted (which means a bogus len is mem-copied into the stream).

Compiled code, using gcc 4.3.4 on CYGWIN wih -O3 (please correct me if I am wrong--my assembler is a bit rusty!):

_iwe_stream_add_event:
        pushl       %ebp
        movl        %esp, %ebp
        pushl       %ebx
        subl        $20, %esp
        movl        8(%ebp), %eax       # stream    --> %eax
        movl        20(%ebp), %edx      # event_len --> %edx
        leal        (%eax,%edx), %ebx   # sum       --> %ebx
        cmpl        12(%ebp), %ebx      # compare sum with ends
        jae L2
        movl        16(%ebp), %ecx      # iwe       --> %ecx
        movl        %edx, (%ecx)        # event_len --> iwe->len (!!)
        movl        %edx, 8(%esp)       # event_len --> stack
        movl        %ecx, 4(%esp)       # iwe       --> stack
        movl        %eax, (%esp)        # stream    --> stack
        call        _memcpy
        movl        %ebx, %eax          # sum       --> retval
L2:
        addl        $20, %esp
        popl        %ebx
        leave
        ret

And for the second link in Michael's answer,

*(unsigned short *)&a = 4;

gcc will usually (always?) give a warning. But I believe a valid solution to this (for gcc) is to use:

#define CAST(type, x) (((union {typeof(x) src; type dst;}*)&(x))->dst)
// ...
CAST(unsigned short, a) = 4;

I've asked SO whether this is OK in gcc-strict-aliasing-and-casting-through-a-union, but so far nobody disagrees.

Miss answered 2/6, 2010 at 14:27 Comment(8)
possible duplicate of gcc, strict-aliasing, and casting through a unionOrcein
caf's answer stackoverflow.com/questions/1926282/… is another reference, but it is only theoretical, not horrific.Miss
Also, to be clear, references to gcc coding bugs are not helpful--all code has bugs. But design 'bugs' may be criticized.Miss
And caf's comment to the quoted AndreyT answer is amusing: "I'm still not sure whether it's possible to use the sockaddr-related parts of the BSD sockets API without breaking the strict aliasing rules.Miss
John Regehr gives two interesting, short, examples of inconsistencies in GCC and Clang.Miss
An old one: gcc.gnu.org/ml/gcc-bugs/2000-03/msg00155.html : When memcpy() was used everything was fine, but when it was inlined by GCC, it doesn't cope with unaligned access ... wrong codeVivienviviene
I think the viewpoint that compiler writers were too lazy to exploit the rule is somewhat revisionist. The type rules in C make it impossible to create functions which manage and recycle memory blocks in type-agnostic fashion. Thus, conformance with the type rules may impose massive (50% or more) time and space penalties compared with what could be achieved in their absence. For many kinds of programming, there are no optimizations the rules could enable that would overcome that massive penalty.Seals
As to the horror story, the code fragment was actually OK. The real bug was inside memcpy. They were using a custom memcpy (a macro) which uses longs to copy bytes. Otherwise it is perfect OK to cast pointers to structs, to pointer to chars, and memcpy them.Smiley
C
36

No horror story of my own, but here are some quotes from Linus Torvalds (sorry if these are already in one of the linked references in the question):

http://lkml.org/lkml/2003/2/26/158:

Date Wed, 26 Feb 2003 09:22:15 -0800 Subject Re: Invalid compilation without -fno-strict-aliasing From Jean Tourrilhes <>

On Wed, Feb 26, 2003 at 04:38:10PM +0100, Horst von Brand wrote:

Jean Tourrilhes <> said:

It looks like a compiler bug to me... Some users have complained that when the following code is compiled without the -fno-strict-aliasing, the order of the write and memcpy is inverted (which mean a bogus len is mem-copied into the stream). Code (from linux/include/net/iw_handler.h) :

static inline char *
iwe_stream_add_event(char *   stream,     /* Stream of events */
                     char *   ends,       /* End of stream */
                    struct iw_event *iwe, /* Payload */
                     int      event_len)  /* Real size of payload */
{
  /* Check if it's possible */
  if((stream + event_len) < ends) {
      iwe->len = event_len;
      memcpy(stream, (char *) iwe, event_len);
      stream += event_len;
  }
  return stream;
}

IMHO, the compiler should have enough context to know that the reordering is dangerous. Any suggestion to make this simple code more bullet proof is welcomed.

The compiler is free to assume char *stream and struct iw_event *iwe point to separate areas of memory, due to strict aliasing.

Which is true and which is not the problem I'm complaining about.

(Note with hindsight: this code is fine, but Linux's implementation of memcpy was a macro that cast to long * to copy in larger chunks. With a correctly-defined memcpy, gcc -fstrict-aliasing isn't allowed to break this code. But it means you need inline asm or __attribute__((aligned(1),may_alias)) (e.g. in a typedef) to define a kernel memcpy if your compiler doesn't know how turn a byte-copy loop into efficient asm, which was the case for gcc before gcc7)

And Linus Torvald's comment on the above:

Jean Tourrilhes [email protected] wrote:

It looks like a compiler bug to me...

Why do you think the kernel uses "-fno-strict-aliasing"?

The gcc people are more interested in trying to find out what can be allowed by the c99 specs than about making things actually work. The aliasing code in particular is not even worth enabling, it's just not possible to sanely tell gcc when some things can alias.

Some users have complained that when the following code is compiled without the -fno-strict-aliasing, the order of the write and memcpy is inverted (which mean a bogus len is mem-copied into the stream).

The "problem" is that we inline the memcpy(), at which point gcc won't care about the fact that it can alias, so they'll just re-order everything and claim it's out own fault. Even though there is no sane way for us to even tell gcc about it.

I tried to get a sane way a few years ago, and the gcc developers really didn't care about the real world in this area. I'd be surprised if that had changed, judging by the replies I have already seen.

I'm not going to bother to fight it.

Linus

http://www.mail-archive.com/[email protected]/msg01647.html:

Type-based aliasing is stupid. It's so incredibly stupid that it's not even funny. It's broken. And gcc took the broken notion, and made it more so by making it a "by-the-letter-of-the-law" thing that makes no sense.

...

I know for a fact that gcc would re-order write accesses that were clearly to (statically) the same address. Gcc would suddenly think that

unsigned long a;

a = 5;
*(unsigned short *)&a = 4;

could be re-ordered to set it to 4 first (because clearly they don't alias - by reading the standard), and then because now the assignment of 'a=5' was later, the assignment of 4 could be elided entirely! And if somebody complains that the compiler is insane, the compiler people would say "nyaah, nyaah, the standards people said we can do this", with absolutely no introspection to ask whether it made any SENSE.

Clercq answered 2/6, 2010 at 16:9 Comment(51)
+1. But gcc will usually (hopefully always?) give a warning in this example.Miss
And one possible solution to Linus's complaint is: #define CAST(type, x) (((union {typeof(x) src; type dst;}*)&(x))->dst) followed by CAST(unsigned short, a) = 4;Miss
The first one is definitely a bug (caused by the compiler's inlined implementation of malloc falling foul of its own strict aliasing rules!).Ovine
@Alok see the second link, the compiler really is slightly insane with -fstrict-aliasingBoxberry
@Boxberry Insane how? Do you suggest that the compiler should detect programmer intent?Normi
@Normi no just that it should notice the blatantly obvious, the compiler does something that it could easily detect is totally and utterly wrong. Go look at that snippet out by itself, the compiler first reorders writes because they "can't alias" then removes one of them because they DO. So clearly it can tell the stores are to the same memory location, so it probably oughtn't re-order the stores, but it does so anyway because they are done through pointers of a different type.Boxberry
"the compiler does something that it could easily detect is totally and utterly wrong." Are you saying that this is actually an ICE? "So clearly it can tell the stores are to the same memory location, so it probably oughtn't re-order the stores" What you say is blatantly absurd. It is allowed to reorder the stores by some rule. I am not saying that the rule is sound. May be it is not. You cannot say that the compiler should stop applying the general rule because it breaks this particular program. You can only say that the strict-aliasing stuff is unsound. This is elementary logic.Normi
Damn, this is really horrifying :\Postscript
The aliasing rules are set by the C standard. If you do stupid things like *(unsigned short *)&a when there is a sound and secure way to do the same thing (by casting through a union) you can't blame the compiler :-)Geneticist
@Alok Linus is spot-on. The intent of the standard's strict aliasing rules are to allow the compiler to optimize in situations where it doesn't and cannot know whether an object is being aliased. The rules permit the optimizer to not make worst-case aliasing assumptions in those situations. However, when it is clear from the context that an object is being aliased, then the compiler should treat the object as being aliased, no matter what types are being used to access it. Doing otherwise is not in line with the intent of the language's aliasing rules.Fowler
@DanMoulding I haven't read the standard in a while, but I didn't know that that was the intent in the standard. My comment was regarding how much work should the compiler do to make sure two different names alias to the same underlying memory. I could have written my comment better though.Wentworth
@DanMoulding Linus is absolutely wrong. The standard defines which C constructs are valid. The intent is to tell programmers what they can or cannot do. The role of the compiler is not to "help" the incompetent programmer the way Linus wants.Normi
It's GCC's fault AND the standard's fault. Dan Moulding is correct about the intent of strict aliasing rules. The standard's problem is that it doesn't explicitly name this intent and guard against malevolent "Jackass Genie" implementations. It should say something to the effect of, "The compiler must do a minimum of XYZ to determine whether two objects alias each other. If they are proven to alias, they must be treated as aliasing. If they are proven not to alias, the compiler is free to optimize accordingly. If the compiler cannot prove either, type-based aliasing rules apply."Backcross
The standard's wording therefore permits sadistically context-insensitive implementations to deliberately break obvious type punning (violating the "spirit" of the law). The standard permits evil, and it should be fixed someday, but it still does not demand it: Evil is still a conscious choice. It's perfectly possible to write a conformant implementation that recognizes common idioms and acts accordingly, but language lawyers revel in violating the principle of least surprise and writing the cruelest implementation allowed by law. Linus's "nyaah, nyaah" characterization is spot-on.Backcross
@MikeS Absolutely wrong. There is no such spirit of the law. The spirit is: follow the rules, dammit!Normi
@curiousguy: The C language specifications are insufficient to allow practical non-trivial programs to be written that do not rely upon things which the standards did not specify. Further, many C compilers have historically documented their behavior in situations that were left "undefined" by the C standard. IMHO, there should be an effort to allow programmers to specify their platform-related assumptions, so as to allow compilers to either behave appropriately if they can, or refuse compilation if they cannot.Seals
@curiousguy: I think the major problem is that the authors of the standard tried to simplify things by ignoring various weird corner cases that at the time were irrelevant. If some action could cause a hardware trap, and the effects of the trap may not be under program control, then from the standpoint of the standard, such action would invoke "Undefined Behavior". Any implementation where the action doesn't trap, or where traps behaved in controllable fashion, could document such things, so there was no perceived need to require that implementations document whether they trap.Seals
@everyone: In 2011 Chris Lattner of the LLVM project wrote a series of blog articles that explain how undefined behavior affects how a compiler can operate. More importantly, he talks about how some rules about UB can affect optimizations. Specifically in part one, blog.llvm.org/2011/05/what-every-c-programmer-should-know.html, he briefly discusses type punning. The entire article series is well worth reading. It gives at least some rationale for type punning being UB. Whether you think the value justifies the decision to make it undefined behavior is still up to you to decide.Clercq
@MichaelBurr: Fundamentally, C-with-aliasing-rules and C-without-aliasing-rules should be regarded as two different languages, each of which can do very well some things which the other does poorly. For some kinds of programming, aliasing rules impose massive time and space penalties for no benefit; for others, they enable massive speedups with little downside. Standardizing memory barriers would have allowed the best of both worlds, but even 25 years on that still hasn't happened.Seals
It is ignorant and stupid to assume the fictional char-based aliasing rule as part of the "spirit" of C. C is never designed as such a language, even it may be occasionally implemented being not conflict with or without such rules. It is more stupid that someone invents the naive dialect which is obviously not more expressive than the standard language at all and then claims it should replace the standard C. These people should educate themselves to gain some common sense of the language, as well as the programing language theory.Truck
@FrankHB: When C89 was published, there existed a few compilers that used type based aliasing and many that didn't. The authors of the Standard didn't want to say that compiler vendors whose customers like their products could not achieve C89 compliance without degrading performance. I see no evidence that they even considered the possibility that anyone trying to produce a quality compiler would pretend that aliasing cannot occur in cases where--in the absence of the rule--available evidence would suggest that it is both likely and useful.Seals
@FrankHB: On platforms where the bit representation of every non-zero float value were a trap representation for int and vice versa, it would be pointless to mandate that a compiler regard int floatbits(float *f) { return *(int*)f; } as being likely to access a float. I see no evidence that the lack of a mandate was intended to discourage recognition of such constructs on platforms where they could be useful.Seals
@Seals There was no consensus that an implementation of C should have type based aliasing or not, and no doubt that implementations with type based aliasing were still genuine implementations of C in that time. So it is true that the rule (or some stronger restriction) is necessary even merely for portability reasons. This renders that excluding the rule today while talking about "spirit of C" is even more stupid.Truck
There is one thing that Linus Torvalds is true: the standard language does not provide possible-aliased access in a sane way (e.g. encoded in the type system) that is intuitive enough. But type punning is explicitly allowed (just in another way), so the remained complains are nonsense. Another horrible thing is mandating hard-coded types as exceptions breaks the orthogonality: how to enable TBAA on char? A worse one is in C++: why char16_t/char32_t always can't alias wchar_t (not a trouble in a sane implementation of C11 where they may be typedef names of a same integer type)?Truck
@FrankHB: Some implementations did have various kinds of type-based aliasing, but that doesn't mean that they would assume the floatbits function above can't possibly access a float in the absence of options or directives for hyper-aggressive optimizations that are acknowledged as being incompatible with a lot of existing code. The notion that memcpy should be used for aliasing goes completely against the spirit of C since it requires a programmer to write horribly inefficient code the programmer doesn't want in the hopes that an optimizer will turn it into what the programmer does wantSeals
@FrankHB: Besides--there's no guarantee that memcpy will fix aliasing problems since it is allowed to treat the destination storage as having the same effective type as the source. And any recognized attempt to copy something as a character array may do likewise. The problem isn't just that the few ways of getting around the aliasing are unintuitive--the problem is that only ways that could be guaranteed to work would force a compiler to generate really hideous code that the optimizer can't clean up.Seals
As to the horror story, the code fragment was actually OK. The real bug was inside memcpy. They were using a custom memcpy (a macro) which uses longs to copy bytes. Otherwise it is perfect OK to cast pointers to structs, to pointer to chars, and memcpy them.Smiley
@ntysdd: Would memcpy have been "broken" if gcc's aggressive type-based aliasing analysis had recognized it? There are enough situations where gcc makes optimizations which would be fine in -fno-strict-aliasing mode, but which get broken by gcc's aliasing logic, that I'd say the real problem is with the aliasing logic, especially since fixing the aliasing logic should be cheaper than having the compiler refrain from any optimizations that would interact poorly with it.Seals
@FrankHB: Even if unions may have been intended to allow type punnng of non-character types, the way 6.5p7 is written makes no provision for accessing a union object via lvalues of member type, even if the lvalue happens to be a member-access expression. The maintainers of gcc may have recognized that even though the Standard would allow a compiler to assume that code which uses member-access lvalues of non-character type to access declared objects of union types will ever be executed, a compiler making such an assumption would fall into the "conforming but useless" category.Seals
@Seals Of course I mean just that. The "memcpy" macro (it was a macro) used long's to copy bytes, and broke the alias rule. The kernel has since been built with -fno-strict-aliasing.Smiley
@ntysdd: I wonder why compiler writers have designed their optimizer's type systems around what "low quality but conforming" compilers would be allowed to do [but in a way that fails to actually be conforming], than in providing useful semantics? There has never been any reason a quality compiler should have any difficulty recognizing that an access via pointer of any type that was freshly visibly derived from a T* is a potential access to a T. The Standard's failure to describe any circumstances where a cross-type-derived lvalue should be usable (not even aggregate.member) is...Seals
...most reasonably attributable to a desire to treat that as a Quality of Implementation issue, and a mistaken belief that compiler writers would seem to produce quality implementations.Seals
@Seals Perhaps it's because of modern optimizing compilers having lots of passes. Those type info needed for a high quality transformation gets lost during these passes. And I am not sure whether it's a good idea to build a high quality compiler that does sensible things in the easy case but breaks badly while programs evolves.Smiley
@ntysdd: Compiler writers who aren't making a bona fide effort to make high-quality implementations suitable for low-level programming can and do discard such information in early passes, but that doesn't say why they'd rather argue that low-level code is "broken" than make any effort to support it. As for your second point, it would be unusual for programs to evolve in ways which would break a compiler that makes any bona fide neffort whatsoever to recognize access non-overlapping access patterns.Seals
@ntysdd: Further, I don't think a compiler can properly and efficiently support code that changes objects' effective type in ways defined by the Standard without being able to carry through--in at least some cases--the kinds of information of the kinds that early stages of some popular compilers discard; if compilers would need to carry through the information sometimes, it should be able to do so consistently. For example, if a function receives a void*, converts it to struct s1*, reads out all the fields thereof, then converts the pointer to a struct s2*...Seals
...and writes the exact same bit patterns back to it, such a function shouldn't need to directly produce any machine instructions, but it would need to be recognized as changing the Effective Type of the storage in question. I wouldn't mind having a recognized dialect of C where effective types are "permanent" (waiving any requirement for compilers to recognize code that changes object's effective type without any actual loads or stores) but a compiler that can efficiently handle changing effective types would need to handle the kinds of information early stages presently strip out.Seals
@Seals C's pointer makes it very difficult for compilers to do aliasing reasoning. I don't believe it is possible to reach the efficiency of Fortran if compilers don't exploit type information (and even with strict aliasing sometimes you need the restrict keyword). Compiler writers say the low-level code is broken because the Standard says so, and the Standard says so because the committee thinks it's more important for C to be fast than C to be safe and easy for programmers to use. However I do agree we want better support (than char*) for programmers to express their ideas about aliasing.Smiley
@Seals Most compilers optimizations nowadays are based on pattern matching. Compilers need not know the precise aliasing status of the program to transform the program, they only need rules. For a weak type language like C, you can only allow the compilers to do more by allow the compilers to assume more about the property of the program. That is, if you allow compilers to assume signed overflows don't happen, they can reason more about numbers, so we have the signed overflow UB; if you allow compilers to assume program don't silently loop forever, they can move more statements here andSmiley
@Seals there, so we have the loop forever UB. If you allow the compilers to assume program only do "type punning" in certain way, compilers no longer need solve the hard problem of alias analysis (and can change the order of stores and loads as they wish and need not conservatively split values into memory), so we have the strict aliasing UB.Smiley
@ntysdd: The cases were programmers get so annoyed at the aliasing rule don't actually involve aliasing as written but instead involve compilers ignoring obvious pointer derivation. Further, if one wants to allow optimization, the right way to do it without gutting the language's semantics is to allow programmers to invite compilers to do various things in certain circumstances which may affect a program's visible behavior, and let programmers decide whether the effects are acceptable, rather than trying to characterize programs whose behavior could be affect as UB...Seals
...and then saying there's nothing wrong with compilers that behave in completely arbitrary fashion in those cases (even if the optimization such allowances intended to permit would not have affected program behavior at all). Besides, if someone needs all the efficiency of Fortran, they can write in Fortran. Ever since Fortran-95 it's been a better language than C for the kinds of things it can do. C would be much more useful for many purposes if compiler writers tried to focus on efficiently handling the kinds of programming tasks that Fortran can't.Seals
@ntysdd: As for pattern-matching and overflow, that kind of behavior I agree with: saying that if one computes x=y*z; and then later computes w=x/z, a compiler may have w yield either x/z or w, chosen in Unspecified fashion, is useful, since it allows a compiler to choose the most efficient way to do something while avoiding the need for overflow checks in scenarios where yielding w or x/z would both be acceptable. Requiring that programmers write code in a way that would prevent such optimizations is not a recipe for efficiency,.Seals
@ntysdd: As for the "loop forever" UB, I think the optimizations the authors of the Standard meant to allow would have been enabled, without confusion, by saying "The time required to execute a piece of code, even if infinite, is not considered an observable side-effect that compilers must maintain." Likewise, "if operations are not defined as having side-effects, the order in which they are performed is not considered observable even if one of the operations might cause an implementation-specific trap."Seals
@ntysdd: Thinking about it, I like the idea of characterizing various aspects of behavior are "non-observable". A major problem with characterizing overflow as Implementation-Defined is that it would compel implementations which trap overflow to sequence all operations that might overflow precisely as specified (so extern int x,y; x=1; y++; x=2; would be required to leave x holding 1 if y++ overflows). Saying that overflow invokes Implementation-Defined behavior, except that the timing of side effects is generally not considered observable, would avoid that difficulty.Seals
@Seals The answer is super easy. Those who can't tolerate UB or arbitrary behavior should use a "safer" language. C was like C when it was born, and it never made any promise that it would make programming easy. And using a "safer" language is indeed what the industry is doing. People don't write C in web pages, they use js; they don't write C for web servers, they use Java or PHP; they only write C when performance is so important that it is worth the effort; then it makes no sense to get C easy to use but slower. If signed integer must not overflow, it is easy to show x * d / d is just x.Smiley
@suprcat And if strict-aliasing is followed, then *pfloat = 1.0f; *pi =42; never interferes each other, and the compiler can do constant folding as it likes. The fact is that, strict-aliasing does make some programs run faster, much fast, especially for numeric code. Whether it is a good trade off is left to the committee, and the committee decides that it is. I personally think it is good to have this aliasing rule, I just don't like it in its current form. I find it hard to understand, and there is no easy way for programmers to override it.Smiley
@ntysdd: For many embedded and systems programming purposes there are no languages other than C, C++, and assembly language. According to the published Rationale, the intention of the Committee was that UB be used in part to facilitate useful variety among implementations. The Standard does not require that all implementations be suitable for every purpose, but UB allows purposes the Standard does not require that implementations serve, to be facilitated by implementations that are designed to be suitable for them.Seals
@ntysdd: Under what I would consider sensible [non-]"aliasing" rules, a compiler could treat the assignments *pfloat = 1.0f; *pi = 42; as unsequenced unless the object accessed first (in execution order) was derived from the one accessed second during the current iteration of the innermost loop (if any) or the current function. Recognizing pointer derivation would allow most implementations to eliminate the performance-robbing "character type" exception without affecting the behavior of the programs they're intended to support, since nearly all code that uses character types...Seals
...to access storage will perform all operations that are going to be done using a character pointer between the time they convert an object's address to a char* and the next time the storage is accessed or addressed via other means. Under a literal reading of 6.5p7, even something that obviously should be defined like struct foo { int x[2]; }; s; s.x[0]=1; would invoke UB unless an access via to an int* which is freshly derived from the address of a struct member is considered an access to the struct. Exactly when implementations recognize freshly-derived pointers might be regarded...Seals
...as a quality of implementation issue, but I would think the authors of the Standard would have thought it blindingly obvious that a quality implementation which can tell that a pointer of one type (e.g. int) is freshly derived from an object of another (e.g. struct foo) should treat an access using the former as an access to the latter in cases where that would yield defined behavior. Because it is so obvious, an explicit statement to that effect might have been viewed as condescending.Seals
PS--Why is there so much fascination with trying to make C compilers achieve Fortran-level performance on the tasks for which Fortran excels, and for which C wasn't designed, at the expense of making them unsuitable for the low-level and systems-programming purposes for which C was designed and is uniquely suitable? Today's Fortran is no longer the clunky language that was standardized in 1977.Seals
K
8

SWIG generates code that depends on strict aliasing being off, which can cause all sorts of problems.

SWIGEXPORT jlong JNICALL Java_com_mylibJNI_make_1mystruct_1_1SWIG_12(
       JNIEnv *jenv, jclass jcls, jint jarg1, jint jarg2) {
  jlong jresult = 0 ;
  int arg1 ;
  int arg2 ;
  my_struct_t *result = 0 ;

  (void)jenv;
  (void)jcls;
  arg1 = (int)jarg1; 
  arg2 = (int)jarg2; 
  result = (my_struct_t *)make_my_struct(arg1,arg2);
  *(my_struct_t **)&jresult = result;              /* <<<< horror*/
  return jresult;
}
Kami answered 22/9, 2011 at 17:19 Comment(1)
I would have no problem with type-based aliasing rules if they only applied in cases where an "outside" access to an lvalue's storage occurred between two "normal" accesses to it, and nothing else occurred between those accesses to suggest that an outside access might be likely. I've seen no evidence whatsoever that the authors of the Standard intended that quality implementations shouldn't support other forms of aliasing on platforms where they would be useful.Seals
M
6

gcc, aliasing, and 2-D variable-length arrays: The following sample code copies a 2x2 matrix:

#include <stdio.h>

static void copy(int n, int a[][n], int b[][n]) {
   int i, j;
   for (i = 0; i < 2; i++)    // 'n' not used in this example
      for (j = 0; j < 2; j++) // 'n' hard-coded to 2 for simplicity
         b[i][j] = a[i][j];
}

int main(int argc, char *argv[]) {
   int a[2][2] = {{1, 2},{3, 4}};
   int b[2][2];
   copy(2, a, b);    
   printf("%d %d %d %d\n", b[0][0], b[0][1], b[1][0], b[1][1]);
   return 0;
}

With gcc 4.1.2 on CentOS, I get:

$ gcc -O1 test.c && a.out
1 2 3 4
$ gcc -O2 test.c && a.out
10235717 -1075970308 -1075970456 11452404 (random)

I don't know whether this is generally known, and I don't know whether this a bug or a feature. I can't duplicate the problem with gcc 4.3.4 on Cygwin, so it may have been fixed. Some work-arounds:

  • Use __attribute__((noinline)) for copy().
  • Use the gcc switch -fno-strict-aliasing.
  • Change the third parameter of copy() from b[][n] to b[][2].
  • Don't use -O2 or -O3.

Further notes:

  • This is an answer, after a year and a day, to my own question (and I'm a bit surprised there are only two other answers).
  • I lost several hours with this on my actual code, a Kalman filter. Seemingly small changes would have drastic effects, perhaps because of changing gcc's automatic inlining (this is a guess; I'm still uncertain). But it probably doesn't qualify as a horror story.
  • Yes, I know you wouldn't write copy() like this. (And, as an aside, I was slightly surprised to see gcc did not unroll the double-loop.)
  • No gcc warning switches, include -Wstrict-aliasing=, did anything here.
  • 1-D variable-length arrays seem to be OK.

Update: The above does not really answer the OP's question, since he (i.e. I) was asking about cases where strict aliasing 'legitimately' broke your code, whereas the above just seems to be a garden-variety compiler bug.

I reported it to GCC Bugzilla, but they weren't interested in the old 4.1.2, even though (I believe) it is the key to the $1-billion RHEL5. It doesn't occur in 4.2.4 up.

And I have a slightly simpler example of a similar bug, with only one matrix. The code:

static void zero(int n, int a[][n]) {
   int i, j;
   for (i = 0; i < n; i++)
   for (j = 0; j < n; j++)
      a[i][j] = 0;
}

int main(void) {
   int a[2][2] = {{1, 2},{3, 4}};
   zero(2, a);    
   printf("%d\n", a[1][1]);
   return 0;
}

produces the results:

gcc -O1 test.c && a.out
0
gcc -O1 -fstrict-aliasing test.c && a.out
4

It seems it is the combination -fstrict-aliasing with -finline which causes the bug.

Miss answered 3/6, 2011 at 21:20 Comment(1)
Fascinating. The whole page is. I haven't lost much time to strict-aliasing (at least not that I know of), but certainly have lost an hour or so reading all the great posts and comments here!Oregano
S
5

here is mine:

http://forum.openscad.org/CGAL-3-6-1-causing-errors-but-CGAL-3-6-0-OK-tt2050.html

it caused certain shapes in a CAD program to be drawn incorrectly. thank goodness for the project's leaders work on creating a regression test suite.

the bug only manifested itself on certain platforms, with older versions of GCC and older versions of certain libraries. and then only with -O2 turned on. -fno-strict-aliasing solved it.

Silvertongued answered 17/12, 2011 at 18:54 Comment(4)
Did you ever figure out what the root cause was?Felucca
absolutely not, would have taken dozens and dozens of hours to track that down - for an obscure corner case when using older tools. CGAL is a massive C++ template hpp library and the binary could take an hour to compile.Silvertongued
Ha yes I'm a little familiar with it, and your characterization is 100% right. Wish I knew what it was, though - strict aliasing, despite being UB, seems to only rarely cause actual bugs.Felucca
it should be possible with a virtual machine to recreate the conditions, i think i listed the OS and tools in the bug report, which is probably still in the openscad mailing list archive. back then there were only about a dozen dependency libraries so you could probably grab those old versions as well and recreate the bug. with a modern CPU and SSD the compilation time would probably be much faster. like i said it would take many many hoursSilvertongued
S
3

The Common Initial Sequence rule of C used to be interpreted as making it possible to write a function which could work on the leading portion of a wide variety of structure types, provided they start with elements of matching types. Under C99, the rule was changed so that it only applied if the structure types involved were members of the same union whose complete declaration was visible at the point of use.

The authors of gcc insist that the language in question is only applicable if the accesses are performed through the union type, notwithstanding the facts that:

  1. There would be no reason to specify that the complete declaration must be visible if accesses had to be performed through the union type.

  2. Although the CIS rule was described in terms of unions, its primary usefulness lay in what it implied about the way in which structs were laid out and accessed. If S1 and S2 were structures that shared a CIS, there would be no way that a function that accepted a pointer to an S1 and an S2 from an outside source could comply with C89's CIS rules without allowing the same behavior to be useful with pointers to structures that weren't actually inside a union object; specifying CIS support for structures would thus have been redundant given that it was already specified for unions.

Seals answered 19/8, 2016 at 22:29 Comment(0)
S
1

The following code returns 10, under gcc 4.4.4. Is anything wrong with the union method or gcc 4.4.4?

int main()
{
  int v = 10;

  union vv {
    int v;
    short q;
  } *s = (union vv *)&v;

  s->v = 1;

  return v;
}
Stunt answered 8/10, 2010 at 20:4 Comment(14)
See gcc documentation - gcc.gnu.org/onlinedocs/gcc-4.8.2/gcc/Optimize-Options.html - regarding -fstrict-aliasing option. Specifically states that this method is invalid. For conforming type-punning, all accesses to the value must be through the union type; in your example the value 10 is not stored via the union type.Cement
@Cement What is an access through a type?Normi
@Normi by "access through the union type" I meant "access through the object which has the union type". Perhaps simply "access through the union object" would be better phrasing. I hope that clears it up.Cement
@Cement I don't know where "access through an object" is defined in std languageNormi
@Normi it's not, in exactly those words. Are you genuinely not able to understand what I mean or are you just complaining about the phrasing? Relevant parts of the (C99) standard are 3.1 (definition of "access"), 3.14 ("object"), 6.5.2.3 (structure and union members). An expression accessing a union member object either does or doesn't use the member access operator applied to an lvalue identifying the containing union object; by "access through the union object" I mean that it does.Cement
@Cement An access is "through a type" if that type is used in that expression? You can't decompose such expression with reference binding (C++) or taking the address of a lvalue than using it? Is there any other case in C/C++ where an expression with fully defined sub-expressions cannot be decomposed? (In C++, a member function call x.f() has two syntaxic components, member access f.x and call EXPR() that cannot be decomposed as x.f in itself is meaningless.) Is an access to a union member in C defined as a member function call in C++, with two syntaxic components for one operation?Normi
@Normi Yes. Bear in mind that the question (and my comments above) are about C not C++. FWIW I don't think there's any other case where you can't decompose expressions while keeping the same semantics and even this case isn't properly formalised in the actual C standard - it's just a consensus largely based on the notion that having to assume that any two pointers might point at members of the same union would pretty much prevent all type-based alias analysis and make 6.5p7 (strict aliasing) pointless. The "access through a type" term comes from the GCC documentation I linked above.Cement
...really matter if compilers behave like garbage impleentations in -fstrict-aliasing mode.Seals
@Seals the "consensus" I refer to is that reached by various compiler vendors, and nothing in the standard requires that a compiler be "non-garbage" by your own definition of "garbage". Please keep your comments objective.Cement
@davmac: The Standard, as noted in the Rationale, allows compilers to be "conforming" while being of such poor quality as to be useless. The Rationale also indicates that some actions are left as UB to allow quality of implementation to be an active force in the marketplace. The Standard would allow a conforming implementation to jump the rails when accessing someUnion.m1 if that lvalue has a non-character type, but I don't think anything is intended to imply that such behavior would be appropriate in a quality implementation. Is there anything in the Standard that would imply that...Seals
...quality implementations have a stronger obligation to handle accesses in that form than accesses which are decomposed in ways *that don't alias anything the root object wouldn't also be allowed to alias? Is there any reason why anyone seeking to produce a high-quality implementation should have any particular difficulty handling decomposed accesses in such cases?Seals
The code given in this answer uses an lvalue of type int to access an object of type int. The code may invoke Undefined Behavior on implementations where short has a coarser alignment requirement than int, or where a union of a short and an int would be larger than, or have coarser alignment requirements than, an int. I doubt any such implementations exist, but the Standard would not prohibit them. An implementation's documentation regarding storage representation should make any such quirks apparent.Seals
A conforming implementation would be entitled to output "10" given the above code because a conforming compiler implementation would entitled to output "10" when given any conforming program. An implementation that output 10 when given #include <stdio.h> int main(void) { printf("%d'\n",3*3); return 0;} because it exceeded a translation limit on the number of times the digit 3 may appear in a source text would likely be of such poor quality as to be almost useless, but that wouldn't make it non-conforming. Some compiler writers may interpret the Standard in a way...Seals
...that would regard as UB the use of the -> operator on anything other than a pointer to a pre-existing objects of aggregate type, and that's likely what's happening here. On the other hand, there are many situations were the authors of the Standard refrained from mandating things because they thought it sufficiently obvious that quality compilers should do them anyway, so it really shouldn't matter whether gcc was acting as a non-conforming compiler or merely a poor-quality one.Seals

© 2022 - 2024 — McMap. All rights reserved.