Is integer overflow undefined in inline x86 assembly?
Asked Answered
C

2

12

Say I have the following C code:

int32_t foo(int32_t x) {
    return x + 1;
}

This is undefined behavior when x == INT_MAX. Now say I performed the addition with inline assembly instead:

int32_t foo(int32_t x) {
    asm("incl %0" : "+g"(x));
    return x;
}

Question: Does the inline assembly version still invoke undefined behavior when x == INT_MAX? Or does undefined behavior only apply to the C code?

Cortege answered 12/11, 2016 at 17:47 Comment(12)
it should set the overflow flag causing undefined behavior. If uint then flag wouldn't be setDemob
Why do you suppose it does not invoke ub when it's asm? (Really asking)Casimiracasimire
It doesn't have to be, you can test the overflow flag in your assembly code. What you'll do when it is set is still undefined :) You can't return from the function so you'll have to terminate the program. Which is roughly the reason that C made it UB, it was a lot easier that way. There are safeint libraries around that do this. Matters in critical fail-safe code, you can't launch a rocket with raw C code.Bloater
@HansPassant Sorry, I didn't quite understand that.. Do you mean that it is undefined behavior whenever the overflow flag is set, even if it is written in assembly?Cortege
No. Terminating your program when the overflow flag is set is not undefined behavior. If that has a side-effect, like blowing up a big expensive rocket, then somebody might disagree with that :)Bloater
@Demob at machine level there is no distinction between signed and unsigned addition. The machine does not know how you want to interpret the binary data, it sets a variety of flags (depending on the instruction) and then the coder or the compiler chooses which flags to test, depending on the context.Protanopia
@AndrewSun the addition is never undefined: the machine just does it. The problem is when you assign the result back to a signed variable when there was an overflow.Protanopia
If inline asm worked that way (using C rules instead of "whatever the platform actually does") it would be completely useless so it wouldn't even exist.Exogenous
The asm keyword is a compiler extension (§J.5.10). So anything and everything you do with inline assembly is implementation defined.Gumshoe
@HansPassant: I think C made it UB because they didn't want to standardize on 2's complement. If detecting signed wraparound is useful, then so is detecting unsigned wraparound, but it's well defined in C. Are you arguing that abort-on-overflow is only/major reason for making it UB, or did you think that's what the OP wanted to implement? Anyway, this LLVM blog post points out some modern optimizing-compiler advantages to signed-overflow UB, for stuff like for(int i...), so there are reasons other than 2's complementHallucinate
@PeterCordes: Allowing compilers to treat the result of overflow as non-deterministically holding the arithmetically-correct value, a "wrapped" value, or any other mathematical integer which would wrap to the same value, is useful. Thus, given long long x, x+1>y could be changed to x>=y since in the overflow case the expression x+1 would be allowed to behave as though it held a value one larger than the max value of a long long. Nearly all of the useful optimizations I've seen based on overflow being UB would still be available under that model, but...Revell
...the level of compiler freedom that would be available with correct code that exploited that model would be far greater than with code that must avoid overflows at all cost.Revell
H
17

No, there's no UB with this. C rules don't apply to the asm instructions themselves. As far as the inline-asm syntax wrapping the instructions, that's a well-defined language extension that has defined behaviour on implementations that support it.

See Does undefined behavior apply to asm code? for a more generic version of this question (vs. this one about x86 assembly and the GNU C inline asm language extension). The answers there focus on the C side of things, with quotes from the C and C++ standards that document how little the standard has to say about implementation-defined extensions to the language.

See also this comp.lang.c thread for arguments about whether it makes sense to say it has UB "in general" because not all implementations have that extension.


BTW, if you just want signed wraparound with defined 2's complement behaviour in GNU C, compile with -fwrapv. Don't use inline asm. (Or use an __attribute__ to enable that option for just the function that needs it.) wrapv is not quite the same thing as -fno-strict-overflow, which merely disables optimizations based on assuming the program doesn't have any UB; for example, overflow in compile-time-constant calculations is only safe with -fwrapv.


Inline-asm behaviour is implementation defined, and GNU C inline asm is defined as a black box for the compiler. Inputs go in, outputs come out, and the compiler doesn't know how. All it knows is what you tell it using the out/in/clobber constraints.


Your foo that uses inline-asm behaves identically to

int32_t foo(int32_t x) {
    uint32_t u = x;
    return ++u;
}

on x86, because x86 is a 2's complement machine, so integer wraparound is well-defined. (Except for performance: the asm version defeats constant propagation, and also gives the compiler no ability to optimize x - inc(x) to -1, etc. etc. https://gcc.gnu.org/wiki/DontUseInlineAsm unless there's no way to coax the compiler into generating optimal asm by tweaking the C.)

It doesn't raise exceptions. Setting the OF flag has no impact on anything, because GNU C inline asm for x86 (i386 and amd64) has an implicit "cc" clobber, so the compiler will assume that the condition codes in EFLAGS hold garbage after every inline-asm statement. gcc6 introduced a new syntax for asm to produce flag results (which can save a SETCC in your asm and a TEST generated by the compiler for asm blocks that want to return a flag condition).

Some architectures do raise exceptions (traps) on integer overflow, but x86 is not one of them (except when a division quotient doesn't fit in the destination register). On MIPS, you'd use ADDIU instead of ADDI on signed integers if you wanted them to be able to wrap without trapping. (Because it's also a 2's complement ISA, so signed wraparound is the same in binary as unsigned wraparound.)


Undefined (or at least implementation-dependent) Behaviour in x86 asm:

BSF and BSR (find first set bit forward or reverse) leave their destination register with undefined contents if the input was zero. (TZCNT and LZCNT don't have that problem). Intel's recent x86 CPUs do define the behaviour, which is to leave the destination unmodified, but the x86 manuals don't guarantee that. See the section on TZCNT in this answer for more discussion on the implications, e.g. that TZCNT/LZCNT/POPCNT have a false dependency on the output in Intel CPUs.

Several other instructions leave some flags undefined in some/all cases. (especially AF/PF). IMUL for example leaves ZF, PF, and AF undefined.

Presumably any given CPU has consistent behaviour, but the point is that other CPUs might behave differently even though they're still x86. If you're Microsoft, Intel will design their future CPUs to not break your existing code. If your code is that widely-relied-on, you'd better stick to only relying on behaviour documented in the manuals, not just what your CPU happens to do. See Andy Glew's answer and comments here. Andy was one of the architects of Intel's P6 microarchitecture.

These examples are not the same thing as UB in C. They're more like what C would call "implementation defined", since we're just talking about one value that's unspecified, not the possibility of nasal demons. (Or the more plausible modifying other registers, or jumping somewhere).

For really undefined behaviour, you probably need to look at privileged instructions, or at least multi-threaded code. Self-modifying code is also potentially UB on x86: it's not guaranteed that the CPU "notices" stores to addresses that are about to be executed until after a jump instruction. This was the subject of the question linked above (and the answer is: real implementations of x86 go above and beyond what the x86 ISA manual requires, to support code that depends on it, and because snooping all the time is better for high-performance than flushing on jumps.)

Undefined behaviour in assembly language is pretty rare, especially if you don't count cases where a specific value is unspecified but the scope of the "damage" is predictable and limited.

Hallucinate answered 12/11, 2016 at 18:40 Comment(6)
Very well explained, thank you! I actually didn't know about the implicit cc clobber (I left it out by mistake, to be honest :-p); if anyone is curious you can find it at the end of the ix86_md_asm_adjust function in the GCC source code.Cortege
@AndrewSun: re: "cc" clobbers: See also stackoverflow.com/questions/21541968/…. Also, I was digging in the mailing list archives of x86-64.org last year and came across a thread where they were deciding whether amd64 inline-asm would have the same implicit "cc" clobber, and they decided that yes, cost in missed optimizations is negligible since most asm statements would need it. And having it not be implicit had huge downsides in bugs. This was 15 years before gcc6 flag output were a thing. :PHallucinate
There is a huge difference between loading a register with an Unspecified value, versus Undefined Behavior. Some processors (e.g. DEC Alpha) have a category of Unpredictable behavior which is more constrained than Undefined Behavior, in that the former will remain bound by things like memory permissions, while the latter might not. Note that any Alpha instruction which could invoke Undefined behavior will trap unless executed from supervisor mode; user-mode instructions yield at worst Unspecified behavior.Revell
@supercat: hrm, yes that's an excellent point. UB in C isn't something you can recover from by not using that result. Updated to correct that last section. CPU designers can leave the potential for system-wide UB in privileged instructions, and just leave it up to the kernel to not cause a problem, without making it impossible to stop untrusted user-space from crashing the machine.Hallucinate
@PeterCordes: Since some platforms may not be able to efficiently support certain kinds of system programming, but could nonetheless usefully process many other kinds of C programs, the authors of the Standard didn't require that implementations provide everything needed to make system programming practical, but figured those implementations which could practically provide such features (e.g. the ability to safely perform relational comparisons on arbitrary pointers) would do so. If a piece of kernel code is supposed to be robust against being passed deliberately-malformed arguments, ...Revell
...having a compiler omit certain checks on the basis that certain things "couldn't" happen would be decidedly unhelpful. At a certain level, security code needs to be able to see things as they actually are, not just as the compiler thinks they should be.Revell
R
3

Well, the C Standard doesn't define what inline assembler does, so any inline assembler is undefined behaviour according to the C Standard.

You are using a slightly different language "C with x86 32 bit inline assembler". You generated a valid assembler statement. The behaviour is presumably defined by Intel's reference manuals. And there the behaviour of an integer addition adding 1 to INT_MAX is well defined. It's defined in a way that it doesn't interfere with execution of your C program.

Inline assembler that tried to read a value via a null pointer would also be well defined on the assembler level, but it's behaviour would interfere with the execution of your program (a.k.a. crashing it).

Ruelas answered 12/11, 2016 at 23:57 Comment(12)
A better way to put the first sentence would be that inline asm is implementation-defined behaviour. Although implementations can define behaviour for things that the C standard say is UB, if they want. (e.g. signed overflow).Hallucinate
@PeterCordes asm("incl %0" : "+g"(x)); is absolutely UB. C does not specify asm as some ID. It is UB by omission. ID refers to code that all compliant compliers handle in some implementation way. A compliant compiler need not handle asm.Cattegat
@chux: We're talking about the GNU dialect of the C language, where asm is a keyword and signed overflow (with C operators) is UB (without -fwrapv), exactly like in ISO C11. Would you be happier if the OP had used __asm__, which GNU C compilers will accept even with -std=c11 (as opposed to -std=gnu11)?Hallucinate
@PeterCordes It is not a question of my happiness. "C rules don't apply to the asm instructions." misleads. C defines the C language, not a GNU dialect of the C language. asm is UB per C. Of course it may be OK with GNU dialect of the C or other dialects. __asm__ is not specified the C spec either. Just because code is well specified by one complier does not make it OK for all C compliant compilers.Cattegat
@chux: My point was that strict C11 requires that the compiler not reserve asm as a keyword, but that anything to do with double-underscore names is implementation-defined. That is a separate point, and not related to the point you're making, though. I think I see what you're getting at, but it seems like pedantic nit-picking. Do you have any suggestions for wording (in my answer or gnasher's) that is as concise but avoids any mis-statements of the kind you're thinking of?Hallucinate
I wouldn't say that asm is UB according to the C standard. I'd rather say it's a parse error, since it's not a grammatically correct expression.Constrain
@PeterCordes This answer's wording is fine. How a compiler is to handle asm statements is not specified in C - therefore undefined behavior. C does not specify the language extension asm to have any behavior - implementation defined or otherwise. As to your answer, the lead "No, there's no UB with this" is directly against this. I do not see a change to your answer that retains that as being correct. As the answer is well UV'd, it would be difficult to change that premise.Cattegat
@chux: I think there's a difference between "not guaranteed to be even compile", vs. the official C meaning of the term Undefined Behaviour. It's an English phrase, but we should be careful to distinguish between UB in an otherwise-valid program vs. a program that might not even compile. The behaviour of the compiler when you feed it this program isn't defined by the standard, but that's not a case of the technical meaning of UB.Hallucinate
@PeterCordes: C89 includes a list of "Common extensions" and doesn't say that such extensions, despite being common, are non-conforming. Combined with the requirement that extensions be documented, I think one could reasonably infer that the intention is that conforming implementations may extend the language in the indicated fashions provided that they document that they are doing so. That isn't how the Standard is interpreted nowadays, but would be consistent with common practices in the early 1990s.Revell
@supercat: That seems to be the argument made by Keith Thomson and others in this comp.lang.c thread about whether using things like #include <unistd.h> or void main(){} that aren't defined by ISO C are actually "undefined behaviour". The general conclusion of everyone except one guy is that it's not UB, and is well-defined when compiled by an implementation that defines the behaviour of an extension. (GNU C inline asm is slightly different, because it's new syntax. Another post in that thread might address that.)Hallucinate
@chux: see my prev comment. If you want to read more about language extensions being UB or not, that thread covers a lot of ground. I meant to post about it sooner after reading it, but I do remember that what I read made me more confident about my argument and the way I phrased my answer.Hallucinate
@PeterCordes: The meaning of "Undefined Behavior" has shifted over the years. It used to be more widely recognized that there were a variety of things which could be sensibly defined on some implementations but not all, and in cases where it seemed obvious that some implementations should define things but others might not be able to, the authors of the Standard thought compiler writers would recognize the things they obviously should define, without the Standard having to single out particular implementations (something which may have been difficult politically).Revell

© 2022 - 2024 — McMap. All rights reserved.