x86 32 bit opcodes that differ in x86-x64 or entirely removed

Asked 30/9, 2015 at 14:9 Answered 30/9, 2015 at 14:27

Solved assembly x86 x86-64 cpu-architecture opcode

I've looked up Wikipedia for x86 backward compatibility in x86-x64 and it says:

x86-64 is fully backwards compatible with 16-bit and 32-bit x86 code.Because the full x86 16-bit and 32-bit instruction sets remain implemented in hardware without any intervening emulation, existing x86 executables run with no compatibility or performance penalties,whereas existing applications that are recoded to take advantage of new features of the processor design may achieve performance improvements.

So I've tested some instructions to see that some are actually yield entirely different opcodes (rather than just applying prefix) such as: INC/DEC. Looking at (x86):

\x40 inc eax
\x48 dec eax

And while assembling the same in x86-x64 yields:

\xff \xc0 inc eax

\xff \xc8 dec eax

I'm trying to figure out the reason and more examples of other instructions that has the same symptoms that yield different opcodes. I am familiar with that push, pop, call, ret, enter and leave are not available 32 bit in x86-x64.

Raye answered 30/9, 2015 at 14:9 Comment(2)

16, 32, and 64bit push/pop are all encodable in amd64 (aka x86-64). The Intel insn set ref lists push r32 as not encodable in long mode, but actually you can, with a REX.W=0 prefix. (16 bit is encoded with an operand-size prefix for the usual opcode). Even push imm8 is available, but not push r/m8. You're right that call/ret and enter/leave always use 64bit operands / addresses in 64bit mode, though. – Rawalpindi 30/9, 2015 at 15:3

Oops, REX.W=0 push/pop is an illegal instruction in 64-bit mode. Only 16 and 64-bit push/pop are encodeable. – Rawalpindi 25/5, 2017 at 10:15

Almost all instructions that are available in both modes have the same opcodes in both modes.

Removed instructions:

Binary-coded-decimal stuff like AAM (ASCII-adjust after multiplication) for fixing up binary-coded-decimal after doing normal binary add/sub/mul/div on a register holding two base-10 digits in each 4-bit half. They ran slowly anyway, and weren't used. Storing numbers as binary integers instead of BCD is widespread.
push / pop of CS/DS/ES/SS were removed. push/pop FS and GS are still valid (those two segments can still have a non-zero base in long mode). mov Sreg, r32 and mov r32, Sreg are still available for the "neutered" segment registers, so you can emulate push / pop using a scratch integer reg. CS still matters; a far jump to another code segment can switch to 32-bit mode, and the others still need valid segment descriptors.
Other obscure segment stuff like ARPL: Adjust RPL Field of Segment Selector. It's really just a bit-field clamp and set flags instructions for integer registers, so could be emulated by a few other instructions in the rare places where a kernel might want it.
Maybe some other obscure or privileged instructions that compilers never used in 32-bit code. (Not that compilers ever emitted any of the above either, without intrinsics or inline asm.)

Removed (repurposed) encodings of some still-available instructions: In your case, 32-bit can use the inc r32 single-byte opcodes (0x40 + register-number). 64-bit mode only has the inc r/m32 encoding, where the register to be incremented is specified with a 2nd byte. (In this case, the 0x4x bytes were repurposed as the REX prefix byte.)

Intel's insn reference (follow the link in the x86 tag wiki) shows the following for inc:

Opcode   Instruction Op/   64-Bit   Compat/
                     En     Mode    Leg mode

FF /0   INC r/m32     M     Valid     Valid     Increment r/m doubleword by 1.
40+ rd  INC r32       O      N.E.     Valid     Increment doubleword register by 1.

N.E. means not encodable. The Op/En column describes how operands are encoded.

Jan Hubicka's AMD64 ISA overview briefly describes the repurposing of single-byte inc/dec opcodes for REX prefixes, and the default operand sizes and how immediate data is still 32-bit. movabs is available for loading 64-bit immediate constants, or load/store from/to a 64-bit absolute address.

AMD's AMD64 manual, Section 2.5.11 Reassigned Opcodes has a table which is quite short. It only lists:

4x inc/dec r32 that turned into REX prefixes
63 ARPL that became MOVSXD (sign-extend dword to qword, when used with REX.W=1 (which means the W bit in the REX prefix = 1)).

Early AMD64 and Intel EM64T CPUs left out SAHF/LAHF in long mode, but later re-added that instruction with the same opcode as in 32-bit. That table also doesn't list instructions that were removed entirely (the BCD instructions and maybe others) to make room for possible future extensions.

They could have simplified things a lot, and made x86-64 a much better cleaner instruction set with more room for future extensions, but every difference from 32-bit means more decoder transistors. There are no machine instructions that moved to a different opcode in 64-bit.

Multiple machine instructions often share the same asm mnemonic, mov being the most overloaded one. There are loads, stores, mov with immediate-constants, move to/from segment registers, all in 8-bit and 32-bit. (16-bit is the 32-bit with an operand-size prefix, same for 64-bit with a REX prefix.) There's a special opcode for loading RAX from a 64-bit absolute address. There's also a special opcode for loading a 64-bit immediate-constant into a register. (AT&T syntax calls this movabs, but it's still just mov in Intel/NASM)

Rawalpindi answered 30/9, 2015 at 14:22 Comment(6)

Thank you for the answer, but I still cannot understand how this makes x86-64 fully backwards compatible with 16-bit and 32-bit x86 code. Are there any other instructions that has different encodings such as INC/DEC? – Raye 30/9, 2015 at 14:29

Ah, I misunderstood what you were wondering about. To actually run existing unmodified 32bit code, a 64bit kernel runs 32bit binaries as compat-mode processes. See ElderBug's answer. Even if all the opcodes were the same, pointers in long mode would still be 64bits. The standard Linux process environment has bits set in the high32 of the stack pointer, so programs that modify / use only the low 32 ESP would segfault. – Rawalpindi 30/9, 2015 at 14:33

@DyCoder: also, calling it a "different encoding" for inc/dec misses the point. The FF C0 encoding for inc eax works just fine in 32bit x86. The 40 inc r32 encoding is used by assemblers for an x86 target because it's shorter, but it's not available for an amd64 target, so they use the inc r/m32 encoding. – Rawalpindi 30/9, 2015 at 14:37

Thank you. For my example lets say I'm trying to figure out if a certain code is either x86 or x86-x64 by examining the REX prefix for x86 instructions (not fully compatible-I know) but then I got to the case where INC/DEC where encoded totally differently. Any rule that I can keep in mind in what cases does that happen? – Raye 30/9, 2015 at 14:40

@DyCoder: so you have a binary blob? Executable and object files normally have flags in the file format indicating what kind of code they contain. I'm not sure what the best way to approach that would be if you don't have that info. Maybe there's something disassemblers can look for, but just try disassembling it both ways. If you get errors from the disassembler for one way but not the other, or it looks like nonsense one way, then there you go. e.g. if you see weird inc/dec instructions mixed in when you disassemble as 32bit, they were probably REX prefixes. – Rawalpindi 30/9, 2015 at 14:55

Yeah, that is sort of a binary blob, merely shellcode in chunks of data. AMD's AMD64 manual, Section 2.5.11 Reassigned Opcodes you references had just the answer for my other question, thanks! – Raye 1/10, 2015 at 11:53

The statement is true, but somewhat misleading. The x86-64 architecture is backward compatible with x86, but the 32-bit instruction set is not compatible with the 64-bit instruction set.

You can run x86 code on a x86-64 CPU by using a compatibility mode. Actually, since the CPU should be transparently x86 to x86 code, it's the opposite : you enter 64-bit mode (long mode) when you want to run x86-64 code. This means you can't run both at the same time, though it is possible to switch from one mode to another.

Insatiable answered 30/9, 2015 at 14:27 Comment(3)

Makes a lot more sense now... Is there by a chance a paper indicating more different opcode encodings that differs in x86 while in x86-x64 such as the example given with INC/DEC? The insructions I've seen so far (haven't much) besides INC/DEC did yield the same opcodes+prefix. – Raye 30/9, 2015 at 14:35

@DyCoder: I found a reference: AMD's AMD64 manual. Section 2.5.11 Reassigned Opcodes only lists the 4x inc/dec that turned into REX prefixes, and 63 ARPL that became MOVSXD (sign-extend dword to qword, when used with REX.W=1 (which means the W bit in the REX prefix = 1)). Early amd64 and intel emt64 CPUs left out SAHF/LAHF in long mode, but later re-added that instruction with the same opcode as in 32bit. That table also doesn't list the BCD instructions and maybe others that were removed to make room for extension. – Rawalpindi 30/9, 2015 at 14:49

@DyCoder I don't know of an exhaustive difference listing, but you can use some x64 listing and compare with some x86 listing. You can see that there is some invalid opcodes in the x64 one. You can also see that the 0xff 0xc0 is actually valid x86 (there is more inc instructions in x86). – Insatiable 30/9, 2015 at 14:49

Recommended topics

Hot tags