Assembling 32-bit binaries on a 64-bit system (GNU toolchain)
Asked Answered
P

2

20

I wrote assembly code that successfully compiles:

as power.s -o power.o

However, it fails when I try to link the object file:

ld power.o -o power

In order to run on the 64 bit OS (Ubuntu 14.04), I added .code32 at the beginning of the power.s file, however I still get the error:

Segmentation fault (core dumped)

power.s:

.code32
.section .data
.section .text
.global _start
_start:
pushl $3
pushl $2 
call power 
addl $8, %esp
pushl %eax 

pushl $2
pushl $5
call power
addl $8, %esp

popl %ebx
addl %eax, %ebx

movl $1, %eax
int $0x80



.type power, @function
power:
pushl %ebp  
movl %esp, %ebp 
subl $4, %esp 
movl 8(%ebp), %ebx 
movl 12(%ebp), %ecx 
movl %ebx, -4(%ebp) 

power_loop_start:
cmpl $1, %ecx 
je end_power
movl -4(%ebp), %eax
imull %ebx, %eax
movl %eax, -4(%ebp)

decl %ecx
jmp power_loop_start

end_power:
movl -4(%ebp), %eax 
movl %ebp, %esp
popl %ebp
ret
Proteinase answered 26/4, 2016 at 10:13 Comment(7)
What have you done to solve this already? Have you tried stepping through the code with a debugger to see where it crashes? Do you know what a segmentation fault is? (It has nothing to do with running 32 bit code on a 64 bit system).Whelm
.code32 doesn't set the object file format. The assembler will make 32bit code, but put it into a file that defaults to running as 64bit code. David has pointed out a separate problem, but it will still crash after fixing that. (Or at least get wrong answers. objdump -d power to disassemble the instructions the way your CPU will interpret them.) Use gcc -m32 -static -nostartfiles power.S -o power.Grice
@PeterCordes : I agree the big issue with segfaulting will be 32-bit code running as 64-bit program, but I don't believe that David's analysis is correct (I don't believe there is a stack issue with the ret that would cause a segfault). the result of 2^3 + 5^2 should be 33 (which it appears to be)Tamar
Looking again, you're right @MichaelPetch. I mistook the %ebp references at first glance this morning.Whelm
@DavidHoelzer : Not a problem David, been there done that!Tamar
Thanks for everyone's help. This problem is due to the "32-bit code running as 64-bit program" just like PeterCordes MichaelPetch said. I followed PeterCorde's gcc -m32 -static -nostartfiles power.S -o power Then I can successfully run the power. The -m32 option allows 32-bit code to be generated on the x86-64 platform. However, I don't know why I should use -static option to statically link programs. And I didn't find the -nostartfiles option's meaning. @PeterCordesProteinase
-static -nostartfiles replicates the behaviour of ld power.o -o power, but with the right -m option to make 32bit ELF executables. See the docs, or the gcc man page... Or build without those options, and use file and objdump -d to see the difference.Grice
G
23

TL:DR: use gcc -m32 -static -nostdlib foo.S (or equivalent as and ld options).
Or if you don't define your own _start, just gcc -m32 -no-pie foo.S

You may need to install gcc-multilib if you link libc, or however your distro packages /usr/lib32/libc.so, /usr/lib32/libstdc++.so and so on. But if you define your own _start and don't link libraries, you don't need the library package, just a kernel that supports 32-bit processes and system calls. This includes most distros, but not Windows Subsystem for Linux v1.

Don't use .code32

.code32 does not change the output file format, and that's what determines the mode your program will run in. It's up to you to not try to run 32bit code in 64bit mode. .code32 is for assembling kernels that have some 16 and some 32-bit code, and stuff like that. If that's not what you're doing, avoid it so you'll get build-time errors when you build a .S in the wrong mode if it has any push or pop instructions, for example. .code32 just lets you create confusing-to-debug runtime problems instead of build-time errors.

Suggestion: use the .S extension for hand-written assembler. (gcc -c foo.S will run it through the C preprocessor before as, so you can #include <sys/syscall.h> for syscall numbers, for example). Also, it distinguishes it from .s compiler output (from gcc foo.c -O3 -S).

To build 32-bit binaries, use one of these commands

gcc -g foo.S -o foo -m32 -nostdlib -static  # static binary with absolutely no libraries or startup code
                       # -nostdlib still dynamically links when Linux where PIE is the default, or on OS X

gcc -g foo.S -o foo -m32 -no-pie            # dynamic binary including the startup boilerplate code.
     # Use with code that defines a main(), not a _start

Documentation for nostdlib, -nostartfiles, and -static.


Using libc functions from _start (see the end of this answer for an example)

Some functions, like malloc(3), or stdio functions including printf(3), depend on some global data being initialized (e.g. FILE *stdout and the object it actually points to).

gcc -nostartfiles leaves out the CRT _start boilerplate code, but still links libc (dynamically, by default). On Linux, shared libraries can have initializer sections that are run by the dynamic linker when it loads them, before jumping to your _start entry point. So gcc -nostartfiles hello.S still lets you call printf. For a dynamic executable, the kernel runs /lib/ld-linux.so.2 on it instead of running it directly (use readelf -a to see the "ELF interpreter" string in your binary). When your _start eventually runs, not all the registers will be zeroed, because the dynamic linker ran code in your process.

However, gcc -nostartfiles -static hello.S will link, but crash at runtime if you call printf or something without calling glibc's internal init functions. (see Michael Petch's comment).


Of course you can put any combination of .c, .S, and .o files on the same command line to link them all into one executable. If you have any C, don't forget -Og -Wall -Wextra: you don't want to be debugging your asm when the problem was something simple in the C that calls it that the compiler could have warned you about.

Use -v to have gcc show you the commands it runs to assemble and link. To do it "manually":

as foo.S -o foo.o -g --32 &&      # skips the preprocessor
ld -o foo foo.o  -m elf_i386

file foo
foo: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped

gcc -nostdlib -m32 is easier to remember and type than the two different options for as and ld (--32 and -m elf_i386). Also, it works on all platforms, including ones where executable format isn't ELF. (But Linux examples won't work on OS X, because the system call numbers are different, or on Windows because it doesn't even use the int 0x80 ABI.)


NASM/YASM

gcc can't handle NASM syntax. (-masm=intel is more like MASM than NASM syntax, where you need offset symbol to get the address as an immediate). And of course the directives are different (e.g. .globl vs global).

You can build with nasm or yasm, then link the .o with gcc as above, or ld directly.

I use a wrapper script to avoid the repetitive typing of the same filename with three different extensions. (nasm and yasm default to file.asm -> file.o, unlike GNU as's default output of a.out). Use this with -m32 to assemble and link 32bit ELF executables. Not all OSes use ELF, so this script is less portable than using gcc -nostdlib -m32 to link would be..

#!/bin/bash
# usage: asm-link [-q] [-m32] foo.asm  [assembler options ...]
# Just use a Makefile for anything non-trivial.  This script is intentionally minimal and doesn't handle multiple source files
# Copyright 2020 Peter Cordes.  Public domain.  If it breaks, you get to keep both pieces

verbose=1                       # defaults
fmt=-felf64
#ldopt=-melf_i386
ldlib=()

linker=ld
#dld=/lib64/ld-linux-x86-64.so.2
while getopts 'Gdsphl:m:nvqzN' opt; do
    case "$opt" in
        m)  if [ "m$OPTARG" = "m32" ]; then
                fmt=-felf32
                ldopt=-melf_i386
                #dld=/lib/ld-linux.so.2  # FIXME: handle linker=gcc non-static executable
            fi
            if [ "m$OPTARG" = "mx32" ]; then
                fmt=-felfx32
                ldopt=-melf32_x86_64
            fi
            ;;
        #   -static
        l)  linker="gcc -no-pie -fno-plt -nostartfiles"; ldlib+=("-l$OPTARG");;
        p)  linker="gcc -pie -fno-plt -nostartfiles"; ldlib+=("-pie");;
        h)  ldlib+=("-Ttext=0x200800000");;   # symbol addresses outside the low 32.  data and bss go in range of text
                          # strace -e raw=write  will show the numeric address
        G)  nodebug=1;;      # .label: doesn't break up objdump output
        d)  disas=1;;
        s)  runsize=1;;
        n)  use_nasm=1 ;;
        q)  verbose=0 ;;
        v)  verbose=1 ;;
        z)  ldlib+=("-zexecstack") ;;
        N)  ldlib+=("-N") ;;   # --omagic = read+write text section
    esac
done
shift "$((OPTIND-1))"   # Shift off the options and optional --

src=$1
base=${src%.*}
shift

#if [[ ${#ldlib[@]} -gt 0 ]]; then
    #    ldlib+=("--dynamic-linker" "$dld")
    #ldlib=("-static" "${ldlib[@]}")
#fi

set -e
if (($use_nasm)); then
  #  (($nodebug)) || dbg="-g -Fdwarf"     # breaks objdump disassembly, and .labels are included anyway
    ( (($verbose)) && set -x    # print commands as they're run, like make
    nasm "$fmt" -Worphan-labels $dbg  "$src" "$@" &&
        $linker $ldopt -o "$base" "$base.o"  "${ldlib[@]}")
else
    (($nodebug)) || dbg="-gdwarf2"
    ( (($verbose)) && set -x    # print commands as they're run, like make
    yasm "$fmt" -Worphan-labels $dbg "$src" "$@" &&
        $linker $ldopt -o "$base" "$base.o"  "${ldlib[@]}" )
fi

# yasm -gdwarf2 includes even .local labels so they show up in objdump output
# nasm defaults to that behaviour of including even .local labels

# nasm defaults to STABS debugging format, but -g is not the default

if (($disas));then
    objdump -drwC -Mintel "$base"
fi

if (($runsize));then
    size $base
fi

I prefer YASM for a few reasons, including that it defaults to making long-nops instead of padding with many single-byte nops. That makes for messy disassembly output, as well as being slower if the nops ever run. (In NASM, you have to use the smartalign macro package.)

However, YASM hasn't been maintained for a while and only NASM has AVX512 support; these days I more often just use NASM.


Example: a program using libc functions from _start

# hello32.S

#include <asm/unistd_32.h>   // syscall numbers.  only #defines, no C declarations left after CPP to cause asm syntax errors

.text
#.global main   # uncomment these to let this code work as _start, or as main called by glibc _start
#main:
#.weak _start

.global _start
_start:
        mov     $__NR_gettimeofday, %eax  # make a syscall that we can see in strace output so we know when we get here
        int     $0x80

        push    %esp
        push    $print_fmt
        call   printf

        #xor    %ebx,%ebx                 # _exit(0)
        #mov    $__NR_exit_group, %eax    # same as glibc's _exit(2) wrapper
        #int    $0x80                     # won't flush the stdio buffer

        movl    $0, (%esp)   # reuse the stack slots we set up for printf, instead of popping
        call    exit         # exit(3) does an fflush and other cleanup

        #add    $8, %esp     # pop the space reserved by the two pushes
        #ret                 # only works in main, not _start

.section .rodata
print_fmt: .asciz "Hello, World!\n%%esp at startup = %#lx\n"

$ gcc -m32 -nostdlib hello32.S
/tmp/ccHNGx24.o: In function `_start':
(.text+0x7): undefined reference to `printf'
...
$ gcc -m32 hello32.S
/tmp/ccQ4SOR8.o: In function `_start':
(.text+0x0): multiple definition of `_start'
...

Fails at run-time, because nothing calls the glibc init functions. (__libc_init_first, __dl_tls_setup, and __libc_csu_init in that order, according to Michael Petch's comment. Other libc implementations exist, including MUSL which is designed for static linking and works without initialization calls.)

$ gcc -m32 -nostartfiles -static hello32.S     # fails at run-time
$ file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, BuildID[sha1]=ef4b74b1c29618d89ad60dbc6f9517d7cdec3236, not stripped
$ strace -s128 ./a.out
execve("./a.out", ["./a.out"], [/* 70 vars */]) = 0
[ Process PID=29681 runs in 32 bit mode. ]
gettimeofday(NULL, NULL)                = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)

You could also gdb ./a.out, and run b _start, layout reg, run, and see what happens.


$ gcc -m32 -nostartfiles hello32.S             # Correct command line
$ file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=7b0a731f9b24a77bee41c13ec562ba2a459d91c7, not stripped

$ ./a.out
Hello, World!
%esp at startup = 0xffdf7460

$ ltrace -s128 ./a.out > /dev/null
printf("Hello, World!\n%%esp at startup = %#lx\n", 0xff937510)      = 43    # note the different address: Address-space layout randomization at work
exit(0 <no return ...>
+++ exited (status 0) +++

$ strace -s128 ./a.out > /dev/null        # redirect stdout so we don't see a mix of normal output and trace output
execve("./a.out", ["./a.out"], [/* 70 vars */]) = 0
[ Process PID=29729 runs in 32 bit mode. ]
brk(0)                                  = 0x834e000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
....   more syscalls from dynamic linker code
open("/lib/i386-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
mmap2(NULL, 1814236, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffffffff7556000    # map the executable text section of the library
... more stuff
# end of dynamic linker's code, finally jumps to our _start

gettimeofday({1461874556, 431117}, NULL) = 0
fstat64(1, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 3), ...}) = 0  # stdio is figuring out whether stdout is a terminal or not
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0xff938870) = -1 ENOTTY (Inappropriate ioctl for device)
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffffffff7743000      # 4k buffer for stdout
write(1, "Hello, World!\n%esp at startup = 0xff938fb0\n", 43) = 43
exit_group(0)                           = ?
+++ exited with 0 +++

If we'd used _exit(0), or made the sys_exit system call ourselves with int 0x80, the write(2) wouldn't have happened. With stdout redirected to a non-tty, it defaults to full-buffered (not line-buffered), so the write(2) is only triggered by the fflush(3) as part of exit(3). Without redirection, calling printf(3) with a string containing newlines will flush immediately.

Behaving differently depending on whether stdout is a terminal can be desirable, but only if you do it on purpose, not by mistake.

Grice answered 27/4, 2016 at 21:57 Comment(6)
When building with -nostartfiles -static it can be dangerous in many environments if the C runtime needs to be run ahead of time. For simple things you'll probably not have a problem, but even printf can become a problem. That is why if you intend to use glibc statically, your code should call __libc_init_first, __dl_tls_setup, __libc_csu_init to be called manually (in that order) at program startup. You can avoid this by using C libraries like MUSL which require no initialization before functions are called. They are designed for static linkingTamar
You can get away with dynamic linking of GLIBC because that shared object will have its initialization code called automatically by the dynamic linker which will do those initialization calls.Tamar
@MichaelPetch: I did mention that in the comment above that line, if you scroll to the right... I didn't want to clutter the answer, but maybe that should be more prominent. Oh right, maybe my comment is wrong, because it only works with dynamic linking and -nostartfiles. Anyway, gtg, bbl in a couple hours.Grice
Edit if you have any nice ideas for how to present that info, otherwise I will at some point.Grice
Take care. Coincidentally I discovered we shared an answer related to this previously stackoverflow.com/questions/35208824/… . I had forgotten about it, although it was in the context of reducing code size so I never mentioned the initialization sequence needed for static linking to work.Tamar
@MichaelPetch: Is this too much stuff in one answer? I think I kept the essential info up front before wandering off into big examples and stuff. The x86 tag wiki links this answer for building 32bit code on a 64bit machine, so I figured that sending a newbie to this question might clear up other confusions, too, as well as illustrate strace and ltrace, and at least mention gdb.Grice
J
7

I'm learning x86 assembly (on 64-bit Ubuntu 18.04) and had a similar problem with the exact same example (it's from Programming From the Ground Up, in chapter 4 [http://savannah.nongnu.org/projects/pgubook/ ]).

After poking around I found the following two lines assembled and linked this:

as power.s -o power.o --32  
ld power.o -o power -m elf_i386

These tell the computer that you're only working in 32-bit (despite the 64-bit architecture).

If you want to use gdb debugging, then use the assembler line:

as --gstabs power.s -o power.o --32.

The .code32 seems to be unnecessary.

I haven't tried it your way, but the gnu assembler (gas) also seems okay with:
.globl start
# (that is, no 'a' in global).

Moreover, I'd suggest you probably want to keep the comments from the original code as it seems to be recommended to comment profusely in assembly. (Even if you are the only one to look at the code, it'll make it easier to figure out what you were doing if you look at it months or years later.)

It would be nice to know how to alter this to use the 64-bit R*X and RBP, RSP registers though.

Jessee answered 16/10, 2018 at 5:5 Comment(1)
Correct, .code32 is useless here, as I explained in my answer. And yes, .globl is the normal GAS directive. .global is an alias for it.Grice

© 2022 - 2024 — McMap. All rights reserved.