How does System V amd64 handle very long return values?

Question

I'm briefly studying the System V ABI for amd64 / x86-64 architecture, and am curious how it handles return values over 128 bits, where rax and rdx aren't enough.

I wrote the following C code on Ubuntu 18.04 64-bit (more generally, any amd64 POSIX-compliant system):

struct big {
    long long a, b, c, d;
};

struct big bigfunc(void) {
    struct big r = {12, 34, 56, 78};
    return r;
}

Compiled it as gcc -S -masm=intel t.c, and inspected t.s:

        .file   "t.c"
        .intel_syntax noprefix
        .text
        .globl  bigfunc
        .type   bigfunc, @function
bigfunc:
.LFB0:
        .cfi_startproc
        mov     QWORD PTR -40[rsp], rdi
        mov     QWORD PTR -32[rsp], 12
        mov     QWORD PTR -24[rsp], 34
        mov     QWORD PTR -16[rsp], 56
        mov     QWORD PTR -8[rsp], 78
        mov     rcx, QWORD PTR -40[rsp]
        mov     rax, QWORD PTR -32[rsp]
        mov     rdx, QWORD PTR -24[rsp]
        mov     QWORD PTR [rcx], rax
        mov     QWORD PTR 8[rcx], rdx
        mov     rax, QWORD PTR -16[rsp]
        mov     rdx, QWORD PTR -8[rsp]
        mov     QWORD PTR 16[rcx], rax
        mov     QWORD PTR 24[rcx], rdx
        mov     rax, QWORD PTR -40[rsp]
        ret
        .cfi_endproc
.LFE0:
        .size   bigfunc, .-bigfunc
        .ident  "GCC: (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0"
        .section        .note.GNU-stack,"",@progbits

No surprise that the struct definition doesn't compile into any instructions, so the output only contains function bigfunc. The output assembly looks pretty straightforward, allocating memory from stack for struct big r and assign initial values, and returning it.

If I am understanding correctly, before ret is executed, register rax contains the value of rdi at the beginning of the function call (from QWORD PTR -40[rbp]). According to SysV, rdi is the first argument supplied to the function, which is impossible because the function accepts no arguments. So I have a few questions here:

1. What is rdi when the function bigfunc takes no arguments?
2. What is rax (as the register that contains return value), when rdx is not touched in this function?
3. How does the function return this 256-bit C structure?

Respirator

This code would be much easier to understand if you enabled optimisations. TL;DR: as specified in the ABI document, structures are returned by passing a pointer to the structure as a hidden first argument. – Napier

@Napier Skimmed through the menu but didn't find the keyword "return value". Mind pointing out the section number? Thanks. – Respirator

It may not be in the amd64-specific document - this is a general rule that applies to all architectures. – Heresiarch

@Respirator See page 22. It's all part of §3.2.3. – Napier

@WumpusQ.Wumbley: what? it's up to each calling convention/ABI on each architecture to specify how large objects are returned. e.g. if multiple registers are used before falling back to memory, and if memory whether to pass a hidden pointer (usually as a first arg, but details need to be specified; it could pass in a register that's normally not an arg-passing register) or to leave the value on the stack (kind of the opposite of callee-pops for args), or whatever. – Intoxicate

All the calling conventions I'm familiar with pass a hidden first arg when falling back to memory, but x86-64 SysV and Windows x64 differ in how they return objects from 9 to 16 bytes. i386 SysV also differs from some 32-bit Windows conventions, e.g. never returning structs in edx:eax, only int64_t. (So that's a difference between i386 and x86-64 SysV in how they pack return vals into regs or not). The generic System V ABI definitely leaves it up to each psABI (platform supplement) to define the details of returning large objects; there's no universal convention. – Intoxicate

Answer 1

According to the ABI (1) ,page 22

If the type has class MEMORY, then the caller provides space for the return value and passes the address of this storage in %rdi as if it were the first argument to the function. In effect, this address becomes a “hidden” first ar- gument. This storage must not overlap any data visible to the callee through other names than this argument. On return %rax will contain the address that has been passed in by the caller in %rdi

Page 17, 18 and 19 describes the classifications, I beliveve the following on page 19 is the clause designating your struct big as a MEMORY class.

(c) If the size of the aggregate exceeds two eightbytes and the first eight- byte isn’t SSE or any other eightbyte isn’t SSEUP, the whole argument is passed in memory.

i.e. the caller have to allocate memory for the return value, and pass a pointer to that memory in %rdi (and the called function returns that same address in %rax)

(1) there's newer offical versions of the ABI at https://github.com/hjl-tools/x86-psABI/wiki/X86-psABI , though the links arn't currently working properly.

Cocteau

The struct big in my example is exactly 4 eightbytes, can you address that? – Respirator

See the updates. You likely have to read page 17,18 and 19 to view the rules yourself, they are not that straight forward. – Cocteau