Different memory alignment for different buffer sizes

Question

I'm trying to educate myself regarding stack overflows and played around a bit with these -fno-stack-protector flag and tried to understand how memory is managed in a process.

I compiled the following code (using Ubuntu 18.04.1 LTS (x86_64), gcc 7.3.0., ASLR disabled)

int main (int argc, char *argv[])
{
    char    buff[13];
    return 0;
}

as follows: gcc -g -o main main.c -fno-stack-protector. I then evoked gdb main, b 4, run and as can be seen from the the following outputs

(gdb) print &buff
$2 = (char (*)[13]) 0x7fffffffd963

0x7fffffffd963: 0xff    0xff    0x7f    0x00    0x00    0x00    0x00    0x00
0x7fffffffd96b: 0x00    0x00    0x00    0x00    0x00    0x10    0x46    0x55
0x7fffffffd973: 0x55    0x55    0x55    0x00    0x00    0x97    0x5b    0xa0
0x7fffffffd97b: 0xf7    0xff    0x7f    0x00    0x00    0x01    0x00    0x00

(gdb) info frame 0
Stack frame at 0x7fffffffd980:
 [...]
 Saved registers:
 rbp at 0x7fffffffd970, rip at 0x7fffffffd978

the 13 bytes allocated for the buffer follow directly after the saved base pointer rbp.

After increasing the buffer size from 13 to 21 I got the following results:

(gdb) print &buff   
$3 = (char (*)[21]) 0x7fffffffd950

(gdb) x/48bx buff
0x7fffffffd950: 0x10    0x46    0x55    0x55    0x55    0x55    0x00    0x00
0x7fffffffd958: 0xf0    0x44    0x55    0x55    0x55    0x55    0x00    0x00
0x7fffffffd960: 0x50    0xda    0xff    0xff    0xff    0x7f    0x00    0x00
0x7fffffffd968: 0x00    0x00    0x00    0x00    0x00    0x00    0x00    0x00
0x7fffffffd970: 0x10    0x46    0x55    0x55    0x55    0x55    0x00    0x00
0x7fffffffd978: 0x97    0x5b    0xa0    0xf7    0xff    0x7f    0x00    0x00

(gdb) info frame 0
Stack frame at 0x7fffffffd980:   
 [...]
 Saved registers:
  rbp at 0x7fffffffd970, rip at 0x7fffffffd978

Now there are additional 11 bytes after the rbp before the buffer follows.

• In the second case, why are there 11 additional bytes? Is this due to the alignment of the stack, e.g. does the buffer have to be 16 bytes aligned (a multiple of 16) starting from rbp?
• Why is the memory layout different in the first case, there seems to be no alignment?

Ideologist

I have checked that that rbp - &buff is equal sizeof(buff) when sizeof(buff) <= 16, otherwise it's equal to sizeof(buff)+15 rounded to 16, I mean: rbp - &buff = sizeof(buff) <= 16 ? sizeof(buff) : (sizeof(buff) + 15) / 16 * 16 – Chosen

@jww: you can't notify people that haven't commented or edited a post. I did see this because it's tagged x86-64, though. – Betty

Answer 1

The x86-64 System V ABI requires 16-byte alignment for local or global arrays that are 16 bytes or larger, and for all C99 VLAs (which are always local).

An array uses the same alignment as its elements, except that a local or global array variable of length at least 16 bytes or a C99 variable-length array variable always has alignment of at least 16 bytes.⁴

⁴ The alignment requirement allows the use of SSE instructions when operating on the array. The compiler cannot in general calculate the size of a variable-length array (VLA), but it is expected that most VLAs will require at least 16 bytes, so it is logical to mandate that VLAs have at least a 16-byte alignment.

Fixed-size arrays smaller than one SIMD vector (16 bytes) don't have this requirement, so they can pack efficiently in the stack layout.

Note that this doesn't apply to arrays inside structs, only to locals and globals.

(For dynamic storage, the alignment of a malloc return value must be aligned enough to hold any object up to that size, and since x86-64 SysV has maxalign_t of 16 bytes, malloc must also return 16-byte aligned pointers if the size is 16 or higher. For smaller allocations, it could return only 8B-aligned for an 8B allocation if it wanted to.)

---

The requirement for local arrays makes it safe to write code that passes their address to a function that requires 16-byte alignment, but this is mostly not something the ABI itself really needs to specify.

It's not something that different compilers have to agree on to link their code together, the way struct layout or the calling convention is (which registers are call-clobbered, or used for arg-passing...). The compiler basically owns the stack layout for the function it's compiling, and other functions can't assume or depend on anything about it. They'd only get pointers to your local vars if you pass pointers as function args, or store pointers into globals.

---

Specifying it for globals is useful, though: it makes it safe for compiler-generated auto-vectorized code to assume alignment for global arrays, even when it's an extern int[] in an object file compiled by another compiler.

Betty