Where are the stacks for the other threads located in a process virtual address space?

Question

The following image shows where the sections of a process are laid out in the process's virtual address space (in Linux):

You can see that there is only one stack section (since this process only has one thread I assume).

But what if this process has another thread, where will the stack for this second thread be located? will it be located immediately below the first stack?

Galatia

C tag is not relevant, C doesn't define their concepts. – Samaria

Also see The Stack Clash. Qualys managed to break the separation between logical memory regions. They could use the one memory region to overwrite objects in another region. Linux was especially vulnerable (except systems running with gresecurity patches). Also see More CONFIG_VMAP_STACK vulnerabilities on the OSS-Security mailing list. – Jesu

Answer 1

Stack space for a new thread is created by the parent thread with mmap(MAP_ANONYMOUS|MAP_STACK). So they're in the "memory map segment", as your diagram labels it. It can end up anywhere that a large malloc() could go. (glibc malloc(3) uses mmap(MAP_ANONYMOUS) for large allocations.)

(MAP_STACK is currently a no-op, and exists in case some future architecture needs special handling).

You pass a pointer to the new thread's stack space to the clone(2) system call which actually creates the thread. (Try using strace -f on a multi-threaded process sometime). See also this blog post about creating a thread using raw Linux syscalls.

See this answer on a related question for some more details about mmaping stacks. e.g. MAP_GROWSDOWN doesn't prevent another mmap() from picking the address right below the thread stack, so you can't depend on it to dynamically grow a small stack the way you can for the main thread's stack (where the kernel reserves the address space even though it's not mapped yet).

So even though mmap(MAP_GROWSDOWN) was designed for allocating stacks, it's so bad that Ulrich Drepper proposed removing it in 2.6.29.

---

Also, note that your memory-map diagram is for a 32-bit kernel. A 64-bit kernel doesn't have to reserve any user virtual-address space for mapping kernel memory, so a 32-bit process running on an amd64 kernel can use the full 4GB of virtual address space. (Except for the low 64k by default (sysctl vm.mmap_min_addr = 65536), so NULL-pointer dereference does actually fault. And the top page is also reserved as error codes, not valid pointers.)

---

Related:

See Relation between stack limit and threads for more about stack-size for pthreads. getrlimit(RLIMIT_STACK) is the main thread's stack size. Linux pthreads uses RLIMIT_STACK as the stack size for new threads, too.

Bryson

"64-bit kernel doesn't have to reserve any user virtual-address space for mapping kernel memory" — wait, really? How would that work? I've tried to do something like this but failed because IIRC addresses in the IDTR/GDTR are linear, not physical so they just had to be mapped in the user-space. – Kampala

@Joker_vD: Right, but they're mapped outside the low 4GiB that 32-bit user-space can access. The Q&A linked in that paragraph shows Linux /proc/<PID>/maps output for 32-bit user-space under a 64-bit kernel with the stack at fffdd000-ffffe000. A 64-bit user-space process does only get the low half of canonical addresses, with the kernel reserving the high half like you were thinking of. – Bryson