kernel stack and user space stack

Question

What's the difference between kernel stack and user stack? Why kernel stack is used? If a local variable is declared in an ISR, where it will be stored? Does each process has its own kernel stack? Then how the process coordinates between both these stacks?

Binns

Answer 1

1. What's the difference between kernel stack and user stack ?

In short, nothing - apart from using a different location in memory (and hence a different value for the stack pointer register), and usually different memory access protections. I.e. when executing in user mode, kernel memory (part of which is the kernel stack) will not be accessible even if mapped. Vice versa, without explicitly being requested by the kernel code (in Linux, through functions like copy_from_user()), user memory (including the user stack) is not usually directly accessible.

2. Why is [ a separate ] kernel stack used ?

Separation of privileges and security. For one, user space programs can make their stack (pointer) anything they want, and there is usually no architectural requirement to even have a valid one. The kernel therefore cannot trust the user space stack pointer to be valid nor usable, and therefore will require one set under its own control. Different CPU architectures implement this in different ways; x86 CPUs automatically switch stack pointers when privilege mode switches occur, and the values to be used for different privilege levels are configurable - by privileged code (i.e. only the kernel).

3. If a local variable is declared in an ISR, where will it be stored ?

On the kernel stack. The kernel (Linux kernel, that is) does not hook ISRs directly to the x86 architecture's interrupt gates but instead delegates the interrupt dispatch to a common kernel interrupt entry/exit mechanism which saves pre-interrupt register state before calling the registered handler(s). The CPU itself when dispatching an interrupt might execute a privilege and/or stack switch, and this is used/set up by the kernel so that the common interrupt entry code can already rely on a kernel stack being present.
That said, interrupts that occur while executing kernel code will simply (continue to) use the kernel stack in place at that point. This can, if interrupt handlers have deeply nested call paths, lead to stack overflows (if a deep kernel call path is interrupted and the handler causes another deep path; in Linux, filesystem / software RAID code being interrupted by network code with iptables active is known to trigger such in untuned older kernels ... solution is to increase kernel stack sizes for such workloads).

4. Does each process have its own kernel stack ?

Not just each process - each thread has its own kernel stack (and, in fact, its own user stack as well). Remember the only difference between processes and threads (to Linux) is the fact that multiple threads can share an address space (forming a process).

5. How does the process coordinate between both these stacks ?

Not at all - it doesn't need to. Scheduling (how / when different threads are being run, how their state is saved and restored) is the operating system's task and processes don't need to concern themselves with this. As threads are created (and each process must have at least one thread), the kernel creates kernel stacks for them, while user space stacks are either explicitly created/provided by whichever mechanism is used to create a thread (functions like makecontext() or pthread_create() allow the caller to specify a memory region to be used for the "child" thread's stack), or inherited (by on-access memory cloning, usually called "copy on write" / COW, when creating a new process).
That said, the process can influence scheduling of its threads and/or influence the context (state, amongst that is the thread's stack pointer). There are multiple ways for this: UNIX signals, setcontext(), pthread_yield() / pthread_cancel(), ... - but this is digressing a bit from the original question.

Obi

@FrankH Excellent answer.. but I have small questions related to it, but in ARM architecture.. How this kernel stack is related to the different processor mode ? – Complot

@Rahul: "the margin of a SO comment is too small to contain such an answer". How stacks / stackpointer registers work in the different ARM CPU modes (which ones implement a banked SP) is a good question, but requires more space to answer than a comment can give. The same applies to things like x86 task gates or ISTs - there is no such thing as a "single" kernel stack pointer (just as there isn't a "single" user stack pointer), and what hardware support / mandate there is for separate stack pointers in different operating modes is ... very hardware-dependent. – Obi

@Obi I created new question for the same ... https://mcmap.net/q/161674/-how-kernel-stack-is-used-in-case-of-different-processor-mode-in-arm-architecture/769260 I hope now you can help me without caring of space :) – Complot

@Obi Can you provide a diagram showing where kernel stack belongs to in the memory layout of a process? – Insolvable

@JithinPavithran use the source, Luke - that diagram can be found in the Linux kernel code, git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/… for ARM64, or git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/… for x86_64. – Obi

x86 CPUs automatically switch stackpointers when privilege mode switches occur This is incorrect, at least in the way it is said. syscall instruction have the following property The SYSCALL instruction does not save the stack pointer (RSP). If the OS system-call handler will change the stack pointer, it is the responsibility of software to save the previous value of the stack pointer. – Lythraceous

Answer 2

My answer is collected from other SO questions with my stuffs.

What's the difference between kernel stack and user stack?

As a kernel programmer, you know that the kernel should be restricted from erroneous user programs. Suppose you keep same stack for both kernel & user space, then simple segfault in user application crashes kernel and needs restart.

There is one "kernel stack" per CPU like ISR Stack and one "kernel stack" per Process. There is one "user stack" for each process, though each thread has its own stack, including both user and kernel threads.

http://linux.derkeiler.com/Mailing-Lists/Kernel/2004-10/3194.html

Why kernel stack is used?

So when we are in kernel mode, stack kind of mechanism is necessary for dealing with function calls, local variables similar to user space.

http://www.kernel.org/doc/Documentation/x86/kernel-stacks

If a local variable is declared in an ISR, where it will be stored?

It will be stored in ISR stack(IRQSTACKSIZE). The ISR runs on a separate interrupt stack only if the hardware supports it. Otherwise, the ISR stack frames are pushed onto the stack of the interrupted thread.

The user space does not know and frankly does not care about whether the interrupt is served in current process's kernel stack or a separate ISR stack. As interrupts comes per cpu, therefore ISR stack has to be per cpu.

 Does each process has its own kernel stack ?

Yes. Each process has its own kernel stack.

 Then how the process coordinates between both these stacks?

@FrankH's answer looks great to me.

Lashkar

Handling interrupts on the user-space stack would also be a privilege escalation security vulnerability, because that memory still has read+write permission for user-space. Other threads of the same process could modify that memory during a sleep system call, for example, replacing a kernel return address with one that jumped to some kernel code which would set UID=0 for the process. – Emalia

Answer 3

1. What's the difference between kernel stack and user stack

Taking reference from Robert Love's Linux Kernel Development, the main difference is the size:

User-space can get away with statically allocating many variables on the stack, including huge structures and thousand-element arrays.
This behavior is legal because user-space has a large stack that can dynamically grow.
The kernel stack is neither large nor dynamic; it is small and fixed in size.
The exact size of the kernel’s stack varies by architecture.
On x86, the stack size is configurable at compile-time and can be either 4KB or 8KB.
Historically, the kernel stack is two pages, which generally implies that it is 8KB on 32-bit architectures and 16KB on 64-bit architectures—this size is fixed and absolute.
Each process receives its own stack.

Also the kernel stack contains a pointer to the thread_info struct holding information about the thread.

Harping