in X86 - linux segment registers are used for buffer overflow check [see the below code snippet which have defined some char arrays in stack] :
static void
printint(int xx, int base, int sgn)
{
char digits[] = "0123456789ABCDEF";
char buf[16];
int i, neg;
uint x;
neg = 0;
if(sgn && xx < 0){
neg = 1;
x = -xx;
} else {
x = xx;
}
i = 0;
do{
buf[i++] = digits[x % base];
}while((x /= base) != 0);
if(neg)
buf[i++] = '-';
while(--i >= 0)
my_putc(buf[i]);
}
Now if we see the dis-assembly of the code gcc-generated code.
Dump of assembler code for function printint:
0x00000000004005a6 <+0>: push %rbp
0x00000000004005a7 <+1>: mov %rsp,%rbp
0x00000000004005aa <+4>: sub $0x50,%rsp
0x00000000004005ae <+8>: mov %edi,-0x44(%rbp)
0x00000000004005b1 <+11>: mov %esi,-0x48(%rbp)
0x00000000004005b4 <+14>: mov %edx,-0x4c(%rbp)
0x00000000004005b7 <+17>: mov %fs:0x28,%rax ------> obtaining an 8 byte guard from based on a fixed offset from fs segment register [from the descriptor base in the corresponding gdt entry]
0x00000000004005c0 <+26>: mov %rax,-0x8(%rbp) -----> pushing it as the first local variable on to stack
0x00000000004005c4 <+30>: xor %eax,%eax
0x00000000004005c6 <+32>: movl $0x33323130,-0x20(%rbp)
0x00000000004005cd <+39>: movl $0x37363534,-0x1c(%rbp)
0x00000000004005d4 <+46>: movl $0x42413938,-0x18(%rbp)
0x00000000004005db <+53>: movl $0x46454443,-0x14(%rbp)
...
...
// function end
0x0000000000400686 <+224>: jns 0x40066a <printint+196>
0x0000000000400688 <+226>: mov -0x8(%rbp),%rax -------> verifying if the stack was smashed
0x000000000040068c <+230>: xor %fs:0x28,%rax --> checking the value on stack is matching the original one based on fs
0x0000000000400695 <+239>: je 0x40069c <printint+246>
0x0000000000400697 <+241>: callq 0x400460 <__stack_chk_fail@plt>
0x000000000040069c <+246>: leaveq
0x000000000040069d <+247>: retq
Now if we remove the stack based char arrays from this function , gcc won't generate this guard check .
I have seen the same generated by gcc even for kernel modules. Basically I was seeing a crash while botrapping some kernel code and it was faulting with virtual address 0x28. Later I figured that thought i had initialized the stack pointer correctly and loaded the program correctly, I am not having the right entries in gdt, which would translate the fs based offset into a valid virtual address.
However in case of kernel code it was simply ignoring , the error instead of jumping to something like __stack_chk_fail@plt>.
The relevant compiler option which adds this guard in gcc is -fstack-protector . I think this is enabled by default which compiling a user app.
For kernel , we can enable this gcc flag via config CC_STACKPROTECTOR option.
config CC_STACKPROTECTOR
699 bool "Enable -fstack-protector buffer overflow detection (EXPERIMENTAL)"
700 depends on SUPERH32
701 help
702 This option turns on the -fstack-protector GCC feature. This
703 feature puts, at the beginning of functions, a canary value on
704 the stack just before the return address, and validates
705 the value just before actually returning. Stack based buffer
706 overflows (that need to overwrite this return address) now also
707 overwrite the canary, which gets detected and the attack is then
708 neutralized via a kernel panic.
709
710 This feature requires gcc version 4.2 or above.
The relevant kernel file where this gs / fs is linux/arch/x86/include/asm/stackprotector.h