FATAL: ThreadSanitizer: unexpected memory mapping when running on Linux Kernels 6.6+

Question

The Problem

Recently on Linux Kernels 6.6.6 and higher it was discovered that thread sanitizer will always lead to this error:

FATAL: ThreadSanitizer: unexpected memory mapping 0x5c9bd4d2b000-0x5c9bd4d4b000

I can reproduce this by writing a hello world example in C

#include <stdio.h>

int main(void)
{
    printf("Hello, World!\n");
}

And compiling it with tsan + running it on Arch Linux (Kernel 6.7.0) with:

clang -o play -fsanitize=thread -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer src/play.c
./play

This will produce the above error (with different memory addresses).

According to the github issue this will also occur in c++ files that just define an empty main() function.

The question

What are your options for dealing with this?

I have only recently started diving deeper into low-level operations and can barely use thread-sanitizer at my current knowledge levels. ASLR (which appears to be the cause of the problem) is entirely foreign to me, as are options to manipulate it and what the consequences are.

The github issue mentions 2 potential workarounds:

1. Disabling ASLR (I only found this SO question for this )
2. Reducing ASLR via sudo sysctl vm.mmap_rnd_bits=30

I have tried 2. and compiled + ran the example again as described, this did not resolve the issue.

I am hesitant to disable ASLR as per 1., as I wouldn't know what the implications of that are or how to "undo" that change.

What other general options to approach this are out there?

Carrasco

Answer 1

In this particular case, sudo sysctl vm.mmap_rnd_bits=30 is indeed sufficient, but requires a more modern clang version than arch-linux provides by default. The version at the time of writing this edit (16.0.6 17.0.6) stems from June 2023 November 2023

That release does not contain a fix from November 2023 that allows mmap_rnd_bits having a value of 30. This fix will be available with version 18.1.0 which has only been out since the start of March this year. It may therefore still take a fair bit of time until this fix makes it into the repositories of various distros.

However, at least on arch linux you can set this value down to 28 at which point it should work again with 16.0.6 and later. However the implication this appears to have is reduced security of the system (?), it definitely beats shutting ASLR off entirely.

You can set it to 28 via sudo sysctl vm.mmap_rnd_bits=28 (Check the value via sudo sysctl vm.mmap_rnd_bits).

Note that this change is not persistent. Once you reboot your machine, you will have to set mmap_rnd_bits again to 28.

Carrasco

That's too bad, as Ubuntu 24.04 comes with vm.mmap_rnd_bits=32 by default and so it would always require the custom sysctl setting to make TSAN work. – Plain

You might have luck with trying to set that kernel parameter (mmap_rnd_bits) on boot: linuxconfig.org/how-to-set-kernel-boot-parameters-on-linux – Carrasco