Can I use Thread Sanitizer for OpenMP programs?

Question

Consider the following example:

#include <iostream> 

int main () {
    int i = 0;
    #pragma omp parallel
    {
        #pragma omp critical
        {
            ++i;
        }
    }
    std::cout << i;
}

Compiling with g++ -fopenmp -fsanitize=thread and running yields

WARNING: ThreadSanitizer: data race (pid=9576)
Read of size 4 at 0x7ffdc170f600 by thread T1:
#0 main._omp_fn.0 (a.out+0x000000400d20)
#1 gomp_thread_start /build/gcc/src/gcc-5.2.0/libgomp/team.c:118 (libgomp.so.1+0x00000000f42d)

Previous write of size 4 at 0x7ffdc170f600 by thread T2:
#0 main._omp_fn.0 (a.out+0x000000400d35)
#1 gomp_thread_start /build/gcc/src/gcc-5.2.0/libgomp/team.c:118 (libgomp.so.1+0x00000000f42d)

Location is stack of main thread.

Thread T1 (tid=9578, running) created by main thread at:
#0 pthread_create /build/gcc/src/gcc-5.2.0/libsanitizer/tsan/tsan_interceptors.cc:895 (libtsan.so.0+0x000000027a37)
#1 gomp_team_start /build/gcc/src/gcc-5.2.0/libgomp/team.c:796 (libgomp.so.1+0x00000000f98f)
#2 __libc_start_main (libc.so.6+0x00000002060f)

Thread T2 (tid=9579, running) created by main thread at:
#0 pthread_create /build/gcc/src/gcc-5.2.0/libsanitizer/tsan/tsan_interceptors.cc:895 (libtsan.so.0+0x000000027a37)
#1 gomp_team_start /build/gcc/src/gcc-5.2.0/libgomp/team.c:796 (libgomp.so.1+0x00000000f98f)
#2 __libc_start_main (libc.so.6+0x00000002060f)

SUMMARY: ThreadSanitizer: data race ??:0 main._omp_fn.0

As far as I can see, this is a false positive. Is there a way to avoid this?

(Something working with clang and libomp would be fine too.)

Kobarid

You would need to build the OpenMP runtime library with TSAN as well if you want to analyze the code; otherwise add a suppression? – Gaal

@KerrekSB I was afraid that this is the only answer. Building modern stuff on my Ubuntu 12.04 work machine and getting it to work is not very fun. – Kobarid

@KerrekSB Can you elaborate on the suppression stuff? Do I need to do that for every false hit? In our production code, there are many, and if I have to check all of them "by hand", the sanitizer kind of loses its sense. – Kobarid

I think you should be able to suppress by code location. I've never done that (the last suppression set I wrote was for Valgrind), but check out the wiki: github.com/google/sanitizers/wiki/ThreadSanitizerSuppressions – Gaal

Answer 1

Yes, at least with Clang this is relatively easy. You'll need to build libomp (which Clang uses instead of libgomp) with ThreadSanitizer support. This doesn't take that long:

git clone https://github.com/llvm/llvm-project
cd llvm-project
mkdir build
cd build
cmake -DLIBOMP_TSAN_SUPPORT=1 ../openmp
sudo cmake --build . --target install

(sudo and --target install is optional if you adjust the path to libomp.so below)

Now running your example works without any errors if you use this libomp.so instead of the system one:

clang++ -fsanitize=thread -fopenmp main.cpp
env LD_PRELOAD=/usr/local/lib/libomp.so ./a.out

Platelet

Apparently this only works on Linux with a 64-bit Intel CPU. The CMake script checks if("${CMAKE_SYSTEM_NAME}" MATCHES "Linux" AND ${INTEL64}). – Buoyage

INTEL64 will be set for AMD CPUs (or other x86_64 CPUs). See the root CMakeLists.txt of llvm-project – Platelet

Sure. But for me the important bit there was to boot Linux, this unfortunately doesn't work on macOS. I thought it was relevant information to add to the answer. Once I was on Linux, this worked brilliantly! Many thanks! – Buoyage

Answer 2

even with the suppression you will still get false positives on the OpenMP runtime because there are some synchronization mechanism within the runtime that Tsan is not able to understand.

We worked on the OpenMP runtime to make Tsan understand this synchronization points and remove all the false positives.

Take a look to this project:

https://github.com/PRUNER/archer

Let me know if you need more help.

Best,

Simone

Devries

Thanks, I'll check it out when I get the chance. Might be a bit though, I'm working on something else now and a colleague does the sanitizer stuff. – Kobarid