Using memory sanitizer with libstdc++

Question

I wish to use the -fsanitize=memory flag in clang to analyse a program like the following:

#include <string>
#include <iostream>
#include <fstream>
using namespace std;

void writeToFile(){
    ofstream o;
    o.open("dum");
    o<<"test"<<endl; //The error is here.
                     //It does not matter if the file is opened this way,
                     //or with o("dum");
    o.close();
}
int main(){
    writeToFile();
}

As far as I know, this program is correct, but when I use clang++ san.cpp -fsanitize=memory It fails (at runtime) with:

UMR in __interceptor_write at offset 0 inside [0x64800000e000, +5)  
==9685== WARNING: MemorySanitizer: use-of-uninitialized-value  
    #0 0x7f48d0899ae5 (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x7bae5)  
    #1 0x7f48d08d1787 (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb3787)  
    #2 0x7f48d08d21e2 (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb41e2)  
    #3 0x7f48d08cfd1e (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb1d1e)  
    #4 0x7f48d08b1f2d (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x93f2d)  
    #5 0x7f48d16d60f5 in writeToFile() /home/daniel/programming/test/santest.cpp:10  
    #6 0x7f48d16d61f4 in main /home/daniel/programming/test/santest.cpp:15  
    #7 0x7f48d0261de4 (/lib/x86_64-linux-gnu/libc.so.6+0x21de4)  
    #8 0x7f48d16d5e42 in _start (/home/daniel/programming/test/a.out+0x61e42)  

SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 ??

How can I make this work properly?

Clang version 3.5, stdlibc++ version 6

Ruralize

Could be handy to comment in the source code line 10 so we can more easily relate it to the error message. – Sklar

Does it give a similar error if you use ofstream o("dum"); instead of o.open("dum");? – Sklar

which version of clang, and which version of libstdc++? – Encore

@ChrisCleeland I'm on 3.5, and how can I get the libstdc++ version? – Ruralize

If it's standard for your platform, use your platform package manager to see what the version is. However, I don't think that'll help either of us. I perused the docs because I thought there was a way to specify an "ignore" file, but I can't find that in the docs anywhere right now. I'll keep looking. – Encore

@ChrisCleeland I am on version 6 (if that makes sense) – Ruralize

stdlibc++ - I'm not sure that's a real runtime :) GNU's runtime is libstdc++. LLVM's runtime is libc++ (sometimes called libcxx). I think the next step for you is to build the LLVM runtime with Msan instrumentation. You can find instructions at Memory Sanitizer Libcxx HowTo. – Montage

Answer 1

The code is fine, of course but many similar errors are cause by the following requirement of clang's memory sanitizer tool:

MemorySanitizer (without a dynamic component) requires that the entire program code including libraries, (except libc/libm/libpthread, to some extent), is instrumented.

from here

The cplusplus runtime you are using libstdc++ is unistrumented and causes errors. You will unfortunately have to follow a somewhat fiddly process as described at that link to rebuild an instrumented libstdc++ or switch to libc++ (easier-ish)

Horus

Any chance you can figure out the ../../../libstdc++-v3/configure bit? What configure is it talking about? – Ruralize

@Ruralize the configuration script for -- wait what have you done so far – Horus

it is a standard build for gcc (in no way a standard build though! :)) you create some folders outside the source tree and muck around - i will find a good tutorial (there's one on OSDev wiki IRRC) and write you a thorough answer in the morning – Horus

I have a standard build tree (SomeLocalDir, build dir, etc) – Ruralize

I have a standard build tree (SomeLocalDir, build dir, etc). I believe I fixed the path issue. I am having other issues, in terms of after making max_align_t=16, I then get the error: "No rule to make target ../src/c++11/libc++11convenience.la', needed by libstdc++.la" Any ideas? – Ruralize

@Ruralize while building gcc (i assume gcc is your system compiler) or when rebuilding libstdc++? If the later, did you patch the code? – Horus

@Ruralize i couldn't even make it that far because gcc was crashing when compiling, maybe i can try later. did you patch the code? – Horus

Yes, but that is not the issue – Ruralize

let us continue this discussion in chat – Horus

You don't need a tutorial for building GCC unless you want to do it wrong. See gcc.gnu.org/wiki/InstallingGCC for the right way. – Copalm

@JonathanWakely what exactly do you mean by that? that link contains all the information i was referring too anyway, but in my experience it is never simple - there is always some incompatible code, changes to paths in makefiles, etc - if you are familiar with gcc, is there any chance you could make an answer detailing a build of (latest) libstd++ with clang? i think it is really needed here – Horus

@user3125280, sounds like you're doing it wrong, you don't build libstdc++ with clang, you build it with GCC. Building Clang to use libstdc++ is a bit harder if it's in a non-standard location, they kept changing how to do that, now I think you configure clang with --with-gnu-toolchain or something similar. But a tutorial for building GCC won't help with that either. – Copalm

Answer 2

There easiest way at this time is to build libc++ with memorysanitizer, then link your program against it.

This is how I did it some time ago, failing to handle libc++ build system: https://code.google.com/p/memory-sanitizer/source/browse/bootstrap/build_libcxx.sh

I heard there've been improvements on the libc++ side, maybe it would be possible to build it as usual (with something like CC=/path/to/clang CFLAGS=-fsanitize=memory).

Tea

you can use libstdc++ but you will have to patch it then build gcc specially – Horus

@Horus so I can just run your script? – Ruralize

@Ruralize it ain't mine and i ain't used it (i think you mean eugenis' script) - my alternative script/bash commands) are clearly well documented, you will just have to follow the instructions and give it a shot. (i've built gcc with non standard libstdc++ but not with clang and not the suggested script - checking out/bulding gcc takes a while too at severl gigabytes) – Horus

I get am unable to pass a good $LLVM_BIN$ I think. I tried /usr/local/bin` and /build/bin ideas? – Ruralize

LLVM_BIN stands for the "bin" directory in LLVM build tree, the one where clang and clang++ are. You may need to tweak the script a little to work with the system installation of clang. – Tea

Answer 3

How can I make this work properly?

You can also unpoison the memory that's triggering the finding. But its not clear (to me) which variable that is based on the stack trace shown.

Here's how to unpoison the memory, but the example is for memory used with FD_SET and FD_ZERO. You will still need to find the name of the variable that's causing it (I'm not sure how well specifying an integral memory address works).

#include <sanitizer/msan_interface.h>
...

__msan_unpoison(&readfds, sizeof(readfds));
__msan_unpoison(&writefds, sizeof(writefds));

---

UMR in __interceptor_write at offset 0 inside [0x64800000e000, +5)  
==9685== WARNING: MemorySanitizer: use-of-uninitialized-value  
    #0 0x7f48d0899ae5 (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x7bae5)  
    #1 0x7f48d08d1787 (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb3787)  
    #2 0x7f48d08d21e2 (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb41e2)  
    #3 0x7f48d08cfd1e (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb1d1e)  
    #4 0x7f48d08b1f2d (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x93f2d)  
    #5 0x7f48d16d60f5 in writeToFile() /home/daniel/programming/test/santest.cpp:10  
    #6 0x7f48d16d61f4 in main /home/daniel/programming/test/santest.cpp:15  
    #7 0x7f48d0261de4 (/lib/x86_64-linux-gnu/libc.so.6+0x21de4)  
    #8 0x7f48d16d5e42 in _start (/home/daniel/programming/test/a.out+0x61e42)  

You may be able to get more information about the offenders by running:

./myprog.exe 2>&1 | /usr/bin/asan_symbolize

For example, here's a program I'm trying to test that has output similar to yours:

$ ./cryptest.exe v 2>&1 | /usr/bin/asan_symbolize
==26988== WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7f51903b2ca8 in _ZNSt8_Rb_treeISsSt4pairIKSsPvESt10_Select1stIS3_ESt4lessISsESaIS3_EE14_M_lower_boundEPSt13_Rb_tree_nodeIS3_ESC_RS1_ /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_tree.h:1260 (discriminator 1)
    ...

If you are up for some punishment, you can pipe the mangled name through c++filt and get a non-mangled name:

$ echo " _ZNSt8_Rb_treeISsSt4pairIKSsPvESt10_Select1stIS3_ESt4lessISsESaIS3_EE14_M_lower_boundEPSt13_Rb_tree_nodeIS3_ESC_RS1_" | c++filt
std::_Rb_tree<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, void*>, std::_Select1st<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, void*> >, std::less<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, void*> > >::_M_lower_bound(std::_Rb_tree_node<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, void*> >*, std::_Rb_tree_node<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, void*> >*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)

---

Finally, according to the Msan folks, you really need an instrumented build of the C++ Runtime. They also recommend you use LLVM's libc++ for the purpose. See Memory Sanitizer Libcxx HowTo and How to unpoison a C++ std::string? on the Memory Sanitizer mailing list.

Montage

Very useful in other contexts. – Ruralize