How to compile a static musl binary of a Rust project with native dependencies?

Question

I have a project with dependencies on Hyper and Diesel, and because of that, on native libraries OpenSSL and libpq. The project builds on nightly Rust because it uses compiler plugins.

My current attempt is to build on a Docker container. I have the MUSL libc and the libraries make'd and installed with prefix /usr/local/musl. I run cargo with the following command: (Not sure if some of the options are redundant, I'm not too well-versed with the compiler chain, and not even sure if they end up to the linker, but I have to try, right.)

LDFLAGS="-static -L/usr/local/musl/lib" \
LD_LIBRARY_PATH=/usr/local/musl/lib:$LD_LIBRARY_PATH \
CFLAGS="-I/usr/local/musl/include" \
PKG_CONFIG_PATH=/usr/local/musl/lib/pkgconfig \
cargo build --release --target=x86_64-unknown-linux-musl

When I ldd the resulting file, it reveals this:

$ ldd server
linux-vdso.so.1 (0x00007fffb878e000)
libpq.so.5 => /usr/local/musl/lib/libpq.so.5 (0x00007f4d730e7000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f4d72e82000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f4d72a85000)
libc.so => /usr/local/musl/lib/libc.so (0x00007f4d727f6000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4d725f2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4d72246000)
/lib/ld64.so.1 => /lib64/ld-linux-x86-64.so.2 (0x000055e2124a2000)

There's all that dynamically linked stuff, and some even with the "x86_64-linux-gnu" chain! What went wrong?

I can make statically linked, simple pure-Rust projects without problems. ldd says that they are statically linked, and they run without problems, unlike the executable I have problems with.

When I used --verbose with Cargo, I got the following rustc command that actually builds the executable: http://pastebin.com/ywv0zNBK (Oops, that one had a custom outdir and -Z print-link-args, added by me) Adding the print-link-args flag, I got the following linker command: http://pastebin.com/Aw43qd7h

How do I get cargo or rustc to believe that I want a static binary?

Despiteful

I'm pretty sure the answer is: "you don't, easily". You will need to rebuild libpq and OpenSSL as static libraries (and linked to MUSL?). You then have to figure out how to change the corresponding Rust libraries to link to the static versions. I'm like 70% sure, but not completely. – Mcgowan

The thing is, I did rebuild them against MUSL, in the prefix /usr/local/musl. And I'm trying to get the corresponding libs to link to the static versions by trying to set the environment right, but apparently I'm failing. The way you put it sounds like I'd have to modify the libraries themselves, is that what you meant? – Despiteful

I mean, once you have compiled the MUSL versions of the C libraries (nicely done!), you will have to poke at each of the Rust bindings to figure out how to change how they link and to what. For example, openssl has environment variables. – Mcgowan

This was cross posted to Reddit. – Mcgowan

Thanks for the pointer about the environment variables! That was a crucial clue. – Despiteful

My friend, probably you are not looking into the right directory. Your result will be in target/x86_64-unknown-linux-musl/release/ – Selfabnegation

Answer 1

The problem was that for each crate providing a native dependency – say OpenSSL – there is the build.rs build script that is in charge of communicating the build and linking options to Cargo and to rustc. (For example: they print out something like cargo:rustc-link-lib=static=ssl which Cargo then reads and acts accordingly.)

So just setting the "standard" GCC environmental variables is hardly going to have any effect. You must check each and every build.rs separately to know how to coerce that exact crate to convey cargo its options. For OpenSSL, its env vars like OPENSSL_DIR, OPENSSL_STATIC etc.

Another hurdle is that if you use compiler plugins, they might be compiled with the target triplet too (at least docker_codegen). On the other hand, they are linked dynamically during the compiling process. This mean that not only must static libraries be linked correctly, you must also have dynamic libraries of the target host variety, like Musl libc.so in place, and correctly set (LD_LIBRARY_PATH etc.).

I made a thoroughly commented Dockerfile that builds my project statically with some native dependencies. It might be of help for others too.

https://gitlab.com/rust_musl_docker/image

Despiteful

Could you please repost this docker file, that link is dead – Lanham

Ah, sorry, the project moved over to GitLab. I'll update the link. – Despiteful

Answer 2

If you want to statically link a Rust program without native dependencies, that is much easier:

$ rustup target add x86_64-unknown-linux-musl
$ cargo build --release --target=x86_64-unknown-linux-musl

Hudis

this cannot find openssl and does not build anything – Galenism

@nikoss, Given you are on Linux, try enabling "vendored" feature of openssl crate and make sure you have musl-gcc installed. – Evaporation

This worked for me. In addition, I did need to install musl-tools. Here's a pull request that demonstrates static building on GitHub Action. – Judejudea

Answer 3

I had the same problem with ldd and GCC. The musl target was generated in a different directory; not in target/release/... but in target/x86_64-unknown-linux-musl/release/....

Selfabnegation