Can gcc use multiple cores when linking?

Question

So when compiling tons of source files with GCC one can use -j to use all available cores. But what about the linker? Is there a similar option to speed up linking or does GCC not support multi-threading? In some larger projects it can really take a while ... (... and I hate to wait!)

Edit: Thanks for pointing out that -j is a option for make and not gcc/g++. But this does not answer my question! I would like to know if gcc can use multi threading while linking a program!

Kimon

You might be interested in distcc distcc.org which will allow you to distribute compiles over several machines in a network. – Misconstrue

@Jon: I am not interested in parallel compilation but in parallel linking! – Kimon

Why did this question get downvoted? God knows gnu linker is dog slow and finding some way to make it link faster would only improve the build cycle. – Weingartner

Linking is not an obviously parallel task. Note that you can sometimes reduce the work of the linker using visibility attributes and/or the -fvisibility gcc option. – Emie

Not about gnu but finally Apple did a lot to speedup their linker and even developed a new library format in 2023. Remebmer that Apple is not using llvm linker but their own – Pattypatulous

Answer 1

Try gold, which was developed by Ian Lance Taylor et al. from Google and released as part of GNU binutils package.

From Wikipedia:

The motivation for writing gold was to make a linker that is faster than the GNU linker, especially for large applications coded in C++

I must admit I haven't tried it myself yet but it's mentioned on WebKitGTK project webpage.

For more information see an article written by the author of gold: A New ELF Linker.

More importantly, see the work on incremental / parallel / concurrent linking by Sander Mathijs van Veen titled Concurrent Linking with the GNU Gold Linker and the bibliography therein.

Eveleen

It must be stressed that gold does not use multithreading unless you pass gcc -Wl,--threads -Wl,--thread-count,$(nproc). – Ossa

Answer 2

lld, the linker developed by the LLVM project, will use multiple cores by default. I have also found it to be about 2x faster than gold running with multiple threads (-Wl,--threads -Wl,--thread-count,xxx)

Sixteenth

I reproduced the LLD 2x faster than gold with a precise benchmark described at https://mcmap.net/q/16938/-can-gcc-use-multiple-cores-when-linking – Ossa

Answer 3

In 2022 the mold linker is another great option.

It is the fastest linker available for ELF targets and has the most aggressive parallelism.

Sixteenth

Answer 4

The -j option you are referring to is handled by make not gcc.

Using make -j n asks make to run the actions in the Makefile with multiple parallel process (Replace n with a number. In the case of make -j 2 it's 2 process).

Make will handle most synchronization tasks well when doing parallel builds.

Tailstock

Answer 5

At Replacing ld with gold - any experience? I have benchmarked LD vs gold vs LLVM LLD on a minimal synthetic benchmark, and the time results clearly show that both gold and LLVM LLD are capable of parallel linking according to time, e.g.:

nogold:  wall=4.35s user=3.45s system=0.88s 876820kB
gold:    wall=1.35s user=1.72s system=0.46s 739760kB
lld:     wall=0.73s user=1.20s system=0.24s 625208kB

We know that parallel link was used because the wall time was smaller than the user time in both of those CPUs

Those benchmarks also reproduce Martin Richtarsky's 2x faster LLVM LLD link times.

Ossa

Answer 6

Here are two options for those on CMake (tested on 3.16.3):

• Switch to gold linker, enable multi-threading (set to number of system cores):

find_program(GNU_GOLD_PROGRAM gold)
if (GNU_GOLD_PROGRAM)
    include(ProcessorCount)
    ProcessorCount(HOST_PROC_COUNT)
    add_link_options("-fuse-ld=gold;LINKER:--threads,--thread-count=${HOST_PROC_COUNT}")
endif(GNU_GOLD_PROGRAM)

• Switch to lld linker (works in multi-threading mode by default):

find_program(LLD_PROGRAM lld)
if(LLD_PROGRAM)
    add_link_options("-fuse-ld=lld")
endif(LLD_PROGRAM)

More complete example: here

Note: if you receive the error message "ld.lld: error: unable to find library ..." you may need to add the list of library directories in a link_directories call like so:

find_program(LLD_PROGRAM lld)
if(LLD_PROGRAM)
    add_link_options("-fuse-ld=lld")
    link_directories(/usr/lib64 /usr/local/lib64)
endif(LLD_PROGRAM)

Husky