How to link with the GNU gold linker instead of ld in Haskell

Question

My Haskell project spends lots of time in Linking dist/build/myapp/myapp ... and also in loading shared libraries when executing TemplateHaskell code.

I suspect this is because ld is slow.

How can I improve link times by switching to the gold linker?

Laboured

Besides switching linker you could use -dynamic flag. It could easily speedup linking tenfold. – Vowelize

@Vowelize Is this actually the case? I have tried in the past to use dynamic linking and it did not make things faster for my entire cabal project. But I may have done something wrong so that it used -dynamic-too, giving me both static and dynamic libraries. It would be great to have a minimal example project that shows if it really makes it faster. – Laboured

you should use only -dynamic GHC option. -dynamic-too links both statically (slow, large executables) and dynamically (smaller executables, faster link times), so you don't get any speedup. – Vowelize

@Vowelize I've tried dynamically linked executables now, they are not a good choice for me, because the startup time of my executable (just for the --help text) increased from 160 ms (static) to 4.5 seconds due to run-time linking. I will stick with static linking for now. – Laboured

That's quite strange (both 0.16s and 4.5s for app start). What platform you're on? And do you have enough free RAM? Perhaps long linking you have is a cache issue. And interesting what was linking speedup on your projects when you tried dynamic linking? – Vowelize

@Vowelize On Linux. I have about 30 GB free RAM. It's not a cache issue, it happens on subsequent starts; the time it takes is 100% user CPU time spent in the ld-linux dynamic linker/loader. This page agrees with the phenomenon: The runtime performance costs of dynamic linking are substantial compared to those of static linking. Note I have 100s of entries in lld. Regarding speedup, link time seems to have gone down from 2.5 s per executable to 1.5, but that's not worth the increased startup time for me. – Laboured

I found this useful comment that suggests that dynamic linking speed can be drastically improved by setting -fvisibility=hidden and manually exporting all exported symbols. Maybe the dynamic loading startup time could be improved by using that. – Laboured

Answer 1

Link 3x faster with gold

Since GHC 7.8, you can tell GHC and cabal (at run time without having to recompile GHC) to link with GNU gold.

You need in your .cabal file:

library:
  ghc-options: -optl-fuse-ld=gold
  ld-options:  -fuse-ld=gold

executable myExecutable
  ghc-options: -optl-fuse-ld=gold
  ld-options:  -fuse-ld=gold

(Note you might want to pass these flags to stack/cabal/Setup.hs on the command line instead of hardcoding them in the .cabal file in order to not reduce the portability of the package.)

For me it's 3.5x faster, bringing down the total link time of a project from 150 seconds to 40 seconds.

---

Update: Link 10x faster with lld

See https://github.com/nh2/link-with-lld-example for a full example; key parts:

library
  ghc-options: "-pgmP clang" "-pgmc clang" "-pgma clang" "-pgml clang" "-optl-fuse-ld=lld"
  ld-options:  -fuse-ld=lld

executable myExecutable
  ghc-options: "-pgmP clang" "-pgmc clang" "-pgma clang" "-pgml clang"
  ld-options:  -fuse-ld=lld

Comparison of link time for the final executable link times my project:

ld   124 seconds
gold  36 seconds
lld   11 seconds

Laboured

How big is your project, if you don't mind me asking? – Shaunna

@Shaunna Not very big, 150 Haskell modules and 8 executables built from them. But it depends on (and links) a couple native libraries like opencv. With ld, each library link takes almost 20 seconds. – Laboured

Confirming build speed up by reducing time by 40 seconds on 489 modules and 13 executables project (tests and benchmakrs included). – Beckibeckie

@SebastianGraf It does not work out of the box with lld, at least not with the options I provided, because (at least on my system) ghc links by calling gcc, and gcc is not (yet) compatible with lld. You would have to tell ghc to use the clang toolchain instead of gcc; I haven't tried that yet. Would be very interested to know what the flags are for that though. – Laboured

Wow, that's awesome! Let's start a GHC proposal to use lld by default! – Cassicassia

Will it work without changes to the .cabal if you simply overwrite the /usr/bin/ld symlink to ldd? – Furman

@Furman It will not necessarily work; lld doesn't accept all flags that ld accepts, and officially gcc (which GHC invokes by default for linking) doesn't support linking with lld. (Also you mean lld, not ldd -- just saying because this typo can actually get confusing, as both are linking related programs). – Laboured

Thanks for the clarification @Laboured – and yes, I've already confused myself with the typo many times :) – Furman

@Laboured it looks like -fuse-ld=lld is supported now (at lease as of ghc 10; just tried it. gcc.gnu.org/bugzilla/show_bug.cgi?id=83243). I wonder how that changes things? Can those ghc-options go away now? – Shaunna

@Shaunna I think you mean gcc 10 instead of ghc 10? I will need to check what can be removed now, but haven't had time yet. Also there's already a new, even faster linker available now: mold. – Laboured