Low performance of Incremental linking in Visual Studio C++

Question

I have a large binary which is built of many static libs and standalone cpp files. It is configured to use incremental linking, all optimizations are disabled by /Od - it is debug build.

I noticed that if I change any standalone cpp file then incremental linking runs fast - 1 min. But if I change any cpp in any static lib then it runs long - 10 min, the same time as ordinary linking. In this case I gain no benefit from incremental linking. Is it possible to speedup it? I use VS2005.

Immortalize

Do you change only the actual .lib or a header file, in that last case it's not really possible (but you could see some benefit from using pch). – Acatalectic

I am changing just one cpp file from .lib. – Immortalize

Check the .lib project for the /Yu and /Z7 options. – Waadt

@Hans I have only /Zi in options for generating pdb. I am not responsible for changing them. Is it absolutely necessary? – Immortalize

If you have to generate symbols than you have to leave /Zi option as it is. You have to check in the output window what happens differently in your "worst case scenario". Does that change imply beside library files compiling only linking of the modified library with the large binary ? Or are there some other dependencies triggered so that it ends up compiling much more ? – Agbogla

Did oyu explicitly check what builds are being performed when you change the CPP file? All targets which depend on that static library should be rebuilt. Are there any other targets which you believe shouldn't be rebuilt? – Vallation

I can't remember if there is an option in VS2005 but it is in later versions. If you have your "Whole program optimisation" set to "Link time generation" then rebuilding your lib might trigger a rebuild of your application in order to take advantage of any common expression optimisations. – Undertow

@Undertow I am building debug version - all optimizations are disabled by /Od option. – Immortalize

@Foo I have only one target which depend on that static library, it's the binary itself. The only targets which rebuilt after CPP file changed are static lib containing that CPP and the whole binary. – Immortalize

Answer 1

Set "Use Library Dependency Inputs" in the Linker General property page for your project. That will link the individual .obj files from the dependency .lib instead of the .lib, which may have some different side effects.

Sulphathiazole

I agree, this setting is very useful for speeeding up linking. – Confirmed

Yes - this answers the question directly. I used to use a product called Xoreax Incredibuild with Visual Studio 2003; it has a feature it dubbed "Incredilink" that did exactly this. It had a HUGE impact on reducing build times. It appears that functionality was pulled directly into Visual Studio 2005. – Denomination

@Sulphathiazole Thanks. My project type is MakeFile project. There is no Linker property page in this type of project. Can I set this property anyway? Or pass it to linker command line? Anyway +1 for the answer. – Immortalize

Though I can't check it right now, according to MSDN this answers my question. – Immortalize

@ks1322, if you are generating the link command line yourself, you will have to enumerate the .obj files to use yourself. That's what visual studio does behind the scenes (which is why it can fail pre VS 2010 for .lib projects with a ton of individual .obj files). – Sulphathiazole

Answer 2

I going to give you a different type of answer. Hardware.

What is your development environment? Is there anyway to get more RAM or to put your project onto an Solid State Drive? I found that using a SSD sped up my link times by an order of magnitude on my work projects. Helped a little for compile times, but the linking was huge. Getting a faster system of course also helped.

Fessler

+1 for hardware solution. SSD would speed it up. Almost -1 for other suggestions like RAM (how large should be a project that linking requires more than mainstream 2-4-6 GB of RAM?) or faster system (what system do you mean?) – Magdeburg

You'd be surprised on what kind of systems people attempt to code on @Andy. When I upgraded from an older Dual core 2Gig RAM WinXP box to a quad core 8gig Win7 box, everything got a bit faster. Not as big an increase as the SSD, but a difference worth mentioning. – Fessler

Answer 3

If I understand correctly (after using Visual Stuio for some years), the incremental linking feature does not work for object files that is part of static libraries.

One way to solve this is to restructure your solution so that your application project contains all source files.

Spay

Thanks. Could you provide links to MSDN or some other resource to confirm this? I want to be sure that incremental linking is impossible with object files from static libs. – Immortalize

Sorry, no, this is based on my own experience working with the tools. – Spay