How to improve link performance for a large C++ application in VS2005

Question

We have fairly large C++ application which is composed of about 60 projects in Visual Studio 2005. It currently takes 7 minutes to link in Release mode and I would like to try to reduce the time. Are there any tips for improving the link time?

Most of the projects compile to static libraries, this makes testing easier since each one also has a set of associated unit tests. It seems the use of static libraries prevents VS2005 from using incremental linking, so even with incremental linking turned on it does a full link every time.

Would using DLLs for the sub projects make any difference? I don't really want to go through all the headers and add macros to export the symbols (even using a script) but if it would do something to reduce the 7 minute link time I will certainly consider it.

For some reason using nmake from the command line is slightly faster and linking the same application on Linux (with GCC) is much faster.

• Visual Studio IDE 7 minutes
• Visual C++ using nmake from the command line - 5 minutes
• GCC on Linux 34 seconds

Answer 1

60 libs to link does sound like a fair few. This may be a bit of an extreme measure, but it might radically speed things up. Create a new solution, with a few projects, and add all the source from your existing projects to these. Then build and link them instead, and just keep the small ones for testing.

Answer 2

Get a quicker computer with multiple processors and enable parallel builds (this might be on by default). To allow the greatest amount of parallism, make sure your project dependencies are correct and you haven't got unnecessary dependencies.

Answer 3

I don't think converting to DLLs would be useful. You could try looking for options to do with optimisation, and turning them off. The linker might be spending a long time looking over the libs for redundant code it can eliminate. Your app may end up bigger or slower, but that may not be a problem to you.

Answer 4

Generally, using DLLs instead of static libraries will improve linking times quite a bit.

Answer 5

Take a look at Incredibuild by Xoreax. Its distributed compilation dramatically reduced our full build/link times from around 40 minutes to 8 minutes.

Additionally, this product has a feature they call Incredilink which should help you get incremental links working even with statically linked libraries.

Answer 6

Several people have reported (and I myself have noticed) that modifying a file in a statically linked library will disable incremental linking for the entire solution; this appears to be what you are seeing. See comments here and here for some information about that.

One workaround is to use the Fast Solution Build Add-In. This might involve making a few changes to your workspace, but the payoff is definitely worth it. For a commercial solution, use Xoreax's Incredibuild, which basically incorporates this same technology but adds other features as well. I apologize if I sound like a salesman for Incredibuild - I'm just a very satisfied customer.

Answer 7

If you are truly talking about link times, then things like fast solution build and Xoreax won't really help much (except for Incredilink, which might). Assuming that you are truly measuring link start to link end, then I would suggest that the number of libs that you have is the issue.

The link phase is, at least initially, IO bound in loading up all of the object and lib files. You might be in a situation where you have 60 libraries along with the main project of some large number of .obj files. I suspect that you simply might be seeing, at least in part, typical windows slowness in loading up all of those libs and .obj files.

You can easily test this. Take all of those lib files and build one single lib file just as a test. Instead of linking with 60 of them, link with one and see where your time goes. That would be interesting.

NTFS is notoriosly slow. It shoudln't be 7m vs. 32 seconds on Linux slow, but it might be part of the issue. Using DLL's will help but you will suffer application startup time, although that will not be early as bad. I would be confident that you won't have 7m application start up times.

Answer 8

I've had similar troubles linking large apps with Visual C++ before. In my case, I simply didn't have enough free RAM and excessive paging to disk was slowing the linking process to a halt. Doubling my RAM from 1GB to 2GB made a dramatic improvement. How much is your dev box running?

Answer 9

you can try looking at this: http://msdn.microsoft.com/en-us/library/9h3z1a69.aspx

Basically, you can run project builds in parallel if you have several cores.

Answer 10

If you're using the /GL flag to enable Whole Program Optimization (WPO) or the /LTCG flag to enable Link Time Code Generation, turning them off will improve link times significantly, at the expense of some optimizations.

Also, if you're using the /Z7 flag to put debug symbols in the .obj files, your static libraries are probably huge. Using /Zi to create separate .pdb files might help if it prevents the linker from reading all of the debug symbols from disk. I'm not sure if it actually does help because I have not benchmarked it.

Answer 11

I just found out that we had by accident defined a large table of strings in a header file which got included in pretty much every (static) lib. (I am talking about a huge C++ project.) When the linker created the EXE, it looks like the unification of the table (there is only a single one ending up in the EXE) or the parsing of the libs took forever. Putting the table in a separate C++ file took a couple of minutes of the link on a relatively slow machine.

Unfortunately, I don't know how to find stuff like that other than by chance.

Answer 12

For debug builds, then one can use incremental linking, which can improve link times a lot.

Sadly enough there are certain pitfalls, and VS2005 will not warn you.

• If using static libraries then incremental linking will not work if modifying a file part for the static library. The solution is to set the linker option "Use Library Dependency Inputs" to "Yes" (This is the same as Fast Solution Build in VS2003)

• If using pragma-comment-lib to include the lib of a DLL, and specifies a relative path instead of the lib alone, then incremental linking will stop working. The solution is to specify the lib alone, and use the linker-option LIBPATH to add additional lib-path.

• Some times the .ilk file will become corrupted (grow beyond 200 MByte) and then suddenly the incremental linker til take more than 10 times the normal time. Some times it will complain about the .ilk file being corrupt, but usually first after several minutes. The solution for me was to setup the following command for the "Build Event" -> "Pre-Link Event"

  for %%f in ($(IntDir)*.ilk) do ( if "%%~zf" GTR "200000000" (del %%f))

Answer 13

See my suggestion made at Microsoft : https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=511300

You should vote for it ! Here is my last comment on it :

Yes we are using incremental linking to build most of our projects. For the biggest projects, it's useless. In fact, it takes more time to link those projects with incremental linking (2min50 compared to 2min44). We observed that it doesn't work when size of ILK files is big (our biggest project generate an ilk of 262144 KB in win 32).

Bellow, I list others things we tried to reduce link time:

• Explicit template instantiation to reduce code bloat. Small gain.
• IncrediLink (IncrediBuild give interesting gain for compilation but almost no gain for link).
• Remove debug information for libraries who are rarely debugged (good gain).
• Delete PDB file in « Pre-Build Event » (strangely it give interesting gain ex: 2min44 instead of 3min34).
• Convert many statics library to DLL. Important gain.
• Working with computer equiped with lot of RAM in order to maximize disk cache. The biggest gain.
• Big obj versus small obj. No difference.
• Change project options (/Ob1, /INCREMENTAL, Enable COMDAT folding, Embedding manifest, etc.). Some give interesting gain other not. We try to continuously maximize our settings.
• Maximize Internal linkage vs External linkage. It's a good programming practice.
• Separate software component as much as we can afford. You can than work in unit test that link fast. But we still have to interate things together, we have legacy code and we worked with third party component.
• Use secret linker switch /expectedoutputsize:120000000. Small gain.

Note that for all our experimentation, we meticulously measured link time. Slow link time seriously cost in productivity. When you implement complex algorithm or track difficult bug, you want to iterate rapidly this sequence : modify some code, link, trace debug, modify some code, link, etc...

Another point to optimize link time is the impact it have on our continuous integration cycle. We have many applications that shared common code and we are running continuous integration on it. Link time of all our applications took half the cycle time (15 minutes)...

In thread h-t-t_p_:/_/b-logs.msdn.com/vcblog/archive/2009/09/10/linker-throughput_dot_aspx, some interesting suggestions were made to improve link time. On a 64 bits computer, why not offering an option to work with file completely in RAM ?

Again, any suggestions that may help us reduce link time is welcome.