What techniques can be used to speed up C++ compilation times?
This question came up in some comments on this question: http://stackoverflow.com/questions/372862/c-programming-style
And I'm interested to hear what ideas there are.
I've seen this related question, but that doesn't provide many solutions:
http://stackoverflow.com/questions/318398/why-does-c-compilation-take-so-long
I had an idea about using a ramdrive. Turned out that for my projects it doesn't make that much of a difference after all. But then they are pretty small still. Try it! I'd be interested in hearing how much it helped.
If you have a multicore processor, both Visual Studio (2005 and later) as well as gcc support multi processor compiles. Something to enable if you have the hardware, for sure.
I will just link to my other answer: How do YOU reduce compile time, and linking time for Visual C++ projects? (native c++) . Another point i want to add, but which causes often problems is to use precompiled headers. But please, only use them for parts which hardly change (like GUI toolkit headers). Otherwise, they will cost you more time than they save you in the end.
Another option is, when you work with GNU make, to turn on -j<N> option:
-j [N], --jobs[=N] Allow N jobs at once; infinite jobs with no arg.
I usually have it at 3 since i've got a dual core here. It will then run compilers in parallel for different translation units, provided there are no dependencies between them. Linking cannot be done in parallel, since there is only one linker process linking together all object files. But the linker itself can be threaded, and this is what the GNU gold ELF linker does. It's optimized threaded C++ code which is said to link ELF object files a magnitude faster than the old ld (and was actually included into binutils).
Here are some:
Upgrade your computer
Then you have all your other typical suggestions
Use
#pragma once
at the top of header files so if they're included more than once in a translation unit, the text of the header will only get included and parsed once.
When I came out of college. The first real production worthy c++ code I saw had these arcane #ifndef ... #endif directives in between them were the headers defined. I asked the guy who was writing the code about these overarching things in a very naive fashion and was introduced to world of large scale programming. Coming back to point, using directives to prevent duplicate header definition was the first thing I learned when it came to reducing compiling times.
Take a look at the pimpl idiom here, and here. Also known as an Opaque Pointer or Handle. Not only does it speed up compilation, it also increases exception safety when combined with a non-throwing swap function. The pimpl idiom lets you reduce the dependencies between headers, and reduces the amount of recompilation that needs to be done.
Wherever possible, use forward declarations. If the compiler only needs to know that SomeIdentifier is a struct or a pointer or whatever, don't include the entire definition, forcing the compiler to do more work than it needs to. This can have a cascading effect, making this way slower than they need to be.
The IO streams are particularly known for slowing down builds. If you need them in a header file, try #including <iosfwd> instead of <iostream> and #include the <iostream> header in the implementation file only. The <iosfwd> header holds forward declarations only. Unfortunately the other standard headers don't have a respective declarations header.
Prefer pass-by-reference to pass-by-value in function signatures. This will eliminate the need to #include the respective type definitions in the header file and you will only need to forward-declare the type. Of course, prefer const references to non-const references to avoid obscure bugs, but this is an issue for another question.
Use guard conditions to keep header files from being included more than once in a single translation unit.
#pragma once
#ifndef filename_h
#define filename_h
// Header declarations / definitions
#endif
By using both the pragma and the ifndef, you get the portability of the plain macro solution, as well as the compilation speed optimization that some compilers can do in the presence of the pragma once directive.
The more modular and less interdependant your code design is in general, the less often you will have to recompile everything. You can also end up reducing the amount of work the compiler has to do on any individual block of at the same time, by virtue of the fact that it has less to keep track of.
These are used to compile a common section of included headers once for many translation units. The compiler compiles it once, and saves its internal state. That state can then be loaded quickly to get a head start in compiling another file with that same set of headers.
Be careful that you only include rarely changed stuff in the precompiled headers, or you could end up doing full rebuilds more often than necessary. This is a good place for stl headers and other library include files.
ccache is another utility that takes advantage of caching techniques to speed things up.
Many compilers / IDEs support using multiple cores/CPUs to do compilation simultaneously. In GCC, use the -j [N] option. In Visual Studio, there's an option under preferences to allow it to build multiple projects in parallel. You can also use the /MP option for file-level paralellism, instead of just project-level paralellism.
Other parallel utilities:
The more the compiler tries to optimize, the harder it has to work.
Moving your less frequently modified code into libraries can reduce compile time. By using shared libraries (.so or .dll), you can reduce linking time as well.
More RAM, faster hard drives (including SSDs), more CPUs/cores will all make a difference in compilation speed.
Use forward declarations where you can. If a class declaration only uses a pointer or reference to a type, you can just forward delcare it and include the header for the type in the implementation file.
For example:
// T.h
class Class2; // forward declaration
class T {
public:
void doSomething(Class2 &c2);
private:
Class2 *m_Class2Ptr;
};
// T.cpp
#include "Class2.h"
void Class2::doSomething(Class2 &c2) {
// whatever you want here
}
less includes means far less work for the preprocessor if you do it enough.
I'd recommend these articles:
Granted, they are pretty old - you'll have to re-test everything with the latest versions (or versions available to you), to get realistic results. Either way, it is a good source for ideas.
Just for completeness: a build might be slow because the build system is being stupid as well as because the compiler is taking a long time to do its work.
Read Recursive Make Considered Harmful [PDF] for a discussion of this topic in unix environments.
Where are you spending your time? Are you CPU bound? Memory bound? Disk bound? Can you use more cores? More RAM? Do you need RAID? Do you simply want to improve the efficiency of your current system?
.
Under gcc/g++, have you looked at ccache? It can be helpful if you are doing make_clean_;_make a lot.
Once you have applied all the code tricks above (forward declarations, reducing header inclusion to the minimum in public headers, pushing most details inside the implementation file with PIMPL...) and nothing else can be gained language-wise, consider your build system. If you use linux consider using distcc (distributed compiler) and ccache (cache compiler).
The first one, distcc, executes the preprocessor step locally and then sends the output to the first available compiler in the network. Requires the same compiler and library versions in all the configured nodes in the network.
The latter, ccache, is a compiler cache. It again executes the preprocessor and then check with an internal database (held in a local directory) if that preprocessor file has already been compiled with the same compiler parameters. If it does, it just pops up the binary and output from the first run of the compiler.
Both can be used at the same time, so that if ccache does not have a local copy it can send it trough the net to another node with distcc, or else it can just inject the solution without further processing.
Dynamic linking (.so) can be much much faster than static linking (.a). Especially when you have a slow network drive. This is since you have all of the code in the .a file which needs to be processed and written out. In addition, a much larger executable file needs to be written out to the disk.
More RAM.
Someone talked about ramdrives above, I did this with a 286 and TurboC++ (shows age) and the results were phenomenal. As was the loss of data when the machine crashed.
There's an entire book on this topic, which is titled Large-Scale C++ Software Design (written by Lakos).
The book pre-dates templates, so to the contents of that book add "using templates, too, can make the compiler slower".
On linux (and maybe some other *NIXes), you can really speed the compilation by NOT STARING at the output and changing to ANOTHER TTY.
Here is the experiment: printf slows down my program
One technique which worked quite well for me in the past: don't compile multiple C++ source files independantly but rather generate one C++ file which includes all the other files, like this:
// myproject_all.cpp
// Automatically generated file - don't edit this by hand!
#include "main.cpp"
#include "mainwindow.cpp"
#include "filterdialog.cpp"
#include "database.cpp"
Of course this means you have to recompile all of the included source code in case any of the sources changes, so the dependency tree gets worse. However, compiling multiple source files as one translation unit is faster (at least in my experiments with MSVC and GCC) and generates smaller binaries. I also suspect that the compiler is given more potential for optimizations (since it can see more code at once).
This technique breaks in various cases; for instance, the compiler will bail out in case two or more source files declare a global function with the same name. I couldn't find this technique described in any of the other answers though, that's why I'm mentioning it here.
For what it's worth, the KDE Project used this exact same technique since 1999 to build optimized binaries (possibly for a release). The switch to the build configure script was called --enable-final. Out of archaeological interest I dug up the posting which announced this feature: http://lists.kde.org/?l=kde-devel&m=92722836009368&w=2