What's the best way to optimise the build of a project which uses Boost?

Question

I was a bit staggered today by the sheer number of auto generated includes that Boost produces when doing a compile if we turn on verbose includes. We're averaging 3000 header files included per compilation unit and sometimes getting up to 5000. Virtually all of it is caused by Boost's preprocessor-meta programming funk with large numbers of the same header file getting included again and again in a massive preprocessor recursion.

Do you think 3000 per compilation is normal for a Boost project? What can I do to optimise Boost builds other than buying an array of SSDs?

Answer 1

One thing that can really help is the use of precompiled headers, so that many or most of the Boost headers get compiled once for the whole build, not once for every translation unit.

Both Microsoft Visual C++ and GCC support precompiled headers (as do other compilers).

Answer 2

Precompiled headers are your friend. Basically the compiler parses those thousands of files once, dumps state to the precompiled header file, and then each .cpp you compile can just instantiate compiler state straight from that without parsing the files again.

Here's an example of a header file which just sucks in all the boost stuff (and more) used anywhere in the rest of the code. Once upon a time, including so many headers would have been a recipe for name clashes or worse (hello windows.h "max" macro I'm looking at you) but these days template abuse has replaced macro abuse, and most libraries (boost stuff especially) are sufficiently well namespaced that it's rarely a problem. In this just-include-it-all style of coding, worrying which particular namespaces a .cpp considers worthy of promotion to top level by "using namespace" directives kind of replaces worrying about which particular headers to include.

Answer 3

Apart from starting to use precompiled headers, also try dividing the project into smaller libraries. That way some of the Boost dependencies are pulled in only for the libraries that actually use them and most builds will be faster after changing just one library.

Answer 4

Since you're using boost, in case you didn't know about it (unfortunately many people don't), consider using bcp utility that comes with it. I got the impression that you're building whole boost every time you're building your project, that's probably not what you really want. Look at the parts of boost you're actually using and ask bcp to copy them for you.

So a use of bcp plus precompiled headers will likely boost (pun intended) your build times a lot.