Is there any way to prevent Boost.Build from recursively scanning header files for #include directives?

Question

Is there a way to limit the header files that Boost.Build recursively scans for #include directives to a particular directory or set of directories? I.e. I'd like it to recursively scan the header files within my project only. I know that they external dependencies are not going to change (and being Boost and Qt they're pretty big). I end up with around 50,000 targets in the dependency tree which takes a while to process (resulting in a 1-2 minute build time even if no files have actually changed).

The only solution I've found so far is to take advantage of the INCLUDE environment variable (I'm using MSVC) - this means Boost.Build need not be informed of the include paths (I'm using the feature) and hence will not scan them. This seems a bit of a hack.

I feel like I must be missing something obvious because I haven't been able to find other people experiencing similar problems, even though I ran into this almost immediately. The closest I've come is here.

Judging from the debug output (bjam -d 3) it also scans most of the header files more than once... I don't know if this means that they are added as dependencies more than once, but certainly the cost of loading a file and scanning the entire contents must add up?

If I could tell it not to bother scanning a particular directory or set of directories in which I can guarantee the header files are not going to change, that would be perfect.

Answer 1

You might want to check out alternative build tools like SCons. SCons has a mode --implicit-cache where it caches implicit dependencies. That should help in the scenario you described.

Here's an extract from the man page.

--implicit-cache
Cache implicit dependencies. This causes scons to use the implicit (scanned) dependencies from the last time it was run instead of scanning the files for implicit dependencies. This can significantly speed up SCons, but with the following limitations: scons will not detect changes to implicit dependency search paths (e.g. CPPPATH, LIBPATH) that would ordinarily cause different versions of same-named files to be used. scons will miss changes in the implicit dependencies in cases where a new implicit dependency is added earlier in the implicit dependency search path (e.g. CPPPATH, LIBPATH) than a current implicit dependency with the same name.

--implicit-deps-changed
Forces SCons to ignore the cached implicit dependencies. This causes the implicit dependencies to be rescanned and recached. This implies --implicit-cache.

--implicit-deps-unchanged
Force SCons to ignore changes in the implicit dependencies. This causes cached implicit dependencies to always be used. This implies --implicit-cache.

Answer 2

This question was also posted on the Boost mailing list and we got an answer to it here: http://lists.boost.org/boost-build/2009/04/21734.php.

So it seems so far that the answer is that, at least out of the box, Boost.Build doesn't have this feature, and the solution is to customise Boost.Build to your needs, which makes a certain amount of sense.

However, I am still curious as to why this is not a more common problem for people. I see that caching the dependencies would reduce the time, but surely if we scan all external libraries we end up with a huge dependency tree, much of it redundant? When I'm working on a project, I'm not going to change the third party libraries very often at all, it seems a shame to pay for dependency checking on them.