My suggestion is not to worry about how many files the compiler is including. Focus more on correctness, robustness, and schedule. If build time is a concern, get a faster machine, build overnight, go on a walk, or divide the code into smaller translation units. Translation units should be small enough to contain code that doesn't change often. Changes are EVIL.
The foundation of the build system is to compile only the few files that have changed. If your development process is functioning correctly, the build time will reside more and more in the linking phase as the project grows.
If the compile time is still lengthy, see if your compiler supports precompiled headers. Generally, this is method for the compiler to store all the declarations and definitions in a more efficient form.