Hi
What is you experience with the make -j flag?
There seem to be some controversial if the jobs are supposed to be equal to the numbers of cores, or if you can maximize the build by adding one extra job that can be cued up while the others "work".
The question is if it is better to use -j4 or -j5 on a quad core system?
And have you seen (or done) any benchmarking that support one or the other?
Thanks Johan
I would say the best thing to do is benchmark it yourself on your particular environment and workload. Seems like there are too many variables (size/number of source files, available memory, disk caching, whether your source directory & system headers are located on different disks, etc.) for a one-size-fits-all answer.
My personal experience (on a 2-core MacBook Pro) is that -j2 is significantly faster than -j1, but beyond that (-j3, -j4 etc.) there's no measurable speedup. So for my environment "jobs == number of cores" seems to be a good answer. (YMMV)
I personnaly use "make -j n" where n is "number of cores" + 1.
I can't however give a scientific explanation: i've seen a lot of people using the same settings and it gave me pretty good results so far.
Anyway, you have to be careful because some make-chains simply aren't compatible with --jobs option, and can lead to unexpected results. I'f you're experiencing strange dependency errors, just try to "make" without --jobs.
Ultimately, you'll have to do some benchmarks to determine the best number to use for your build, but remember that the CPU isn't the only resource that matters!
If you've got a build that relies heavily on the disk, for example, then spawning lots of jobs on a multicore system might actually be slower, as the disk will have to do extra work moving the disk head back and forth to serve all the different jobs (depending on lots of factors, like how well the OS handles the disk-cache, native command queuing support by the disk, etc.).
And then you've got "real" cores versus hyper-threading. You may or may not benefit from spawning jobs for each hyper-thread. Again, you'll have to benchmark to find out.
I can't say I've specifically tried #cores + 1, but on our systems (Intel i7 940, 4 hyperthreaded cores, lots of RAM, and VelociRaptor drives) and our build (large-scale C++ build that's alternately CPU and I/O bound) there is very little difference between -j4 and -j8. (It's maybe 15% better... but nowhere near twice as good.)
If I'm going away for lunch, I'll use -j8, but if I want to use my system for anything else while it's building, I'll use a lower number. :)