Do you know any build systems with decent support for parallelization?

Question

Hi,

I am looking for a build system (working on ms windows) that has good support for parallelization of tasks/targets (or whatever you call them). To be more specific - during build (that is initiated on MS Windows machine) I need to copy source files to a number of different machines (which are not necessarily running Windows) and start a remote job on each of them - and I really like to do that on all machines at once. Does anyone know a build system that's capable of executing such a task in parallel.

From what I googled, the options currently available are:
-j switch in make - but i don't know if nmake supports this
-some custom nAnt tasks
-msbuild has some form of support for parallelization - seems similiar to make (meaning you don't specify what to do in parallel, just specify that it would be nice to build things that way)
-fake (f# make) is written in functional programming language which are known to have good parallelization support - but I'm not very skillful in functional programming area.
-TeamCity has great support for such things but this is a little to 'heavy tool' (and too expensive too)

Any other solutions I could explore?

Answer 1

MSBuild supports parallelization. http://www.hanselman.com/blog/FasterBuildsWithMSBuildUsingParallelBuildsAndMulticoreCPUs.aspx
http://blogs.msdn.com/b/msbuild/archive/2007/04/26/building-projects-in-parallel.aspx

Ant has the concept of parallel tasks.
http://ant.apache.org/manual/CoreTasks/parallel.html

Of course Make has it also.

Look at this post also...
http://stackoverflow.com/questions/601970/how-do-i-utilise-all-the-cores-for-nmake

I guess it really depends on what your definition of decent is.

Answer 2

CruiseControl.NET supports parallel builds, but what you need to be aware of is that parallel builds ONLY work WELL in situations where the projects being built aren't highly dependent on shared resources such as files and other projects. The build process generally locks files which other parallel builds maybe trying to delete. This can cause your builds to fail for "non-code" reasons.

Answer 3

I expect that you can do what you want with SCons, but if you don't know Python it may not be worth investing your effort in. If you do, it's another matter.