MSBuild isnt making the build completely parallel

Question

I have an MSBuild project where within it I have a task that calls multiple projects where I set BuildInParallel = "true"

Example:

  <Message Text="MSBuild project list = @(ProjList)" />
  <!-- Compile in parallel -->
  <MSBuild Projects="@(ProjList)" 
           Targets="Build" 
           Properties="Configuration=$(Configuration)" 
           BuildInParallel="true" />

These sub-projects actually invoke a command-line tool to do the actual 'building' - call it compile.exe. Doing crude profiling (thank you taskmgr.exe) of the build process has the following results:

based on the /m setting - I see that exact number of MSBuild.exe processes started which is expected of course - the total available concurrent build processes.

However what I expect to see is around that many number of processes of compile.exe. Basically each MSBuild process will just turn around and invoke compile.exe. What I see is that a number of compile.exe's are started, then they slowly finish until I just see one sole compile.exe still around. The tasks that each compile.exe take a different amount of time, so it's expected that one of them takes a lot longer than the others.

However no other compile.exe's are spawned until the first 'batch' of them are finished. In other words if I have /m:4 - I will see 4 compile.exe's until all finish, then another 4 will be spawned.

This isn't exactly completely parallel to me. Has anyone else seen this behavior. Am I just misunderstanding something?

Answer 1

Have a look at the newer posts. Have a look at the Hashimi MSPress book - highly recommended.

If you're using TeamBuild, MaxProcesses needs to be tweaked in the build service config file. [But you would have said.]

If you're running msbuild yourself, you need to be invoking msbuild with /m to make anything happen.

Do the files in @(ProjList) all have ToolsVersion = 3.5 in them?