How to decrease MSBuild times.

Question

my situation

In the C# project I am now working on we have a fairly big solution (80+ projects). Now rebuild times of 5 minutes+ are really becoming a quite problem using MSBuild from VS 2008.

In a analysis I did last week it turned out that my build time was spent as follows

1) Copying files to the projects and recopying it to the projects that depend on it (CopyToLocal) etc. (60%) 2) Invoking a postbuild to decompile/compile. (20%) 3) Doing the actual compilation etc (20%)

Apart from the 'normal' project bin\debug folders output is also copied to an external dir to setup the main 'loader' program. The main program structure is a bit like this

\loader\bin\loader.exe \loader\plugin\plugin1\plugin1.dll \loader\plugin\plugin1\somedependency.dll

what I did

In an attempt to make things go a little faster I thought of the following:

1 Copy all the files to one a big bin directory and don't use CopyTolocal. I don't like this because we can no longer use different versions of the same dlls and my bin directory is getting quite a mess.

2 Use parallelism (/m) for MSBuild. This helps only very little in build times. 3 Try to reduce dependencies between projects which is always a good thing offcourse.

4 Invest in hardware. I found some research on SolidState drives but this does not seem promising.

my question

I also noticed that when I make a change to a project that is at the root of my dependency tree everything gets rebuild. Even if the change was only in the 'private' part and the interface of the project did not change.

MSBuild uses a timestamp of dependend projects to determine if a project needs a rebuild??

Can this be changed to a different condition?? For example the checksum of the file?

Apart from this specific suggestion i would sure appreciate all suggestions to make build times faster.

Best regards, Martijn

Answer 1

We also have huge solutions. Build and compilation is all about I/O.

Solid state drives are very promising. A co-worker put a solid state drive in his laptop, and found that it is now much faster than his humongous main dev box. Don't have the details, but he claims many times faster.

We've been fiddling with solution folders to group parts of the project: This makes it easier for devs to unload projects they aren't working on.

/m rarely helps with .NET debug builds. I ran a series of tests on this a few weeks ago, and found minor differences. Findings:

• Because my build is IO constrained, using /m:2-4 makes debug builds slower for me.
• Release builds usually much faster.
• Code analysis adds a lot of time.

Big picture: In reality, compilation is a pretty minor cost for me, compared to getting source and running unit tests. On our build server, the integration tests and packaging are almost all the time. For my dev box, I schedule a batch file that gets source, builds, runs unit tests before I come to work and while I'm at lunch. Good enough.

On the build server, it's more complicated. We're thinking of setting up chained parallel CC.NET builds on various machines. We are using VSTS build, but it is too expensive (and time consuming) to scale horizontally like this.

My numbers for detail oriented folks. Configurations from slowest to fastest, running msbuild "bigsolution.sln" /target:clean - between each.

1. /m: 4, Debug with code analysis 1:38
2. /m: 1, Debug with code analysis 1:30
3. /m: 4, Debug with no code analysis 1:30
4. /m: 1, Debug with no code analysis 1:30
5. /m: 1, Release with code analysis: 1:30
6. /m: 4, Release with code analysis: 1:05
7. /m: 4, Release with no code analysis: 0:53

Build time without rebuild or clean: ~ 4-10 seconds

Answer 2

MSBuild build time is a multi-dimensional problem. The good news that it easy to solve:

Unlike most of the processes running on the build machine, build processes are notorious for being CPU, RAM and I/O -consuming. A general recipe for speeding up MSBuild builds is "get best machine money can buy", particularly:

1. CPU - at least two Intel 3.0Ghz Core 2 Duo.

2. RAM - at least 4GB DDR3. If this is a developer and build machine, a 64bit OS and 8GB RAM is a better option.

3. HDD - The fastest options is a high-end 3ware RAID-1 with an on-board battery and an enabled write cache. A fast SSD maybe another option to consider.

4. Network - minimum 1Gbit/s card.

This simple hardware optimization can speed up your MSBuilds 2-3 times.

Hope this helps.

Regards,

Slava Imeshev