Comparison of CI Servers?

Question

I am searching for a comparison of different continuous integration (CI) Servers (esp. focusing on .NET) and couldn't find any.

Therefore I'd like to know what you think about the different solutions available, what are the pros and cons, what are the hosting requirements and why CI Server XY is the Server of your choice.

I am interested in your thoughts on (feel free to comment on others to):

• Hudson
• CruiseControl
• CruiseControl.NET
• Teamcity
• CIFactory (uses CruiseControl.NET)

Points of Interest are:

• Configuration (easy, flexible)
• Integration with SCM (esp. DSVC like git or hg)
• Integration with build sytems (MSBuild, NAnt, Rake)
• Integration with Testing Frameworks
• Integration with Source Anaylsis (Simian, NDepend, FxCop, NCover etc.)
• WebInterface/Dashboards
• Infrastructure requirements

Answer 1

TeamCity has a wonderful feature of allowing the developer to perform a personal build before committing. Very useful!

CruiseControl.NET is the granddaddy of the bunch and is hence a little dated visually etc. As it has been around for a while, Google knows how to fix many issues you will come up against.

For these reasons (amongst others), I use CruiseControl.NET at work and TeamCity at home and in my open source life :)

Answer 2

CruiseControl.NET - it can be a bit of a pain to set up (as can most CI systems), but it is worth perservering with. I currently have it set up to run unit tests on completion of builds, and to produce Wix installers on-demand. As Dan said, it looks a bit dated, but that doesn't matter, as it provides you with plenty of information that's easy to get at and easy to read.

One thing - make sure all your developers have CC Tray installed, running and pointing to their builds. It's a great feeling to get "Another successful build" in your notification tray.

Answer 3

Nothing helpful in:

• Anything better than CruiseControl for .NET CI?
• Continuous Integration Servers
• Hudson or Teamcity for continuous integration?
• What is your experience with CI server git support (Windows)?
• Language-agnostic automated build and test server for multiple projects
• Choosing the correct CI Server for my needs.
• Cruise versus TeamCity versus CruiseControl.NET
• CruiseControl [.NET] vs TeamCity for continuous integration?
• etc, etc, etc

EDIT: A pointed out by Jonik in a comment, I missed What is the difference between Hudson and CruiseControl for Java projects? and How and why do I set up a C# build machine?. You'll find very insightful answers. In other words, I think that everything you're looking for is already on Stack Overflow.

Answer 4

I'm fairly new to the CI-scene and I've been concentrating my efforts on CruiseControl.NET, using NAnt and Ivy to build my .NET projects.

I've found that CruiseControl.NET is very adaptable to lots of other tools, such as NCover/NUnit/etc. They all plug into this and integrate the results for a combined build process.

I'll be looking into TeamCity in the near future for my own interest, but I think that CruiseControl does a good job, but only as good as your build-scripts! If these are pants, then your builds can only be expected to be that good.

But in summary, CruiseControl.NET is a good solution, but I'm yet to find out how good the competition is in comparison.

Answer 5

We're using ccnet at work, which is fine for most of our needs (we have about 50 automated builds), but it needs one person for full time tweaking and fixing.

If you're starting from scratch, please take a look at Bamboo. We have looked into it and it looks really promising, but it doesn't completely match our needs and we have invested way too much time in ccnet to switch to Bamboo now.

Regards,

Sebastiaan

Answer 6

We use Hudson at work. The main reason is, that it is very easy to setup. You can directly execute the war (it's a executable jar) or deploy it at any servlet-container. And you're ready to start. Also Hudson supports many tools and is extensible through it's plugin-system.

Answer 7

I'm a CruiseControl.NET user all the way. My teams uses it at work and I use it at home for personal projects.

In particular, CruiseControl.NET allows me to run through the whole CI process: builds, version update, unit and integration tests, archival of source or release candidate, code coverage, even deployment to our test system at work. It's highly customizable, works well with MSBuild and NAnt, and even had an extensible plug-in architecture.

It pretty much does everything I need.

The biggest disadvantage: configuration is sometimes a pain, and can take time. But once it's done, it's done, and as another poster said, I love seeing the "successful build" signal because I know that not only did the build itself work, but that also that my unit and integration tests all ran successfully.

Answer 8

Team Foundation Build is an option as well as it interacts very well with Team Foundation Server. It's free as long as long you've licensed TFS.

Answer 9

I inherited a luntbuild server. Not a good option for a .NET project. If you find that you're constantly falling back to using the build server to run generic command line tasks, then something is wrong. A good build server had a good understanding of unit tests output and msbuild tasks as more than opaque commands to be run when the source control system changes.

I'm enjoying migrating to Team City.

Answer 10

We switched from CruiseControl.NET to TeamCity primarily because of ease of configuration. TeamCity also has more features, but the main reason was that a nice Web UI is simpler to use than XML configuration files.

EDIT: Most tasks TeamCity will do out of the box; when necessary we use NAnt.

Answer 11

We're happy with Hudson. I don't have anything to compare it to, but it was simple to configure and get running. Right now it only builds Win32 C++ projects and an installer, but we're porting to Linux and it should work with that as well.

Gets Subversion repositories without any problems and mails out alerts, etc. We like it so far. Again, we have limited experience with comparisons.

Answer 12

No question like this is complete without a link to the big CI Feature Matrix which lists just about every CI option out there.

But I think it is important to look ahead to the scope of what you want to include in your CI system. Is it going to be just builds or are you going to be bringing in other elements like static analysis, cross-project dependencies, deployments, functional tests, etc. To help with that planning I created this wallchart on the Elements of Enterprise CI (PDF; no registration required). Please don't let the "E-word" put you off; I just mean stuff beyond the basic fast feedback CI build. :)

It isn't tool specific but lists a variety of practices you might consider while you're in the planning/evaluation stages.