Continuous Integration Build Configurations

Question

I have been charged with setting up a CI server in my company and I am looking for some advise on what build configs i need for my projects. As a first step i have set up the builds as:

Commit Build: Compiles code and runs unit tests
Integration Build: Compiles code and runs long running integration tests

I am unsure what else i would need to complete the CI picture. For example What build configs do you have in your shop?

I know there must be a step for deploying my successful builds, but would l make the deployment part of Integration?

Using TeamCity, MSBsuild and SVN

Looking for much needed advice.

Answer 1

Things that we run in a previous project on each CI run where Code Coverage Records, publish the automated generated documentation and Checkstyle reports.

This gave us some stats about the statistical quality of each checkin for planing and improving our work habits.

Answer 2

We have build configurations for

• compile + unit tests + static analysis (findbugs in our case) + code coverage (triggered on commit)
• integration tests (triggered on a schedule as long as there is a commit)
• deployment to test (manually triggered)

The deployment configuration allows a non-technical QA resource to deploy to the test environment whenever they're ready to test something, and avoids the confusion of whether a bug fix has hit the test environment yet.

Answer 3

The most complete build I ever saw did the following things in the given order. There are two groups, all targets in each group are executed regardless of failure, but the group fails if a member of the group fails. So we see ALL problems.

First group working on sources:

• clean work directory
• update to newest sources, get everything from SVN
• compile sources, rmic etc.
• valudate XML resources (as at least in Java there were a lot, like deployment descriptors, stylesheets etc.)
• do static code analysis available for sources, e.g. checking whitespace, coding conventions namings, file names, or more sophisticated checks done on the AST of the source (like PMD does it for Java).
• check naming conventions of other files, e.g. we checked the names of all dependant libraries to contain a version number.

Second group is working on the produced code, only if the first step succeeded:

• run the unit tests
• run fast integration tests
• do static code analysis available for sources (most tools for Java do it this way), e.g. checking for typical bug patterns (like Findbugs does it for Java)
• do reference checking, e.g. enforce layering of architecture, allowance of usage of certain classes from other classes etc.
• create the full deployment package

This is the main build that is triggered time after time for commits. It did a lot, but with some powerfull machine using several cores it was around 4 mins for 500k LOC. Testers can get the newest snapshot builds if they like to.

Long running integration tests (2 hours each) would run once per night and only do

• compile
• run long running tests

Another build was a purely documantion build, triggered once per night. It would never fail.

• create API documentation
• do a full static code analysis with all rules and produce some kind of indicator for the overall project quality
• produce coverage reports (it's a pity we did not enforce coverage to grow) from all projects build before
• produce all kind of fancy documentation like Maven site or Sonar report etc. This is for management/QA most of the time.

Answer 4

We had a similar conversation at the most recent CITCON North America (Continuous Integration and Testing conference) where we all shared our experiences and tried to put together a road map from simple CI to very built out CI and release systems.

The original conference notes are here. Along with a Flickr photostream. A cleaned up version is available at the urbancode blog as well.

The Aussies revisited the topic at CITCON Brisbane and a pencast of that is available

Hope some of those resources are useful.

Answer 5

I am working on this at the moment. Our build configuration does the following:

Build:

• Compile.
• Test.
• Merge different environment specific configuration values with a base config (this leaves us with Staging.config, Test.config etc)
• Create a file called VERSION.txt that lists build times, revision numbers etc.
• Publish all of this to a clean directory. That is then picked up as a build artifact by teamcity.

Now we have an application that can be published to any server simply by copying it to the deployment directory, and renaming the appropriate config file to web.config

We then have 3 more configurations for deployment. The first gets deployed to a development environment after every successful build. This gives us a working version of the latest codebase at all times. The second gets deployed to staging manually. This is set to deploy from the last pinned development build. Finally there is a live deployment configuration then deploys from the last deployed staging build. This does a number of extra things:

• Tag the released version
• Create an archive of it an place it in a directory for save keeping
• Go over all checkin comments since the last live build and extract ones with ticket numbers. Then use the titles of the tickets to generate a preliminary change list. This is edited by the PM before being saved for posterity.