What should we do when the buildserver is treated like a goldmine?

Question

A year ago I started to create some automated builds on our build machine (TFS2008). Not so much for combining with full scale TDD (we still have a lot of old legacy code), but for being able to detect at an early stage if builds got broken. Another objective was also to minimize the packaging/deployment work.

This has been working quite well so far, but lately some coworkers are starting to treat the buildserver as a goldmine of quick releases, and the testing process seems to get less priority more often. Refactoring some of our code during 2-3 days proved that the builds on our buildserver potentially could reach our customers. :)

I guess our buildserver over time has shifted from being a 'consistency tool' for the developers, into being a server producing packages that is expected to be release quality 24/7.

This is clearly a communication problem, and there should be a set of rules on this. Only problem is that I don't know where to begin. Does anyone have similar experiences with this?

Answer 1

You're correct, it is a communications problem. If your developers and management are expecting release-quality builds all the time, they're not understanding the process of build/test/release.

The only thing you can do is clarify the purpose of a build server: a single, centralized location for builds. You need to clarify the distinction between a build and a release. Builds should always succeed (no one should break the build) but the ability to create a build does not have any bearing whatsoever on build quality or the suitability of a given build for release.

Build quality is measured by unit, functional, and user acceptance testing. There is no replacement for these tests in preparing a build for release. The long-term costs of not doing these tests far outweigh the short-term benefits of getting a release out the door.

Answer 2

Our unittestserver does tests, and tags CVS. Then we go on a buildserver which has ea script to create a release which isready for customer installation. This release is then installed on a test server as if it was the customer's server, and then tested.

Judging your story, you are hoping to find some script or setting which will prevent the buildserver from getting used as "quick release" server. The only real way to do this is process.

Rules in our company:

1. Developers check into CVS, they get mails from the unittest server if it fails, and have to fix that in code. No access to the build/test server for devs.
2. There is 1 specific developer who can create a release which he can send to the test department.
3. The test department installs the release on their test server and tests it.
4. The testers, and only the testers, can give a "Go" for release.
5. The release is done by a designated person who is also the customer contact.

As you can see the developers are seperated from the testers and the customer (formally speaking). In practice it is not all that rigid ofcourse, but people need to understand that if this process is not in place, the customer will get inferior quality software.

The customer has to be educated that "fast" means "low quality". We can do it Fast, Good, or Cheap. Pick two. http://www.sixside.com/fast%5Fgood%5Fcheap.asp

Answer 3

I suggest that all builds created by the internal build server state in the splash screen "INTERNAL BUILD - NOT FOR CUSTOMERS", and the release build server plops in the official splash screen.