What exactly is considered "breaking the build?"

Question

In a CI environment, what exactly is considered a broken build?

There are several answers I can imagine (any combination of compiles, tests pass, metrics are in range, documentation exists etc.) , but I am not sure which of these are cannoncial.

For example, just today it happened to me that I actually checked in all code changes but forgot to commit the Visual Studio project file, thus breaking the unit tests. (even though I literally triple checked my commit, as it's a public OSS project on google code).

I was easily able to fix this in under a minute after my first commit, but should I consider myself a buildbreaker now?

How do you configure your CI environment: Is every revision built or only the newest version after each complete build, or do you use time based checking for new revs?

Answer 1

for us, we use the term "breaking the build" any time the test suite fails after a new commit.

so, in your instance, yes you would have broken the build (according to our company at least)

Answer 2

Breaking the build is commiting any changes that make it impossible (or possible but not smart to do) to deploy the project.

Fixing a broken build does not repair the broken build, but makes it possible to create a new non-broken build.

I configure my CI-server to create a minimal build on every commit, and create a maximum build every period of time. The period depends on the number of people working on the project (more people is more commits) and the build duration (You can run your unit test suite every time, but the 30min. acceptance test suite once or twice a day).

Answer 3

Breaking a build is preventing any user that relies on the same standard tools and set of code than your CI environnment to get a compiling and running system.

If your coworkers cannot compile the system when they're up to date, because some config is missing, the build is broken, isn't it ?

If you coworkers cannot be confident that unit tests pass, because one of them is flaky, the build is broken, isn'it ?

If you have automated performance tests, and you're project have to be optimized, I would go as far as too say that if you code doesn't run fast enough, you've broken the build (but that is arguable)

I would not be so strongly minded about code coverage, or other metrics for example.

Breaking the build can happen. CI is just there to make sure it doesn't happen too much on the day you're supposed to ship ;)

Answer 4

A broken build is anything that doesn't pass the automated test suite.

You're a build breaker. ;) That's not really a big deal as long as you fix it fast. The whole point of a CI environment is to catch bugs, not make people afraid to commit.

My company builds the tip of each branch that is either in production or next up for production. We do this on every commit and each day at 4am. We're using Mercurial, so commit here means pushing changes to the integration repo.

Answer 5

Ideally, you have

1. automated script that is scheduled to run each night to build the application from source code.

2. Scripts to copy the binaries to a directory/set of directories from which another script can be run to deploy the application if it is running in your environment, or used to create deliverables for a customer.

3. Automated suite of tests that run and verify all components pass all tests.

4. Automated script that verifies that the build has been built correctly.

5. Automated script/monitoring system that sends out an alert if the verification script fails.

When an alert is generated by the above process, then that is considered "breaking the build".

But since procedures/processes can vary from company to company, there may be alternate definitions. In some places it may be breaking the unit tests. Others it may be checking in source code that results in the code being unabl to compile.

Answer 6

As long as you broke the build because of a simple human error (like forgetting to commit something) and as long as this is an exception and not the rule, I would say it's OK. As long as you take care to fix this quickly :)

On the other hand, if somebody breaks builds on a regular basis by not executing the complete build locally on his/her machine before committing, then this is a sign of an ill-disciplined team member who doesn't really care for other team members and for the development process.

My experience is that one effective way to make people be more careful is to set up your CI server to send an email when (and only when) the status of the build changes, with the "culprit" on the To: list and the rest of the team on the Cc:. I guess you can could call it a "shame factor" ;)