Continuous Integration - How To Get Developers Bought-In To The Idea

Question

I'm a freelance developer and have recently set up source control and automated builds (CI and nightly) for a software development team at a client which did not previously use these techniques.

Whilst they are now using the source control (albeit somewhat begrudgingly), I just don't seem able to get them bought into the concept of continuous integration. The builds are broken most of the time, and the developers express surprise when it comes time to release and they can't integrate their work.

Short of extending the build process such that failures deliver mild electric shocks to their mice and keyboards, how can I persuade the team of the benefits of keeping the builds succeeding?

Update - incidentally, the builds aren't very complex - just compiling the combined code. I had hoped to include unit testing, static analysis, etc, but have had to temper my plans.

Answer 1

Hmm... I've never had difficulty pursuading people of the benefits. You'd think they were obvious. You might find this question useful:

http://stackoverflow.com/questions/211426/what-punishment-do-you-have-when-someone-on-the-team-breaks-the-build/211669#211669

But I suspect it's dependent on the team already grasping the need to keep the build in a good state.

Answer 2

Have another process that does "Nightly Builds". This way as soon as something is checked in, that night you would know. This prevents from deploying broken code.

Also try implementing custom rules for FxCop

Answer 3

You could wire up an USB-alarm that goes off when the build breaks...

I think it's futile to change everybody at once. Start this way:

1. There must have been someone who had to do the actual work to fix other people's code so things compile at the end. Let's hope it's not just you all the time.

2. Show this guy the magic of annotate (or blame).

3. So now there's at least you, this guy (#2), and the blame guy (#3) - a total of 2 or 3 people. Now enlighten them that wouldn't it be nice if when people were checking in things worked, then no-one would be staying late (#2 guy) or have to explain to the manager (#3 guy) why the build's going to be delayed.

4. Repeat a few times? Something should click.

Answer 4

It's hard for me to imagine developers who "begrudgingly" use source control. That being the case, the first step is to get the devs to do frequent check-ins and updates to their source code so that a broken build becomes a hassle during on-going development and not just right before release. This will reinforce "don't break the build".

At that point, the benefits of CI are obvious. I recently moved my team to Subversion (from SourceSafe) and TeamCity and it has had tremendous benefits. I always know what's going on with our projects and I can always access the latest builds.

Answer 5

At the last place I worked, the CI system (we used CruiseControl) would send an email detailing anything that was broken to anyone who checked in code on the previous day. This way, everyone who checked in code was alerted that the build was broken and who the responsible party was. People were very careful about testing their changes on their own machines before checking them in.

Answer 6

This is fundamentally a human problem.
If your teammates do not see the value of CI, then you have two choices:

• convince them of the value of CI
• Have management come down with an edict (convince management of the $ savings)

The fact that your team did not even have source control leads me to some conclusions:

• They are day workers (9 to 5ers, just a job, not craftsmen, etc...)
• They are probably not considering the business implications
• Your company probably does not produce software as it's primary product.
• You will have a difficult time convincing them

With that in mind, you could try management, but don't expect chocolates for valentine's day from your coworkers if you succeed.

Intrinsic Value of continuous Integration:

• You know very soon when there is compilation problems
• You still have the code fresh in your mind when it breaks (vs. coming back next week)
• You avoid lots of integration issue, even if you don't have tests in place
• Helps avoid delays to releasing software
• Potentially saves the company money by avoiding missed deadlines

If this is not enough to convince a developer that cares about his craft, I don't know what is...

If you want to implement a disincentive to breaking the build, here are some ideas:

• Build Breaker pays $1 fine into a "Break Jar"
• Build Breaker has a "Build Breaker of the Day" Sign on their cubby
• Build Breaker gets an Ugly Mascot/Trophy on they desk/cubby wall
• Team announcement: "Joe broke the build"

But again, if nobody wants to participate (or cares) that might not work.
You might be perceived as a trouble maker.
Make sure you have management support before trying these steps.

Answer 7

Well, penalties for the person breaking the build do in fact, help. They don't have to be too harsh. Maybe that person has to get coffee for the rest of his colleagues until someone else breaks the build. Which does get annoying after the third cup.

That being said, I do think it should be possible to have them see how important it is to not break the build. Measure the time it takes to fix the build. Confront them with those findings. Show how much time is lost by fixing broken builds to management (time = money, so I'm sure they will listen to that).

Include unit tests in your CI. If anything, they show everyone that a piece of code just doesn't work. And every developer should understand that's not good. If they still don't see it, you might just be working at the wrong company.

Answer 8

Three steps to convince them:

1. Whoever breaks the build fixes it

2. Instead of nightly builds (giving everyone a good night sleep) try 'build on check in' and reject check ins that break the build. You now have to manage branches and merges but the instant pain effect will wake your devs up.

3. Publish a scorecard to the development team that shows :

   • Number of bugs outstanding and fixed per developer
   • Number of failed builds per developer
   • Number of successful builds per developer
   • Amount of time spent cleaning up their mess

And finally, I recommend Visible Ops by Gene Kim of Tripwire. While it's not software development-centric it is applicable to the kind of troubles you are having.

PS Unit testing may actually be a good next step towards a build that never breaks. Start with a 'every bug fix gets a unit test' philosophy and require all unit tests to pass before check in. Sneaky build requirement!

Answer 9

I've found that all the talking I did produced no results. However, when I just set the darned thing up to build on check-in and made it email the group when a build was broken people figured out the value themselves.

Make it public and make it near real-time. You can't make people agile, but you can involve them with as little negative impact on their lives as possible. If they see the new hotness and love it, then cool. If not, well, at least now you have CI set up for you!

Answer 10

If they don't care about the build, the build isn't being used. I was convinced not to break the build by the QA folks coming and telling me they couldn't work. Get some QA guys.