Daily Build vs. Zero Defect

Question

How do you go about doing a daily build and striving for a zero-defect environment? Does it mean I never get to go home until I've killed all the bugs in my new code? Or does it mean I just don't check my code back in until I've fully tested it, which leaves the code effectively branched for a much longer time?

I'm working with a handful of programmers for the first time (as opposed to working on my own, or with just one other coder), so I'm just wrestling with decisions like this for the first time. Should we adopt a software development process?

Answer 1

Simple: Never check in code with (known) bugs in it. This doesn't mean you check in once per day. Check in when you have a meaningful change implemented so the other developers can get access to it.

We always integrate locally, run our tests against the code, and when all passes, we check in. I check in about 20-30 times a day when working. The build server picks up changes and it runs builds against the system. Continous Integration (CI) is a good thing. :D

Continuous Integration - Automate Your Builds

Start out with building successfully and keep it that way as much as possible. It is essential in a team environment. Just remember that builds will break. It's expected that they break every once in awhile. It is a sign that you just inadvertently checked in something bad, and you stop what you are doing to make the build green again. A build server that never has broken builds is a warning sign!

I also agree with chadmyers' answer: Whatever you decide, it needs to be automatic and automated. The best thing about automating tools to do this kind of stuff for you is that you no longer have to think about it or remember to do it. Or like Chad said, you don't stop doing it. I could recommend make a recommendation for CI tools but have a look here: http://stackoverflow.com/questions/99535/what-tools-do-you-use-for-automated-builds-automated-deployments-why

Once you have CI, you can get higher quality if you can inject some humor (and shame) by introducing a broken build token! http://ferventcoder.com/archive/2008/08/20/continuous-integration-enhancement--the-broken-build-token.aspx

Use a Good Tool for Automated Builds

Most people in .NET land use NAnt or MSBuild scripts to have automated builds that they can later hook up to their CI server. If you are just starting out, my suggestion would be to use UppercuT, it is an insanely easy to use build framework that uses NAnt. Here is a second link with deeper explanations: UppercuT.

Branches vs Trunk for Active Development

You would not have to branch unless you leave trunk open only for releases (which means that everyone else is working in the same branch as you). But I would have CI on both the trunk and the active development branch.

Software Development Process

Also to answer the question on a software development process, the answer is a resounding yes. But don't rush into anything unless a drastic change is required. Pick a process you want to migrate towards and slowly start adopting processes. And evaluate, evaluate, evaluate. If the particular process is not working for your group, figure out if you are doing something wrong or if you just need to eliminate it. And then do. Whatever process you end up with needs to work for you or it won't work.

Answer 2

If you don't go home till all your defects are gone then you will never go home.

My thoughts on this are that the daily build should be automated at certain times. Any code not checked in before this doesn't get built and if there are no checkins from someone for 2 days(builds) in a row then the build system should notify them and the tech lead as this is a warning sign.

Answer 3

One perhaps more pragmatic approach is to have zero defects on the trunk, and branch for all development, then having daily builds is possible on both the trunk and branches, but zero defects does not apply to dev branches.

While there may still be a certain level of stigma from having your branch breaking its builds, it is less of a problem than breaking the trunk.

Answer 4

Yes, please adopt a software development process. There are a variety out there, of which I'm sure more than one will fit your team. Even one that isn't a perfect match is much better than no process at all.

So how does my company go about having daily builds and striving for zero-defects? We run our test suite before we check in our code. The unique problem for us is that a full run of our test suite takes over 72 hours, so we run a limited set of unit tests before checking in code. For our nightly builds, we run a set of tests that take about 8 hours to run. Then on the weekends we run the full test suite. Each stage catches more and more problems, but over 90% are caught with the 5-minute developer tests and probably over 98% with the nightly tests. This still alerts us pretty early to problems before they get out to our customers and cost a lot to fix.

Answer 5

Integrate early, integrate often, integrate fast. Rather than a 'daily build', build every time someone commits and commit often (at least once a day, preferably more than 2).

Important: Fast feedback is necessary for low-defect. If your build takes many minutes or even over an hour, eventually you will grow to hate the build, learn to avoid it, run it as little as possible, etc. It's value drops rapidly to the point of being worthless and your defect count will start skyrocketing.

Invest some time up front to getting your build running fast. If there is slow stuff, find out why it's slow and eliminate that. If you can't, then at least set up a cascading builds so the rest of the build goes fast (think <2-5 minutes) and the long-running stuff can follow immediately after and take as long as it wants (though try to keep it under 10m tops).

Can't say it enough: Fast feedback cycle on changes is extremely important!

Answer 6

It means make much smaller commits. The more often you commit working revisions, the less often your own working copy is ever broken. Iterative development starts with you.

Answer 7

The trick is to check-in as often as possible, just made some tests pass, nice check it in! Fixed a single bug, check it in! Try to find the smalles increment possible and check it in! This has the added benefit of actually making it possible and convinent to write checkin comments that actually are relevant so thats a nice bonus.

Ofcourse that requires that you have a CI environment that builds more often than the nightly, as often as possible really is the best option there.

Oh and remember, if it never breaks, then you're doing it wrong. (I.e. you're being overly conservative, a little bit of breakage now and then only goes to show that you're hopefully pushing it.)

Answer 8

Depending on what you're building, adopting an approach that defects are not allowed may not be appropriate. My personal opinion is that it rarely, if ever, is.

The whole point of a defect management system is exactly that - to allow you to manage defects. If the defect is a show-stopping one then sure, you probably don't want to check it in, but if it's something minor or an edge case then it may make sense to check it in with the known defect as long as you're tracking it.

Allowing defects to exist lets you focus on more important things - for example if you only have a limited amount of time in a release you may not have time to fix everything as well as to get all functionality in, so if it's a choice between fixing ten edge-case minor bugs, or creating a piece of value-adding functionality then the pragmatic choice may be to ship with known bugs.

I'm not saying zero defects is a bad idea - in fact we strive for this by the end of each release cycle - but like many things in software development, pragmatism normally works better in the real world than puritanism.

Answer 9

I'd go with @feverentcoder on the CI arguments. CI is your friend, let him help you!

As for the Branch/Trunk point- everyone should always work on the trunk, branches are for spikes and POCs, tag all releases

When it comes to processes, it's usually beneficial to find practices that are are relevant to you and then build micro-processes around them. Also, use only those practices/processes that you feel enhance productivity. Lastly, be courageous- try a practice for a week or two to see if it works with you; if it doesn't, throw it away. You just learned one more way not to make a light bulb!

Answer 10

About the zero-defect-strategy: You can go home, if known bugs are in your code. It's more about, that defects should be fixed, before new features are implemented.

That must not apply for the whole team, but if a developer has a bug assigned, that bug has priority over new features this developers has to create.

Answer 11

Looking through the answers I'm surprised that nobody has mentioned Test Driven Development. If your goal is zero-defects that's the best place to start.

After that I would strongly recommend pair-programming.

Finally understand that tools like CruiseControl are useful but, as Jim Shore said, continuous integration is an attitude not a tool. It is the group commitment to keeping the code working that is key.