Defect-free Software

Question

About ten years ago, a "Terry Colligan" published a buncha interesting pages on defect-free software. He noticed that a woman who worked for him produced code with lots fewer bugs and his company expanded on this --- even advertising "automatic QA".

At least some of his web pages are still up: http://www.tenberry.com/nodefect.htm

What ever happened to him? Were any of the ideas any good? How about automated QA?

Regards, Bill Drissel Grand Priarie, TX

Answer 1

I don't know what happened to him, but his ideas have been merged or still live on today in various forms.

Consider this list he has: http://www.tenberry.com/nodefect/steps.html

The later steps, the one that talk about real implementation details, are embodied in the modern TDD movement. Repeatability is the primary virtue here. With Repeatability you can achieve more scientific analysis on your code and its behavior and test appropriately.

How do you achieve Repeatability? Today it's:

• Use source control religiously
• Automate your build process, eliminate all magic. I should be able to check out your code and run a "build.bat" and it should work. It's OK to have a few basic requirements like "Windows XP or later, .NET Framework 3.0 or later, .NET Framework SDK" but make sure these are verified IN YOUR BUILD SCRIPT! If I run your script and I don't have the SDK installed, it should fail immediately and tell me this
• Do TDD and commit to running your unit tests frequently. Automated unit tests, run frequently, is key.
• Do Continuous Integration. The code should be able to be built and tests run successfully on a SEPARATE box that is NOT a developer machine. This is necessary to eliminate the "Works on my box!" magic problems
• Automate higher levels of testing. Use things like WatiN or Selenium to automate web testing or NUnit Forms to automate Windows Forms programming (in a .NET World, there are equivalents for Java and other platforms).
• Automate the deployment process. Remove as much magic in your deployment process as possible to avoid problems where only one member of your team knows the correct order of incantations and incense necessary to produce a good build AND deploy it.

At the risk of plugging myself, I wrote up a blog post on this subject that covers it pretty well. http://www.lostechies.com/blogs/chad_myers/archive/2008/03/16/time-to-login-screen-and-the-absolute-basic-requirements-for-good-software.aspx

If anyone objects to me linking this, I'm willing to contribute all the content on that post to this public forum under a Community Commons "Share Alike" license. I didn't want to just copy/paste it here, but if the community wills it, I will.

Answer 2

Although this is probably not what you are asking about, model checking may apply here too. Put simply, model checking is a technique that can semi-automatically prove the absence of certain types of bugs in certain types of programs. It is typically used when programming parallel systems as it can detect for instance deadlocks.

Answer 3

While it's a laudable goal, in reality one can never assume that any given software is going to be truly defect free. If high reliability is called for, then the service being offered must be supported by redundant, parallel servers.

No matter what we do, no matter how careful we are, or how thoroughly we verify and test, our software will fail eventually. Whatever SLA (formal or informal) that is offered must take this into account.

Answer 4

His ideas are quite common in software development world, but I stand firm on my believe that: No matter how perfect your process is, how good your developers are, defect-free software is rather un-achievable.

RWendi

Answer 5

While I agree that defect free software may be unachievable, it should be a goal. Striving for zero defects and falling short is a much better path than writing sloppy software because "bugs are inevitable".

I find it funny that this stuff was written 10 years ago, and we (software engineers) are still barely adopting techniques to prevent defects even though the complexity of our software is exploding.

The Practical Guide to Defect Prevention is a recent work that covers similar topics.