Exhaustive testing and the cost of "Bug Free"

Question

When I was learning software development, we were taught that actual "bug free" software was mathematically impossible for anything but the most trivial programs. For a mathematical mind, it's very simple to see how basic thingslike the number of possible inputs and the variability of platforms makes bug free not only impossible (in realistic time), but economically stupid for anything short of nuclear power generation.

However I'm constantly hearing business people spout off with "It's understood that software will be bug free, and if it's not all bugs should be fixed for free". I typically respond with "No, we'll fix any bugs found in the UAT period of (x) weeks" where x is defined by contract. This leads to a lot of arguments, and loss of work to people who are perfectly willing to promise the impossible.

Does anyone know of (or can express one) a good explanation of why "bug free" is NOT realistic OR standard -- that your average middle manager can understand?

Answer 1

See Gödel's incompleteness theorems

See also Hilbert's second problem

See also Turing: Halting Problem

See also MSFT financial reports for headcount numbers and Microsoft Connect for bug reporting on Microsoft products for a less intellectual explanation.

Answer 2

Perhaps the best way to explain it would be to read up a little on places that really, really, really don't want software bugs, and what they do about it. (You can use NASA as an example, and the Ariane 5 first launch as an example of what happens with such software bugs.) Middle managers tend to relate to stories and parallel examples.

Alternately, find out what happens with deals where one side does promise the impossible (and it happens much more often than developers like). If you can show that it doesn't end well for the promisee, that might help.

Also, you might want to go into what you would need as a bare minimum to promise bug-free software, which would be a truly comprehensive spec.

Answer 3

I usually walk managers through a simple explanation of how most programs are really state engines, and that as soon as you start interfacing with the real world the number of possible input states rapidly reaches infinity: because the user over here is inputting X while Y happens over here within 50ms of Z happening over there... etc, etc.

Granted, after about five minutes of this their eyes tend to glaze over. But at least I tried.