Deliberately adding bugs to assess QA processes

Question

How do you know that as many bugs as possiblle have been discovered and solved in a program? Couple of years ago I have read a document about debugging (I think it was some sort of HOWTO). Among other things, that document described a technique in which the programming team deliberately adds bugs into the code and passes it to the QA team. The QA process is considered completed when all the deliberately known bugs have been discovered.

Unfortunately, I cannot find this document, or any similar one with description of this trick. Can someone please point me to such a document?

EDIT

To make Evgeny happy, let me paraphrase the last sentence of the first paragraph:

"The QA process isn't complete before all the deliberate bugs are found"

Answer 1

I've never seen such a document, but I'd be wary of saying the QA process is "complete" just because your deliberate bugs have been found. It's a good way of ensuring that your QA team doesn't get too lazy, but you can't really ensure they did enough testing.

It's fundamentally impossible to know that all bugs (or "as many bugs as possible") have been discovered. All you can do is follow a thorough, well thought-out testing approach to give yourself a good shot at it.

Answer 2

One name for the technique is 'Fault Injection'. One of the older books on the subject is 'Software Fault Injection: Inoculating Programs Against Errors' by Jeffrey Voas and Gary McGraw.

Answer 3

I usually find that there are plenty of bugs for QA to find, without having to inject any deliberate ones ! I'd much rather have my QA team continue to find bugs that I never even imagined could be in the system.

Like Evgeny, I'd be very wary of calling the QA process complete when a set of known bugs has been found. I prefer to use exit criteria such as:

• Have all features defined for this release been completed ?
• Have all planned test cases been executed ?
• Is the open bug count within the accepted limits (e.g. no critical or high priority bugs, less than 10 low priority bugs, etc.)