How do you ensure the same bug does not creep into the product the second time?

Question

I understand the short answer to it is by doing testing but still how do you do this testing? Do you modify your test cases to include the bugs as additional test cases to run or do you just verify all the bugs in your bug tracking system starting from the oldest to the newest releases.

Thanks for the answers. Looks like my question was not clear. I understand we need to write bug report, fix the bug & do the testing for fix verification. However, under which test phase should this test go so that during the next version release, we are sure to re-run the test again to make sure none of the new changes have re-introduced the bug. Should it go under regression testing or should it go under integration testing for that specific project or should we just test all the bugs in the bug tracking system since version 1.0?

Answer 1

You create an automated test that reproduces the problem. You then fix the problem, and make sure that the new test and all your existing tests still pass.

---

This should be regression testing, and should be automated to the extent possible.

Answer 2

When you find a bug, the first thing you need to do is make a unit test that demonstrates the bug. You'll know when you've fixed it, because the test passes. You're protected from regressing it, because that test will suddenly fail.

Answer 3

1. Get a bug report
2. Write test case to reproduce bug report
3. Fix bug
4. Enjoy not having to fix that bug again

Answer 4

What do you mean by saying "You're protected from regressing it, because that test will suddenly fail."?

The point of "unit tests" is that they will be run automatically as part of the compile/check cycle: you'll run the tests after compiling and before checking code in. If a unit test fails, then you know that some code that you just wrote will recreate the old bug.

edit:

How can you be so sure that just because your unit tests passed all your bugs have been fixed.

Generally, if you can reproduce a bug, you can make a test to duplicate it in code. The most important part is once you've written a test to check for the existance of a bug, then if you run the tests, and the test fails, then you reintroduced the bug. One great book on the subject is "Working Effectively with Legacy Code"

Is it not possible for a single bug to span multiple unit tests.

Of course it can span multiple tests.

Let's presume the bug is "can't save files."

And let's presume that there are several different scenarios that can cause this:

• out of disk space,

• the file you're writing on top of is opened and locked by another process,

• there are permissions problems (like you can't access the directory, or don't have write permissions),

• or there is a network error while you're saving the file.

Ideally, your test harness (a collection of tests) would have a separate test for each one. For some of the tests, you may need to use stuff that are called "mock objects" (they can also be used when you haven't written the code for the component yet).

Answer 5

A "bug" in a system is just some unit of functionality not working. By createing unit tests that covers the unit of functionality you can know if the bug reappears for that unit of work. Now if you write additional unit tests testing other areas of functionality that might fail in a simular situation you will get more coverage.

That said unit testing is only part of the answer. Adding a test case for this bug as well will also help catch it from occuring but unfortunatly it will be later in the development lifecycle and more expensive to fix.

Answer 6

As a QA you need to anticipate the areas that would get affected as a result of some fixes. Thorougly test the application for the same though it is highly recommended to regress the entire build after each freeze.

If it is a long term project and you have time at hand, then it is a good practice to automate the test however if the project is being developed in agile model within short span of time and changing requirement then do it with manual testing trying to cover as much as possible.

Answer 7

Hey.
Depending from which perspective you ask, you can attack issue in different ways. From developers perspective, it could be best to have all bugs since 1.0 (or earlier) checked with proper unit test, and let it run even with CI. When it becomes to much hassle try to group it, essential tests that will run with every build, some once a day, rest once a week (or something).

From QA/Testing perspective it requires a bit more effort.

1. Prepare regression suit for application. Main business processes/use cases, the happy paths. Make it your baseline. Set of tests that must fail under no circumstances. Run it always before release or when big changes are made, when many changes are made, when critical parts are modified. Put some effort to automate it (and this time I'm not talking about unit tests, but full application tests).
2. Than create regression tests suits for areas of the application. More tests than in previous suit, but each regression suit focusing on different application area. When you change one part of application, run regression for that part. it can be automated, but it can be hard to maintain tests for areas that change often.
3. Educate your qa team to look through bugs repository as they start working on tests to some requirement/functionality. They should check for previous bugs if they seem relevant.

Keep in mind that you must be careful with regression testing, test automation, and desire to check for all bugs from version 1.0. Maybe application changed so much since version 1.0 that some bugs are no longer relevant? That should be taken into consideration when building regression tests based on old bugs.