I want to write (or find) a guide to effective bug reporting in a style similar to ESR's How To Ask Questions The Smart Way
What are your top tips for effective bug reports?
A:
Write the steps to reproduce the bug. If you can't reproduce it, it won't get fixed.
Kibbee
2008-10-27 15:37:34
So no fixing race conditions or deadlocks then?
tloach
2008-10-27 18:36:23
+7
A:
- Step-by-step instructions on how to recreate the bug
- Make sure you've attempted to isolate the bug to what you are actually writing a bug against, instead of something else that could be the cause.
- List attempts to isolate the bug to something other than the software you are writing a bug against
- Make yourself available to answer questions and be available to help troubleshoot/recreate the bug
The bottom line is you have to engage some level of critical thinking when the bug is encountered. Once you've exhausted all possibilities that it could be your fault, write up a bug. If you find out its your fault, but the software you are using/testing could have done something more usable to indicate its your fault, still write a bug.
Also, to be a truly great bug-reporter, you must avail yourself to those testing the bug to help them recreate it. Its likely you've just "got the knack" for recreating that bug and there may be steps you are not conscious of. You can't just complain and walk away, participate in the process and help the team out by testing, recreating, and troubleshooting.
Doug T.
2008-10-27 15:39:16
Good answer. One more step - try to minimize the amount of 'stuff' that's necessary to reproduce it. If the bug report involves a 50,000 line program, it is not going to be treated as well as a 100 line program that reproduces the problem.
Jonathan Leffler
2008-10-27 16:02:14
@Jonathan: What if it requires that entire 50000 line program to reproduce it? Scared of the big ugly bug?
tloach
2008-10-27 18:53:25
If you can't do it by hand, use http://delta.tigris.org/ to automate minimizing the 50000 lines down to the minimal set required.
ephemient
2008-10-27 20:45:52
+2
A:
- Procedure used to re-create the bug including what was being done, what area of the application was being used and what event was happening at the time.
- Statement of reproduceability (reliably, not) - helps the developer know how hard it should be to reproduce so they don't give up to quickly
- Screen shots or documentation of error message / stack trace produced
- Criticality/Priority of the bug (can it be avoid, avoidance steps, is it catastropic, does it have a business impact, what's the business risk, etc)
- Environment - which environment was the bug found in. Remote, local, etc.
Too often, our QA people think they can just put in a ticket saying, here's my exception without any backup documentation. Its near impossible to reproduce let alone fix the issue without more information.
ScottCher
2008-10-27 15:44:31
+3
A:
Report the observable facts and then your interpretation of those facts.
Sometimes the best bug reports include something that is a gut feel of an understanding of the problem. Facts-only bug reporting discounts this valuable human resource.
Andy Lester
2008-10-27 15:48:43
+2
A:
Don't assume the reader of your bug report knows the software as well as you do. Even the person who wrote the software may not know what you are talking about if enough time has passed since they wrote it. Write it so that anyone can understand and reproduce the problem.
raven
2008-10-27 16:05:40
A:
- Always report version number of software under test
- Always report versions of any other software (browser, OS, etc.)
- Always list all hardware
- Steps to reproduce
- Symptoms of bug
- Screenshots, traces, logs, other attachments (if any)
- How critical -- crash, UI, etc.
- Report whether reproducible
- Anything else tried, that did or did not reproduce the bug
Laura
2008-10-27 18:29:03
+1
A:
For all the people who won't look at something without steps to reproduce:
My first programming co-op job I was assigned a bug that was essentially a random race condition that was making the system unstable. It happened at any point in the system execution, and all we had were a few stack traces pointing to a section of code that was pretty obviously fine. Somewhere another thread was mucking about with data it shouldn't be and if this thread was at the right point it would crash. Our QA got crashes about once a month. It took two weeks of combing through the system to find the culprit (yup, unchecked access to shared resources, about a 2 line fix) and fix it. There never was a decent steps to reproduce because there was no general way to reproduce it (save shoving a bunch of yield()'s in the right spot). If you're going to work on a multithreaded system, you better be ready to deal with bugs that can't be reproduced reliably, may not have stable steps to reproduce, and not just whine to QA because you couldn't reproduce the bug.
Note that the above is no excuse for QA to not include as much detail as they can when possible, just pointing out that it isn't always possible on modern software.
tloach
2008-10-27 18:48:42