How do you enforce or maintain the quality of the bug reports in your bug tracker?

Question

High-quality bug reports are essential for effective bug tracking - in an ideal world all bug reports would contain essential information such as which version of the software it affects and a step-by-step description on how to reproduce the bug.

In reality, though, bugs reported can vary a lot in quality. They might be on-liners ("feature X doesn't work, fix it!"), feature requests, PEBKACs or unintelligble.

How do you enforce or maintain the quality of the bug reports in your bug tracker to stay effective?

Answer 1

This depends on whether you're talking about a closed QA review and a public beta.

If it's a public beta it's not advisable to let users directly edit your bug list. Someone should be assigned to aggregate user comments and reports and discern which are actual bugs and which are duplicates and which give some sort of clue on how to replicate them.

If this, however, is a bug item that is posted by your legitimate QA personnel, you have a competence problem with regards to your employees. Proper guidelines must be set on how to report bugs, especially in getting replication steps straight.

Answer 2

I agree with Jon Limjap - your QA personnel must be competent enough to post appropriate bug reports, given the right basic training and guidelines. Nevertheless, there are ways to enforce and encourage better bug reporting:

• Most bug tracking software have a way of marking some fields of the bug report as mandatory, so that the reporter has to actually choose the appropriate value in order to successfully create the bug
• There is usually a possibility to include a basic template for the bug report, something in the lines of

Scenario:

Expected Results:

Actual Results:

Remarks:

• You can (and should) provide a bug-reporting tool that will be run on the problematic machine, gather the relevant information and pack it into an archive file (and maybe place it on the desktop). You then instruct your staff to run it whenever they encounter a bug they wish to report and attach the archive to the bug. This tool should be easy to use (just running an executable) so that they would attach the diagnostics information to any bug without having to think if it is relevant or not. This tool is usually also very useful with customers.
• Last but not least - "education, education, education". People learn the best from experience - just make sure that whenever someone opens a bug without the proper information included, the person handling the bug would go and talk to the one who opened the bug, and explain what is missing and why it is important.

These are practices we have been using quite successfully in my current workplace and I believe them to be quite universal to fit to most working environments.

Answer 3

Tough question. I'd try and see if the system has any way to enforce that certain fields that you require are entered, try and have whatever bugs that are critical coming across your eyes somehow (email, rss) so you can pounce quickly, but mostly that your team is aware of the quality standard and keep to it, guidelines are published and public, that sort of thing.

Assuming it's your team: If you can have a certain structure that is used every time in a comments field, of what is expected when it is entered, then that would be good too - even better if your software has a default notes outline where you can define that structure on a blank form.

To some degree though, it's up to the individual, they have to be aware that it's part of the communication standards, it's expected as a job requirement, and that they're responsible to every other member of the team - because other people shouldn't be in a position of hunting them down to find out any further details if it could be avoided.

Especially since the turnaround time on fixing bugs on lower priority items could be some time and people are bound to be forgetful on the details.

Assuming it's users: You can't to a high degree, but I'd try - if possible, to ask questions on whatever form in a way that people could understand.

Not entirely on this topic, but in a "how you ask the questions" kind of way, is this post on 37 Signals blog - link text

Even if you have to have another form asking the questions visible to your users, that only feeds mostly the data to bug program, I'd do it just to ask the right questions.

What product? What version (pictures showing how to find it)? It'd be helpful to include a screen dump, if they could open the program and press a button to send through a log file automatically, whether it stopped them from doing further work, whether it lost their changes, etc.

For the users it's probably more about how you ask the questions, and letting them know that you require certain ones to be answered, or which ones you'd find more helpful, then you'd probably get better responses.

Answer 4

Write a good but not too lengthy tutorial on using the tracker and what is required for each field. Make a general purpose reference example that others can use if they get stuck.

I have a reference copy for editing Docbook manual pages and by using this repeatedly I already know most of the syntax by heart.

Answer 5

I used to think that the quality of the bug report was tantamount. I still think so... bugs that I report have much more useful information in them than the ones entered by QA or by Operations. However, I've come around to admiring FogBugz's model. It's extremely simple to enter a bug. Just knowing there is an error condition is helpful, even if there's not a lot of supporting information. Plus, users feel like something is getting done.

Answer 6

Use something like UserVoice for the end-users to report bugs and feature requests. Bug tracker entries really should be internal - they are too technical for end-users and also, IMHO, expose a little too much internal workings.