How to prioritize bugs?

Question

In my current company there isn't clear understanding between the test and development teams as to how severe a bug should be? There are arguments which go back and forth to reduce or to increase the severity. We are not as of now aware of any documents which lays the rules. The tester raises the bug and assigns priority based on his intuition. The developer would request a change based on his load or some other factor.

How are severity/priority of bugs classified? Are there any standards which guide how software defect priorities needs to be determined based on customer needs, time lines and other things?

Answer 1

Replace your bug tracking system with fogbugz and get rid of severity field altogether.

See Priority vs Severity

Answer 2

As for a standard, IEEE guide to classification for software anomalies although I am not sure how widely this is adopted. IEEE 1044.1-1995

Answer 3

Set the requirements of the project so you can base the priority of a fix on the priority of the requirements interfered by the bug.

Answer 4

One option is to have the product owner determine the priority of the bug. While there is some general intuition on how "bad" a bug is, it can be the responsibility of the owner of the product to set an order of precidence (i.e. bug A should be fixed before bug B etc...).

The more information (clear and concise) that can be provided to the product owner can assist that individual make those determinations (i.e. how many users have experienced the bug, what features are not available as a result of the bug, etc...)

Answer 5

1. Must be done now
2. Must be done before we ship
3. Minor annoyance (Doesn't prevent the user from exercising the functionality)
4. Edge case/Remote/Tester-from-Mordor scenario

Well I just made that up... my point being categorizing bugs should not be a weekly hour+ long ritual..
IMHO, prioritizing acc to a flowchart is wasted time. Fix bugs in Cat#1 and #2 - as quickly as they surface. If you find yourself swamped by bugs, slow down and reflect. Defer Cat#3 and Cat#4 if the schedule doesn't permit or higher priority items override.
The critical thing is that all of you have a shared understanding of this severity and expected quality. Don't let compliance to the holy standards of X slow you down from delivering what the customer wants... working software.

Answer 6

• Use priority levels that deliberately have nothing to do with severity or impact, and describe only the conceptual position of the bug in the schedule. This field will determine which bugs get worked on, so it needs to be very clear that the facts of the bug aren't open for negotiation.

• Use severity levels that deliberately have concrete, verifiable definitions, that have nothing to do with scheduling or priority. I've worked successfully with the severity definitions used by the Debian BTS, generalised to apply to programming projects in general.

That way, the severity is much more a matter of verifiable fact, independent of a statement of priority. The priority is then free to be tweaked up and down by negotiation or whatever, without affecting the factual information in the severity field.

Attempting to conflate both “severity” and “priority” into a single field will lead to soul-draining arguments and wasted time. The bug reporter needs a firm guide of fact to determine how “bad” the bug is, and this needs to be easily agreed on by independent parties. The priority, on the other hand, is the correct target for negotiation and scheduling games.

Answer 7

I use the following categories both for features and bugs:

1. Showstopper, the program (or a major feature) will not work
2. Must have, a significant part of the customers will be bothered by this
3. Would have, some customers will be bothered
4. Nice to have, a few customers want this

Normally you plan to fix 1, 2 and 3, but 3 is often postponed to a next release due to time constraints.

Answer 8

I had the same issue with one of our customers. In the end we set up a document together describing what kind of bugs would match to a certain severity. Aside from an occasional discussion using this document as a guideline appears to work.

But be well aware that test teams and development teams may have very different opinions on what is a severe bug and what is not. From the point of view of the testers a small layout bug can be high priority when a developer would just say that no one will notice.

In our document those bugs can be high priority if they are "brand damaging", i.e. if the layout bug is in the logo or one of the products then it is severe - if it's just a paragraph on the page that is 2 pixels off then it's not.

Answer 9

I work on emergency control centre systems, so this set of bug levels is a little, well... extreme:

• someone dies
• total system failure requiring DR invocation
• server failure requiring engineer response
• failure involving loss of call continuity
• failure involving loss of data
• incorrect data recorded
• application failure - non-recoverable
• application failure - non-recoverable, but automatically restarted
• does not meet requirement spec, no workaround
• does not meet requirement spec, but has workaround
• cosmetic - layout etc.
• actually a feature request

That's off the top of my head. In case you were wondering, it's from most extreme to least :-)

Answer 10

Personally I favour the two tier severity/priority model. I know the arguments for a single level but the places I've worked generally I've just seen a two level heirarchy work better

Severity is set by the support team (based on input from the client). Priority is set by the client (with input from the support team).

For severity I use:

1 - Blocker/show stopped
2 - Major functionality unavailable (or effectively unavailable), no practical work around possible
3 - Major functionality unavailable (or ...), work around possible
4 - Minor functionality unavailable (or effectively unavailable), no work around possible
5 - Minor functionality unavailable (or ...), work around possible
6 - Cosmetic or other trivial

Then for priority I just use High, Medium, Low but anything from 3 - 5 levels works (much more than that is just over the top).

I'd generally then order by Priority first and then severity within that. The important thing about this is that the client has the most important say. If they say the way their logo is printing out on a report is the highest priority then that's what gets looked at BUT it gets looked at after the other client's high priority which is stopping them logging in.

Generally speaking I wouldn't release with any high priority issues or any medium priority issues with severity 1 - 4. Obviously in an ideal world you'd fix everything but I've never been lucky enough to have that option.

Answer 11

• The tester tells what is broken
• The developer estimates how much work it will be to fix
• The customer decides the business value, i.e. the priority.

Answer 12

I think this is the scale we used at a previous job:

1. Causes loss of files or system instability.
2. Crashes the program.
3. Feature doesn't work.
4. Feature doesn't work, but there are workarounds.
5. Cosmetic issue.
6. Request for enhancement.

Sometimes this was abused - if a feature was so poorly designed that someone couldn't figure out how to use it, that was classified as a 6, and it never got fixed.

Answer 13

Some stuff we used before. We split the defect rating into priority and severity.

Severity (set by submitter during submission of defect)

• Highest (5): Data loss, hardware damage possible, or a security-related failure
• High (4): Loss of functionality without any reasonable workaround
• Medium (3): Loss of functionality with a reasonable workaround
• Low (2): Partial loss of a function or a feature set (feature still hits the design requirements)
• Lowest (1): A cosmetic error

Priority (adjusted by development, management and QA during defect evaluation)

• Highest (5): The system is practically unusable with this defect.
• High (4): The defect will have a serious impact on the company’s ability to sell and maintain this system.
• Medium (3): The company will lose some money if this defect is in the system, but it might be more important to meet the schedule. Fix after release.
• Low (2): Do not delay the release, but do fix this problem afterwards.
• Lowest (1): Fix as time and resources allow.

Both numbers together create a risk priority number (RPN). Simply multiply severity with priority. Higher result means higher risk. 25 defines the ultimate defect bomb. 1 can be done during idle time or if someone is bored and needs something to do.

First goal: Defects with a rating of highest or high of any kind should be fixed before release. Second goal: Defects with RPN > 8 should be fixed before releasing the product.

---

This is of course a little bit artificial but helps to give all parties (Support, QA/Test, Engineering, and Product Managers) a tool to set priorities without blowing away the opinion of other side.

Answer 14

"Been there done that".

I've had this discussion over and over again, on different projects. We've tried to combine priority with severity, but the lesson I've learned: do not combine severity with priority !

We've had a lot of brainstorms and meetings which ended with the words "this is it". Multiple guideline-documents have been created and spread between the different "parties", but after a while we discovered that it didn't work at the end. Different "parties" think different about bugs: our helpdesk has another understanding of priority than the development team or the sales has.

Having both a severity and a priority level will very quickly become very confusion because:

• when using numbers (between 1 to 5) one will not know what each number means
• what if an issue has the highest possible priority, but the lowest possible severity - and I'm sure that this will happen!
• what if someone reduces a severity, does he need to reduce the priority also?

"So what should you do then?":

• Only use one kind of indicator for the 'level' of an issue: Doesn't matter how you call it.

• Use numbers (eg 1 - 5, but could be more or less depending on your needs) to clearly indicate the importance but combine it with a keyword so that it's clear what it means (eg. 'nice to have', 'show stopper'). For some people prio 1 means the most import, for others 5 does -> therefore a keyword to indicate what a number means is necessary.

• Make a distinction between a 'normal issue' or a 'red alert'. In our case a 'Red Alert' must be solved immediately and put in production immediately. A normal issue will follow the normal development-test-deployment-flow. The priority/severity/however-how-you-call-it should only be set for normal issues and will be ignored for 'red alerts'. *> In practice, a 'Red Alert' can become a

  'Normal Issue': the support team discovered a major bug and created a 'Red Alert'. But after some investigation we discovered that data had become 'corrupt' in the database since it was inserted there directly and not via the application.*

• Choose a good tool that allows you to customize the flow; but most tools do.