Should we fix that bug?

Question

When triaging bugs for a release, what criteria are typically used to determine whether the bug will be fixed for the release?

Answer 1

• Severity of effect on user
• Frequency of appearance
• Is workaround available?
• Cost to fix
• Time to fix
• Time remaining before release deadline
• Availability of resources who can make the fix

Answer 2

In my experience, it's just:

• Severity of the bug
• Amount of free dev time before the release.

Answer 3

Criticality, impact and money usually.

1. Criticality: What happens? Does it corrupt data, bring down the system, that kind of thing.
2. Impact: How many people will be affected?
3. Money: Will someone pay us (or worse, withhold payment) if we (do not) fix it?

Answer 4

I write RDBMS server software, so any bug that may cause data corruption is immediately a show-stopper. Also, any bug that can cause the database server to crash under relatively normal use would qualify, as would returning incorrect data from a query.

It also has to depend on how destabilizing the fix is and how long it will take.

Answer 5

The cannonical article on this is My life as a Code Economist, by Eric Sink.

It's really worth reading the article, but if you want it summed up for you in a checklist:

1. When this bug happens, how bad is the impact? - Severity
2. How often does this bug happen? - Frequency
3. How much effort would be required to fix this bug? - Cost
4. What is the risk of fixing this bug? - Risk

Answer 6

Quite frequently for us if the severity is low and the product is nearing release, it is better to save the fix for the next release.

The let sleeping dogs lie principle.

There comes a point where the code needs to be locked down. Code fixes require further regression testing and that takes time.

Sad but true.

Answer 7

This is definitely a domain specific question. I write large trading apps for Hedge Funds, Prop Trading Desks, Mutual Funds etc. so:

1. Things that violate compliance or cause other legal problems are very^4 bad
2. Things that can cause overtrading (you wanted to buy 100,000 shares of DTE but we got you 100,200) are very^3 bad
3. Things that prevents the client from trading when they want to are very^2 bad
4. Things that inconvenience the client are very bad

I did talk to a 'Bug Triage Manager' the other day who works in a different field. He put it simply:

First I fix the crashing, then I fix the things that cause us to loose money, then I fix the things that makes us look bad, then I fix everything else.

Answer 8

Things that affect severity I haven't seen yet in this post.

• SLA - Does it have an impact on SLA?
• Internal(maintenance) vs External Benefit - External benefit almost always gets higher priority
• Visual vs Functional - functional generally gets higher priority
• Who found it? (customer vs co-worker) - Customer gets higher priority

Answer 9

As a rule we will patch any bug that crashes the system outside of the release cycle.

Any other bug gets added to the product backlog and and is prioritized along with the new features. When determining what goes in the release we just take those tasks with the highest priority. This works well for us as we have a monthly release cycle.

We use much the same system of prioritizing bugs as mentioned by others on this thread, with the added exception that a client can pay to raise the priority of a bug (or feature).

Answer 10

There is a lot to be said for the "zero defect" mentality. Bugs of any kind should always go to the top of the stack, or eventually they will overwhelm you.

Of course, in the real world, getting that sparkly new feature out there so you can win that big new client might be more important than fixing some slight annoyance. Time to market really does beat quality sometimes.

Answer 11

Is it worth getting into technical debt by not fixing it, or doing a quick and dirty fix with the intention of fixing it properly later.

Answer 12

There are lots of great answers already and of course circumstances vary depending on project/company/bug/time to release/dependencies/...

I have however found the following two metrics a useful guideline.

• How many users are affected?
• How badly are they affected?

The higher the bug is marked on both variables, the sooner it should be fixed.

Answer 13

I agree with @Chris Upchurch, but I think a key factor is to have agreement from all parties before you get into crunchtime. That way, you can run down your checklist with a minimum of teeth-gnashing and chest-pounding.

Answer 14

In terms of making the decisions, can't better the top answer based on severity, impact, cost etc.

Need to be a little careful about when you apply these rules however. I have worked on too many projects where bugs were left to get out of control because they weren't anybodys responsibility. The best working rules I came up with were

• If you find a bug raise a bug report
• If you can fix it, do so. ( report is raised as the existence of the bug may need reporting )
• If the Team Lead understands the bug a priority must be assigned (or other measures used by the team). Add 'what this really means' information. It is important that the Team Lead has (some) control over the bug list.
• Review bugs not fixed / prioritised at management level. Not understood bugs are a gaping risk and the powers that be should have the opportunity to understand the risk.
• At all times, have a project wide view of what the top bugs are (this can be separate to priority e.g. a top 5 list)
• Encourage programmers to fix bugs alongside development tasks

THEN when you are considering a release.

• branch the code and instigate a feature freeze
• identify the bugs you intend to fix (no need to order them :-) )
• only add bugs to this list that are reproducible on the frozen code and agreed