QA vs Development Ratio

Question

I am working as a software developer and I had a quarrel today with our QA team about the following:

How much the QA team members should exceed the number of developers that are working on the same product?

I know that this is not a question about how to program something, but I think that this question is pretty much connected to the software development. So I hope that this question will not be closed. Instead I will get answers from professional programmers that have a good experience on working in SW developing companies so that I could make a good statistics.

Answer 1

Well this depends on quality of staff at the end of the day. If one programmer does as much work as two QA's, then the ratio is 1:2, and vice versa. Quality of staff here would be #1.

Answer 2

For most projects at the company I work at the ratio is 1:1. But this can vary on several factors:

• Dev output. I've seen one dev who had a high amount of output and had 3 QA working on his features.
• Quality bar for the product. A mission critical, high reliability system should have a higher QA bar than an internal reporting website, and will need more QA people.
• Some projects must be tested in higher number of configurations and scenarios. Devs might stay constant, but you will obviously need more QA to cover the full test matrix.
• How automatable the testing is. If testing can't be easily automated, you'll need more people to do manual passes.

Answer 3

Think in terms of hours spent as opposed to number of people. It is entirely possible that for a well tested and "Approved" application, for every development hour, there might be an hour of QA effort. I refer specifically to the technical QA role, not the functionals testing. A QA team and development team need to be able to work closely together, so the QA team can write test cases at the same time as development occurs. This means everything needs to be written to an implementation contract (function names, input params, etc).

Answer 4

The answer is very subjective, but here is my experience.

At Microsoft we have a strong test development organization. That's a little different than traditional QA because we hire programmers to test and involve them in the process as early as the design phase. Their job is to test and especially to automate the testing for the product. In my experience, it takes the tester approximately as long to test and automate a feature as it does the developer to code and fix bugs in the product. That implies a 1:1 mapping. This is very similar to the rule of thumb that writing unit tests takes about as long as writing the code.

This mix will vary depending on several criteria:

1. How much unit testing the developers are doing. The more they do, the less test needs to do.
2. How much the developers are writing from scratch versus leveraging existing libraries. If there is a lot of pre-existing code in use and the testers need to verify that functionality as well, that sunk development cost must be taken into account for the 1:1 mapping.
3. How dynamic the development is. If you are writing a UI where relatively small developer tweaks cause a large change to the testable surface, you will need more testers invovled.
4. How mission critical the feature is. To write something like GMail where it is used casually and bugs can be tolerated and fixed in the field, fewer testers are needed. At the other extreme, if you are working on medical imagin devices, you need a lot more testing because bugs are hard to fix in the field and very bad when they happen.

Answer 5

There are many factors that go into answering that.

1. Do you have automated testing?
2. How are your release cycles structured?
3. What is your unit test coverage % ?
4. How good are your people (both QA & Dev)?
5. Are you including QA in the entire project life cycle or do you dump on them at the end?
6. Are you doing incremental releases to QA?

I've worked places where it varied from 3:1 (QA/DEV) to .5:1 (QA/DEV). It boils down to how many QA resources it takes to adequately test the product and there's no catch-all answer to that.

Answer 6

In my experience, there are two major kinds of QA staff: those who simply follow a written script and interact with an application in an effort to find edge cases, and those who can actually write automated testing code themselves, and seek to find new and innovative ways (fuzzing, Selenium, writing API clients) to break the dev team's code.

If your QA team is made up primarily of folks of the first type, then a 1:1 ratio or better vs. your developers is probably a must. Otherwise, they'll struggle to keep up with any new features introduced by the dev team, and will often resist any changes made to the product, because it further complicates their testing workflow.

The latter type (i.e., test engineers who can code), on the other hand, are a godsend to any productive development team. The coders can communicate with them as peers, and the testers can find useful ways to automate and improve their own processes by writing smarter and better-abstracted test harnesses and utilities. One really good test engineer can probably support the work of 2-3 developers, especially if those developers are already writing useful unit and integration tests themselves that the tester can use as a starting point.

Answer 7

Right now where I work there are 3 developers for each QA person. This has its ups and downs as at times QA will find a problem but there are solutions outside of code changes,e.g. don't click where it is pointless to do so.

A couple of times there is no QA where I've worked which at times is almost like a recipe for disaster as customers then become the QA as their problems become developer problems.

Answer 8

Who are those mythical "QA team members" you speak of?

Answer 9

My place of work is currently around 8:1 dev:qa. The reason for this is that we take automated testing very seriously. All work is required to have close to full unit test coverage. We also engage in functional testing with Fitnesse (all user stories must have a fitnesse test), checkins trigger full test runs with a CI server, developers check in frequently, we release frequently.

This is all on a HUGE application with several thousand classes and countless scenarios. The advantage is speed, agility and of course cost. Whatever extra time a developer (even a expensive one) spends writing tests is less than the human capitol of hiring/managing more QA staff, or of finding bugs in production (even QA staff are human after all).

What little QA staff we do have gets to spend their time writing their own automated tests with Selenium or engaging in new functionality. They spend relatively little time rehashing the same functionality over and over again.

Answer 10

The amount of QA staff should not depend on the amount of developers. It should depend on the desirable quality of the product, but not on anything else.

Many here say that "to QA" a work of a good developer is an easier task than "to QA" a work of a worse developer. Hell, why's that true? To "assure quality"--and QA is "quality assurance"--means to design a process that marks the product with "QA passed" and "QA failed". I know only two processes that depend on the code itself: static code checking and peer review. While the former is somewhat used, and it sometimes needs QA guys to maintain it, the thing that's called "quality" of the code is not what matters for a soulless machine. And peer review is done by programmers, not QA. I hope this convinces you that amount of QA doesn't depend on developers, doesn't it?

In the domain our company is working in, there is no competition and the market is very narrow. Therefore we get all necessary info from bug reports and we have no QA, so the ratio is zero. All testing is done by developers. Still, we do live, because the quality required doesn't need hirind any QA guys.

Answer 11

In our organization the ratio is dev:QA is 5:2, and for this we need to understand sone more scenarios like

1. who is working on unit testing, In our case one person is fully dedicated to write unit test plan and team of 5 members are executing unit test cases and fixing the bugs our PL is not at all involved in coding he is doing only process/ review oriented things

2. functional testing is done by a part time testers you can say a half resource and one cycle of Functional testing done by developers

So it depends on the project size, LOC written and company resources based on their CMM level

Answer 12

There are many factors, the most important being quality level required for the application, is it a small web site? or a major medical instrument? or a financial system? A single line of code changed for the space shuttle could require weeks of testing...

In a progressive development shop, QA resource needs (as a ratio to development) should reduce over time as QA improvements are implemented, such as TDD, Code Reviews, etc. I think QA focused on pure hands on testing is a waste, QA should be utilized to improve process and help developers feel foolish (by removing bugs prior to release).