Can writing unit tests be a profession unto itself?

Question

Are there any developers out there who's primary job description is to write unit tests and build test frameworks for other developers at the company?

It would seem to make sense for several reasons. For example, there are companies who are specialists in wind tunnel testing. They have standardized tests that they've created which product developers subject their products to.

Likewise, the mindset of a separate test developer may be different, causing them to catch more cases.

In terms of test driven development, the test writer could be someone like a senior developer who essentially outlines the framework that junior developers have to build against; refining their code until it passes their seniors' tests.

Naturally, I'm not saying that a developer and a test writer would never talk, they'd have to collaborate heavily. However, it would seem like a decent idea since it would keep both of them in the same headspace, rather than shifting from "test mode" to "development mode" all the time. After all, video game developers have game testers (yes, I know, not unit testing, its usability testing, but still).

Answer 1

Do you have testers? If your team doesn't have dedicated testers, at least one for every two or three programmers, you are either shipping buggy products, or you're wasting money by having $100/hour programmers do work that can be done by $30/hour testers. Skimping on testers is such an outrageous false economy that I'm simply blown away that more people don't recognize it. Joel Spolsky

Having a separate QA team is very important. You need people just running through the software to avoid wasting developer's time.

Writing test is also very important once the software becomes large. I saw some companies spending ages re-doing manually some tests over and over again. Every developer should spend some time thinking of ways to test his own code, but if you write code, you're not 100% able to test it correctly. What I mean is that if you have your hands totally in the code, you won't be able to take a step back and really look at what could go wrong.

Having an eye for testing is something that should be in every programmers skill set, but someone who will only do testing will be obviously better at it, so that makes sense having someone only doing this.

The best process would be something like this:

• Specs are done
• Programmers write code and Testers write test cases
• Programmers push code into staging environement
• Testers test (manually or using softwares/code like JUnit, RSpec or others)
• The bugs go back to the programmers
• Bugs are fixed
• Testers test again
• Everything goes into production if it's working. If not, start over

Edit: of course everyone should know how to use testing tools and write unit testing, because a lot of companies don't want to hire to do testing.

Answer 2

At Google we have the role of Software Engineer in Test (sample role description). SWETs don't tend to write the unit tests for the code (although they may write other tests, particularly integration tests) but they mentor SWEs in terms of testing and develop the various testing tools we have. Very useful indeed.

Note that this is very different from manual testers - SWETs are still primarily coders, whereas manual testers spend most of their time actually using the applications (and designing test cases etc of course).

Answer 3

Testers, yes.

But TDD unit-tests are not about the testing that testers do. (Note that there was a traditional definition of unit testing which was done by developers/testers prior to integration testing) By definition, TDD unit-tests come first, are automated, runnable specs, and really have no direct relationship to (what we would traditionally call) testers, except that fewer "misinterpretation of spec and/or invalid assumption" errors should sneak through to testers.

Unit-test developers, I'm not sure. This would be similar to an architect or designer role, perhaps? When this role is separated from the implementation developer you also get some communication and assumption problems. Of course, with TDD, you now have a runnable spec, but some of those issues might remain where the stakeholder defines the product and the test writer writes the tests, but then a problem is found later and a new test needs to be added. Does the developer resolve the issue with the stakeholder and just add the test and fix the code to meet the test requirements or does it go all the way back to a discussion with the designer etc. More people, more communication. It really depends on the environment and the size of the team.

Answer 4

Yes. We've got an automated test team where I work and they do this kind of stuff. Mainly they're writing system and integration tests, but they unit test too.

Answer 5

Not really, but depends on your definition of unit in unit testing. I tend to use the term "unit" for small-scale units such as classes or compilation units, tested as whiteboxes.

Unit testing TDD style at this level of abstraction is a design activity. There's little point of having separate designers-testers and coders who just implement the design.

Extensive unit testing non-TDD style on the other hand is rarely worth the effort, i.e. just writing a lot of tests for a code base where the design has already settled down. You get better value from higher level automated tests such as integration tests. Of course this does not mean that you should not be adding any unit tests at a late stage, just that the value of doing so should be understood. For example, adding unit tests to pinpoint an error and to make sure it stays fixed.

Answer 6

It definitely makes sense to have testers, and it may make sense to have people specializing in automated testing, but having people specialize as unit testers would be counterproductive, especially if you're using TDD, but also in other cases.

The whole concept of TDD is based on constant switching between writing a failing test, making it pass, and refactoring. If you find a bug, or a case you haven't covered, you also start by writing a failing test, and then you make it pass by fixing the bug. You lose that quick cycle by separating the roles, and you lose the essence of TDD. Someone would sit all day writing tests, guessing at the design that might emerge from implementing the relevant code, and then someone else would have to second-guess the test writer to write the code he was expecting. Neither of the two would get a full understanding of and feeling for the entirety of the design.

Often when I write code to satisfy one test case, that leads me naturally to the next test case - I discover a corner case that isn't covered by the code I just wrote, so I write a test case for it. What would you have your non-unit-test-guy do in that case? Or when he finds a bug?

Your wind tunnel testing example does have analogies in the software testing world: specialized types of testing that require special tools or skills: automated regression (system) tests, usability testing, performance testing, security reviews, etc. But not unit testing. Unit testing is a core part of programming.

Answer 7

To answer your question as it was asked...

No way!

A unit test is too small and focused a test, too entwined with a developer's duties and concerns, that it just plain doesn't make much sense to anyone but a developer.

To answer your question as I think you intended...

If we were to rephrase the question to "Can writing automated tests be a profession unto itself?" the answer is undoubtedly.

Not every automated test is a unit test. Sure, they run using a unit testing framework, because that's the easiest way to automate them, but that doesn't mean it's a unit test. That goes double for any tests that we write at the functional or acceptance testing level. These are not necessarily developer tests (in that developers write them as a coding practice). If you're talking about these kinds of tests, then sure, another "test engineer" would be perfect to help write and maintain them. But not unit tests.

<rant> We really need to stop calling everything a unit test. It's like we're small children that just learned the word "truck" and use it to refer to anything with wheels and a motor. Not the same thing. I wrote a blog entry about this a few months ago. </rant>

Answer 8

Well, let's assume you really enjoy writing unit tests. Nothin' wrong with that.

Now, writing unit tests for developers will tend to interfere with the TDD cycle. So we don't want to do that. But there are several options ....

1) You could be a "smoke jumper", specializing in jumping into old, legacy code modules, refactoring them and adding tests. Michael Feathers and Mike Schwern are two people who do this; you can google them. Feathers tends to do more training along with his work. (He's even written a book called "working effectively with legacy code" - http://www.amazon.com/Working-Effectively-Legacy-Robert-Martin/dp/0131177052/ref=sr_1_1?ie=UTF8&s=books&qid=1244467505&sr=1-1 ) As I understand it, Schwern has a long-term contract with a particular employer that brings him in full-time for a week or two at a time, every month two - or something like that.

2) You could specialize in unit test evangelism and training at a Full Time Employee. Google has had jobs of that flavor that pop up now and again. Microsoft has an engineering excellence group and, within that, a test engineering group. Search for "Alan Page." (Hey, here's a list of who some of the testing luminaries at Microsoft are are, and what they do: http://msdn.microsoft.com/en-us/testing/bb880947.aspx )

3) You could work as a full-time tester with an environment that /feels/ like a unit test - say testing a REST API, or a compiler, or something with business rules you can reach out and touch "behind the GUI" - tax preparation software, for example.

4) You could become a consultant and do unit-test training, or teach it in academia. The Florida Institute of Technology, for example, has TDD-focused courses.

5) Finally, you /might/ be able to find a shop that does not practice TDD, where unit test is split out from development. For example: Embedded Medical Devices or Avionics may be regulated in such a way that this seems like a good idea to an auditor. I have not worked in those environments; I would be skeptical.

So, in conclusion, while it might be hard to find a unit-test-writing job, you could certainly find a niche where you are happy doing more unit-testing than a pure developer and training others to do it.