More bugs in unit tests than in production code

Question

I just started a new project, and now with ASP.NET MVC being designed in an extremely composable way, I figured it might be a good time to start with unit testing. Most of my code is fresh, and I'm writing the tests before I'm writing the actual production code.

My frustration, though, is that I spend a lot more time fixing mistakes in my tests than in fixing anything wrong with my production tests.

My typical workflow ends up being something like this:

1. Write a stub
2. Write the test
3. Ensure that the test fails
4. Fill in the stub
5. The test still fails, so spend a while going over the expected and actual output.
6. The error turns out to be in the test, not the actual code. Fix the test.

If you think about it, this is somewhat to be expected: unit tests involve producing output by hand and so is error-prone; code written in a strict language and with good coding practices has behavior that is specified very much automatically.

Of course, there are odd times when my production code is the actual cause of a test failing, but it's just really comparatively rare.

There's no reason to eliminate unit tests entirely, of course; there are times when I simply don't trust my own code at all. On the other hand, I'm starting to feel like it's not all that valuable -- especially the test-first philosophy.

Anyone else feel this way?

Answer 1

I feel you there, here are a couple of things I have found useful when writing unit tests to make them more solid:

• If you haven't already, try breaking down your unit tests to one assertion per test. You may have to do many assertions per unit of code that your testing if the methods are massive (this may mean your methods should be broken down too). But this will loosen up your unit tests and make them less prone to time consuming maintenance. Theres a good pattern to put into practice called Arrange, Act Assert (AAA)
• Make sure your unit tests are completly isolated, i.e., there are no interactions or calls outside your code i.e., databases, web services etc.
• Use available frameworks to make your unit testing experience easier and to do most of the work for you, e.g., Nbuilder to hack up lists of objects and Moq to mock out objects you need for testing.
• If you don't already, use test fixtures so you dont have to set everything up for each test.

TDD and unit testing is definitly something that starts off a real pain in the butt, but its one of those things that gets much easier and faster to do the more you keep at it.

Richard Dingwall's blog has quite a few articles on best practices for unit testing and TDD, many actually regarding unit testing with MVC.

Answer 2

First thing, TDD takes time to get adapted to.

We as a team started with TDD a year ago. Initially, we started with first writing the code(class/a single module) and then writing tests covering that code.

We then progressed to writing tests in parallel with the code(write a method/group of methods and then write test cases coverring those methods).During this period, we used coverage tools to check our coverage.

Now after doing this for a year, the team is well versed with test-first approach. @Scozzard has mentioned valid points about using AAA / mocking frameworks.

Answer 3

Forget about TDD the new fad is TDTDD! Just write tests for you unit tests. Next year we will do TDTDTDD.

The simplest solution is to know what you are doing. You don't need tests then.

Answer 4

If your tests are comparing actual results to hand written expected values (which are occasionally entered wrong) you might be able to get a bit of mileage just by refactoring your tests. Instead of verifying that method X returns an instance that is identical to the test instance, verify that the instance returned by the method has the same important property as the test instance.