How did you adapt your unit tests to deal with changing requirements?

Question

I have a project where I've been using TDD and unit tests as "software vises". In essence I translate the requirements into tests that verify that the code conforms to the requirements. I rarely have to go back and edit the unit tests, which rather is the point: only the "real" code should be modified. At the moment, there are 900 unit tests.

Now some requirements have been changed by the gold-owners. Since the former requirements are so thorougly encoded in the existing unit tests, it seems that changing them to conform to the new requirements would be inviting disaster. How do you adapt your unit test suites to handle this kind of change?

Answer 1

Per definition the unit-tests don't replicate the requirements for the application. They describe the requirements for a module. That's a difference, the module can be reused even in an application with different requirements or isn't used at all. So the changing requirements don't affect real unit-tests (except that you have to write new for new modules or abandon old tests for modules no longer needed for the changed requirements).

On the other hand: acceptance-tests deal with the requirements on application-level. So I think you talk about acceptance-tests.

I would add the new requirements as new acceptance-test. But for the old ones you have to look through them, how they are invalidated by the changed requirements.

Answer 2

I would add the new tests and make them pass. Then you look at what tests have been broken as a result. If you believe the old tests are in contradiction to the new tests then you may have to remove the old tests. Otherwise, you alter the code to make the old tests pass as well.

Answer 3

In essence I translate the requirements into tests that verify that the code conforms to the requirements

While I agree with Mnementh's answer, this, to me, is the key comment. If the tests are a translated version of the requirements, then if the requirements have changed, the tests must change.

Or they're testing for something that doesn't meet the requirements of the customer.

As John Maynard Keynes is reported to have said, "When the facts change, I change my opinion. What do you do, sir?"

I think there's an analagous situation here. Your facts have been changed for you

Answer 4

I have two answers to your question, one philosophical and the other tactical.

On the philosophical front it is important to consider your unit tests as code. That means all the normal traits of good code are usually appropriate for good tests: intention revealing, remove duplication, etc. Many, perhaps most, of the failures that I've seen with unit testing has come because people haven't treated their tests this way, but rather than just coded them and never revisited them to see if they should be refactored.

In my experience, if you've reached a point where your unit tests are a hinderance to change it is because you've got technical debt in your tests.

So my tactical suggestion is that before you attempt to change your requirements you look to refactor you tests. Each test should have a unique reason to pass/fail, and the behavior outside of that should be in shared code. This means that for any given behavior change you'll have two places to change the tests:

1. The test that actually validates that behavior
2. The places that behavior is used in shared fixture code

You might find this article useful: Grow Your Harness Naturally. It was really about a reusable test harness for functional testing but I find the ideas very useful in my unit tests as well.

Answer 5

Since the former requirements are so thorougly encoded in the existing unit tests, it seems that changing them to conform to the new requirements would be inviting disaster.

Any specific reason why you would think so? I sense some fear or is it just 'don't break it when its working'

Change happens. In which case, it means more work-time-money. If the business has no problem with it, neither should you (unless the schedule is inhumane:). If the spec has changed,

• make sure the working version has been checked in.
• Repeat step 1 just to be sure.
• scan your test suite. Find ones you need to take out. Find ones that need to change. Find new tests that you need to understand. Use a blank sheet of paper to take notes
• Now proceed one test at a time.. Unless you have not been following the DRY / Once and only once principle, any changes that you need to make should be in one place. If not, you should have refactored earlier.. but it isn't too late.. extract code into a single place before making the change
• Repeat previous step till done

Answer 6

If your unit tests no longer match requirements then they shouldnt be there - after all - all they now tell you is that your code conforms to requirements that no longer exist!

Whenever you have a change in requirements you should alter the tests that represent the changed requirements and verify that the test now fail (wheras previously they all passed, right? ;))

Then alter your source code so that the rewritten tests now pass.