When is it OK to change "completed" unit tests?

Question

I've taken my first baby steps into Unit Testing and, due to a better understanding of the domain, have made a change to a domain model that has broken a unit test. So this brought up the question:

When is it permissible to change previously working unit tests?

I figure I'm missing an important aspect of unit testing in having to ask this question...

Answer 1

When you make a change that breaks tests:

1) First find out if the test is now broken or whether your change has broken a test that it shouldn't have.

2). If it's the former, fix the test. Else fix your change.

Answer 2

For 'proper' TDD you change the test first, then change the code.

So actually you never have broken tests, only broken code. You should always strive to be in a position where the tests are the definitive expression of correct functionality, and as such are, a priori, correct.

Answer 3

The point of each particular unit test is not that it never breaks, but that it continues to work as long as the feature it tests for also works. That way, an accidental change to a tested feature produces a failed test you find out and not your end user.

If the feature is changed on purpose, then you should expect to see some tests break. If you don't, then you didn't have enough coverage of the feature in the test suite.

Answer 4

When requirements change. Regardless of whether you change your code, and then see which tests break for what reason (as Mitch suggested), or change your tests and then change your code (as Visage alluded to), the tests will only change if the functionality is supposed to do something different.

Answer 5

Every time you need to.

• Tests should be changed if they become inaccurate, or don't follow specs anymore.
• Tests should be removed if they have become irrelevant.

The point is, your tests should be an accurate picture of how your software is supposed to work. Granted this will be a bit difficult to maintain, and indeed this is one of the bigger turn-offs from TDD, second only to tests-first programming.

Answer 6

Tests are the specification of the program's behaviour. You change them when you need to change the specifications, because they are the specifications. Some that come to my mind...

• When the requirements for the code have changed and the test does not match the new behaviour, you should update them.
• When some requirements become obsolete, the tests that specify that behaviour should be removed.

The primary quality that test code needs to have, is readability. Therefore you should change the tests regularly because of...

• If a test is hard to read and does not reveal every intention of the programmer, it should be refactored to improve its readability.

Then there are also cases where the tests are broken, for example brittle tests for concurrent code that mostly pass but every now and then fail, even though the code is correct. In such cases the tests should be fixed to be more repeatable (it might require changing the code to be more easily testable - the best thing is to avoid/restrict concurrency with suitable design patterns).