Unit test documentation

Question

I would like to know from those who document unit tests how they are documenting it. I understand that most TDD followers claim "the code speaks" and thus test documentation is not very important because code should be self-descriptive. Fair enough, but I would like to know how to document unit tests, not whether to document them at all.

My experience as a developer tells me that understanding old code (this includes unit tests) is difficult.

So what is important in a test documentation? When is the test method name not descriptive enough so that documentation is justified?

Answer 1

In the test code itself:

• With method level comments explaining what the test is testing / covering.

• At the class level, a comment indicating the actual class being tested (which could actually be inferred from the test class name so that's actually less important than the comments at the method level).

With test coverage reports

• Such as Cobertura. That's also documentation, since it indicates what your tests are covering and what they're not.

Answer 2

When I come back at an old test and don't understand it right away

1. I refactor if possible
2. or write that comment that would have made me understand it right away

When you are writing your testcases it is the same as when you are writing your code, everyhting is crystal clear to you. That makes it difficult to envision what you should write to make the code clearer.

Note that this does not mean I never write any comments. There still are plenty of situations when I just know that I will going to have a hard time figuring out what a particular piece of code does.

I usually start with point 1 in these situations...

Answer 3

Comment complex tests or scenarios if required but favour readable tests in the first place.

On the other hand, I try and make my tests speak for themselves. In other words:

[Test]
public void person_should_say_hello() {
     // Arrange.
     var person = new Person();
     // Act.
     string result = person.SayHello();
     // Assert.
    Assert.AreEqual("Hello", result, "Person did not say hello");
}

If I was to look at this test I'd see it used Person (though it would be in PersonTest.cs as a clue ;)) then that if anything breaks it will occur in the SayHello method. The assert message is useful as well, not only for reading tests but when tests are run it's easier to see them in GUI's.

Following the AAA style of Arrange, Act and Assert makes the test essentially document itself. If this test was more complex, you could add comments above the test function explaining what's going on. As always, you should ensure these are kept up to date.

As a side note, using underscore notation for test names makes them much more readably, compare this to:

public void PersonShouldSayHello()

Which for long method names, can make reading the test more difficult. Though this point is often subjective.

Answer 4

As requested by Thorsten79, I'll elaborate on my comments as an answer. My original comment was:

"The code speaks" is unfortunately completely wrong, because a non-developer cannot read the code, while he can at least partially read and understand generated documentation, and this way he can know what the tests test. This is especially important in cases where the customer fully understands the domain and just can't read code, and gets even more important when the unit tests also test hardware, like in the embedded world, because then you test things that can be seen.

When you're doing unit tests, you have to know whether you're writing them just for you (or for your co-workers), or if you're also writing them for other people. Many times, you should be writing code for your readers, rather than for your convenience.

In mixed hardware/software development like in my company, the customers know what they want. If their field device has to do a reset when receiving a certain bus command, there must be a unit test that sends that command and checks whether the device was reset. We're doing this here right now with NUnit as the unit test framework, and some custom software and hardware that makes sending and receiving commands (and even pressing buttons) possible. It's great, because the only alternative would be to do all that manually.

The customer absolutely wants to know which tests are there, and he even wants to run the tests himself. If the tests are not properly documented, he doesn't know what the test does and can't check if all tests he think he'll need are there, and when running the test, he doesn't know what it will do. Because he can't read the code. He knows the used bus system better than our developers, but they just can't read the code. If a test fails, he does not know why and cannot even say what he thinks the test should do. That's not a good thing.

Having documented the unit tests properly, we have

• code documentation for the developers
• test documentation for the customer, which can be used to prove that the device does what it should do, i.e. what the customer ordered
• the ability to generate the documentation in any format, which can even be passed to other involved parties, like the manufacturer

Properly in this context means: Write clear language that can be understood by non-developers. You can stay technical, but don't write things only you can understand. The latter is of course also important for any other comments and any code.

Independent of our exact situation, I think that's what I would want in unit tests all the time, even if they're pure software. A customer can ignore a unit test he doesn't care about, like basic function tests. But just having the docs there does never hurt.

As I've written in a comment to another answer: In addition, the generated documentation is also a good starting point if you (or your boss, or co-worker, or the testing department) wants to examine which tests are there and what they do, because you can browse it without digging through the code.

Answer 5

Improving the unit tests as executable specification is the point of Behaviour-Driven Development : BDD is an evolution of TDD where unit-tests use an Ubiquitous Language (a language based on the business domain and shared by the developers and the stakeholders) and expressive names (testCannotCreateDuplicateEntry) to describe what the code is supposed to do. Some BDD frameworks pushed the idea very far, and show executable written with almost natural language, for example.