Designing a robust unit test - testing the same logic in several different ways?

Question

Hi,

In unit test design, it is very easy to fall into the trap of actually just calling your implementation logic.

For example, if testing an array of ints which should all be two higher than the other (2, 4, 6, 8, etc), is it really enough to get the return value from the method and assert that this pattern is the case?

Am I missing something? It does seem like a single unit test method needs to be made more robust by testing the same expectation in several ways. So the above expectation can be asserted by checking the increase of two is happening but also the next number is divisible by 2. Or is this just redundant logic?

So in short, should a unit test test the one expectation in several ways? For example, if I wanted to test that my trousers fit me, I would/could measure the length, put it next to my leg and see the comparison, etc. Is this the sort of logic needed for unit testing?

Thanks

Answer 1

If you assert that your array contains 2,4,6,8 -- then your testing logic might be flawed because your test would pass if you just returned an array with those elements, but not with, say, 6,8,10,12. You need to test that calculation is correct. So you need to test it with multiple arrays, in this particular case.

I find that making sure the test fails, then making the test pass, in the true spirit of TDD, helps flush out what the correct test is...

Answer 2

Your unit tests should check all of your assumptions. Whether you do that in 1 test or multiple tests is a personal preference.

In the example you stated above, you had two different assumptions: (1) Each value should increment by 2. (2) All values should be even.

Should (-8,-6,-4,-2) pass/fail?

Remember, ensuring your code fails when it's supposed to is just as important, if not more important, then making sure it passes when it's supposed to.

Answer 3

The array you are testing, must be generated in some sort of logic. Isn't it better to test this logic to ensure that the resulting array always meets your requirements?

Answer 4

For example, if testing an array of ints which should all be two higher than the other (2, 4, 6, 8, etc), is it really enough to get the return value from the method and assert that this pattern is the case?

Perhaps you need to think a little more about how the function would be used. Will it be use with very large numbers? If so, the you may want to try some tests with very large numbers. Will it be used with negative numbers?

Am I missing something? It does seem like a single unit test method needs to be made more robust by testing the same expectation in several ways. So the above expectation can be asserted by checking the increase of two is happening but also the next number is divisible by 2. Or is this just redundant logic?redundant logic?

Hmm... well 1,3,5,9 would pass the assertEachValueIncrementsByTwo test, but it would not pass the assertValuesDivisibleByTwo test. Does it matter that they are divisible by 2? If so, then you really should test that. If not, then it's a pointless redundant test.

You should try to find more than 1 test for your methods, but redundant tests for the sake of more testing is not going to help you. Adding the assertValuesDivisibleByTwo test when that is not really required will just confuse later developers who are trying to modify your code.

If you can't think of any more tests, try writing a random input function that will generate 100 random test arrays each time you run your tests. You'd be surprised how many bugs escape under the radar when you only check one or two input sets.

Answer 5

I'd recommend multiple tests. If you ever need to change the behaviour you'd like to have as few tests to change as possible. This also makes it easier to find what the problem is. If your really blow the implementation and get [1,3,4,5] your one test will fail, but you'll only get one failure for the first thing you test when there are actually two different problems.

Try naming your tests. If you can't say in one clear method name what you're testing break up the test.

testEntriesStepByTwo

testEntriesAllEven

Also don't forget the edge cases. The empty list will likely pass the 'each entry is 2 more than the previous' one and 'all entries are even' tests, but should it?