Unit testing several different results in one method

Question

I am using NUnit 2.5, and I want to test a method that recieves several parameters.
Is it reasonable to try to test them using one Test method, with several different TestCases, or will it be harder to maintain (Or just pin point the exact failure) in the long run?

My general idea is to try something like

[TestCase("valid", Result="validasd", 
  Description = "Valid string test, expecting valid answer")]
[TestCase(null, ExpectedException = typeof(ArgumentNullException),
  Description = "Calling with null, expecting exception")]
[TestCase("", ExpectedException = typeof(ArgumentException), 
  Description = "Calling with an empty string, expecting exception")]
public string TestSubString(string value)
{
  return MethodUnderTest(value);
}

Is it also a recommended usage - without an explicit assert, just checking on the return value? Or should I stick to the normal way or asserting the value inside the method?

Answer 1

I would avoid that. It would be extremely hard to maintain. I would have one test method for each case.

Answer 2

I think the jury is still out on this kind of parametric testing - hard-core TDDers will likely say that it violates the "one assert per test" principle, and they are probably right. The most important thing in unit testing is that when a test fails, you can quickly determine exactly what failed, what inputs caused it to fail, and reproduce it by re-running the same test. While the first two might not be a problem with this kind of testing structure, the third probably will be (although with a smart enough tool you might be able to get around that).

Personally I would say that at minimum you should split up the test cases into result sets, i.e. if two sets of parameters have the same expected result (such as ArgumentException), it is ok to put them together on the same test method. But in your example, all three parameters have different expected results, so you should put them on different test methods.

Answer 3

To me, the best way is often the one which makes you write the less code. The Attribute way is better on my opinion because:

• the test runner will group those methods under the same tree node
• visually, it's obvious that those 3 test cases apply to the same code, only the method parameters are changing, and if the test fails the error message will still tell you which parameters made it fail
• less code!

The disadvantage is that the parameters and return value are hard-coded, so you can't test against dynamic or randomly generated parameters, which is in some cases useful.

Answer 4

I have found the TestCase attribute to be useful when testing methods that primarily accept some simple values, compute with them, and then return a result. In these cases, it takes substantially less overhead and duplication to write such tests.

This approach does not work when the method requires complex object parameters, because the TestCase attribute can only accept primitive types as input parameters to pass to the method.

You may also want to consider writing a more typical unit test if:

1. other developers on your team are not familiar with this technique
2. the signature of the method likely to change or accommodate more parameters
3. if there are more than a handful of test cases or combinations of values to test
4. you need the tests to execute in a particular order
5. you want to run your tests using ReSharper for Visual Studio (it currently does not recognize the TestCase attribute)

Answer 5

Is not it better to write something like this?

[Test]
public string TestSubString()
{
    Assert.AreEqual("validasd", MethodUnderTest("valid"));
    AssertEx.Throws<ArgumentNullException>(() => { MethodUnderTest(null); });
    AssertEx.Throws<ArgumentException>(() => { MethodUnderTest(""); });
}