Should I create a new test method for each assertion?

Question

I know that this is subjective, but I'd like to follow the most common practice. Do you normally create one test method for each class method and stuff it with multiple assertions, or do you create one test method per assertion?

For example, if I am testing a bank account's withdraw method, and I want make sure that an exception is thrown if the user tries to overdraw the account or withdraw a negative amount, should I create testOverdaw and testNegativeWithdrawal, or would I just combine those two assertions in a method called testWithdraw?

Answer 1

Make multiple test methods; do not combine them into one. Each of your test methods should test one behavior of the method. As you are saying, testing with a negative balance is a different behavior then testing with a positive balance. So, that would be two tests.

You want to do it this way so that when a test fails, you are not stuck trying to figure out which part in that test failed.

Answer 2

Personally I would create one test for each assertion otherwise you have to dig to find the reason for the failure rather than it being obvious from the test name.

If you have to write a few lines of code to set up a test and don't want to duplicate that then depending on your language and test framework you should be able to create a set of tests where each test will execute a block of code before it runs.

Answer 3

I would recommend having one test method per assertion, or rather per a expected behavior. This allows to localize the erroneous code much faster, if any test fails.

Answer 4

I would make those two seperate assertions.

The first represents a valid operation that would happen if a user was using the account regularly, the second would represent a case where data sanitizing was not done, or not properly done.

You want separate test cases so that you can logically implement the test cases as needed, especially in regression scenarios where running all tests can be prohibitively expensive.

Answer 5

One way to do it is have one separate method for each different scenario or setup. In your case you'd probably want one scenario where there is sufficient funds, and one scenario where there is insufficient funds. And assert that in the first one everything works, and in the second one the same operations won't work.

Answer 6

Think of it this way: each test should stand on its own and exercise a relatively discrete set of functionality. If you want to assert whether three things are true about some method that you have created, then you should create a test that includes those three things.

Thus, I have to strongly disagree with the others who have answered. Arbitrarily limiting yourself to one assertion per test will do nothing for you except make your testing unwieldy and tedious. Ultimately it may put you off testing altogether - which would certainly be a shame: bad for your project and career.

Now, that does not mean you have license to write large, unwieldy or multi-purpose testing routines. Indeed, I don't think I've ever written one that is more than 20 lines or so.

As far as knowing which assertion fails when there are several in one function, you will note that both nUnit and MSTest give you both a description and a link when an assertion fails that will take you right to the offending line (nUnit will require an integration tool such as TestDriven.net). This makes figuring out the failure point trivial. Both will also stop on the first failure in a function and both give you the ability to do a debug walkthrough as well.

Answer 7

testOverdraw and testNegativeWithdrawal are two separate behaviors. They shall be tested separately.

A good rule of thumb is to have only one action on the method under test in one unit test (not counting setup and assertions).

Answer 8

From the NUnit documentation: "If an assertion fails, the method call does not return and an error is reported. If a test contains multiple assertions, any that follow the one that failed will not be executed. For this reason, it's usually best to try for one assertion per test."

http://www.nunit.org/index.php?p=assertions&r=2.4.6

However, nothing forces you to follow best practices. If it isn't worth your time and effort for this particular project to have 100% granular tests, where one bug means exactly one test failure (and vice-versa), then don't do it. Ultimately it is a tool for you to use at the best cost/benefit balance that you can. That balance depends on a lot of variables that are specific to your scenario, and specific to each project.