Unit testing: Is it a good practice to have assertions in setup methods?

Question

In unit testing, the setup method is used to create the objects needed for testing.

In those setup methods, I like using assertions: I know what values I want to see in those objects, and I like to document that knowledge via an assertion.

In a recent post on unit tests calling other unit tests here on stackoverflow, the general feeling seems to be that unit tests should not call other tests: The answer to that question seems to be that you should refactor your setup, so that test cases do not depend on each other.

But there isn't much difference in a "setup-with-asserts" and a unit test calling other unit tests.

Hence my question: Is it good practice to have assertions in setup methods?

EDIT:

The answer turns out to be: this is not a good practice in general. If the setup results need to be tested, it is recommended to add a separate test method with the assertions (the answer I ticked); for documenting intent, consider using Java asserts.

Answer 1

They're different scenarios; I don't see the similarity.

Setup methods should contain code that is common to (ideally) all tests in a fixture. As such, there's nothing inherently wrong with putting asserts in a test setup method if certain things must be true before the rest of the test code executes. The setup is an extension of the test; it is part of the test as a whole. If the assert trips, people will discover which pre-requisite failed.

On the other hand, if the setup is complicated enough that you feel the need to assert it is correct, it may be a warning sign. Furthermore, if all tests do not require the setup's full output, then it is a sign that the fixture has poor cohesion and should be split up based on scenarios and/or refactored.

It's partly because of this that I tend to stay away from using Setup methods. Where possible, I use private factory methods or similar to set things up. It makes the test more readable and avoids confusion. Sometimes this is not practical (e.g. working with tightly coupled classes and/or when writing integration tests), but for the majority of my tests it does the job.

Answer 2

Having assertions in the Setup/TearDown methods is not advisable. It makes the test less readable if the user needs to "understand" that some of the test logic is not in the test method. There are times when you do not have a choice but to use the setup/teardown methods for something other than what they where intended for.

There is a bigger issue in this question: a test that calls another test, it is a smell for some problem in your tests. Each test should test a specific aspect of your code and should only have one or two assertions in it, so if your test calls another test you might be testing too many things in that test. For more information read: Unit Testing: One Test, One Assertion - Why It Works

Answer 3

Follow your heart / Blink decisions. Asserts within a Setup method can document intent ; improver readability. So personally I'd back you up on this.
It is different from a test calling other tests - which is bad. No test isolation. A test should not influence the outcome of another test.

Although it is not a freq use-case, I sometimes use Asserts inside a Setup method so that I can know if test setup has not taken place as I intended it to; usually when I'm dealing with components that I didn't write myself. An Assertion failure which reads 'Setup failed!' in the errors tab - quickly helps me zone in on the setup code instead of having to look at a bunch of failed tests.

A Setup failure usually should cause all tests in that fixture to fail - which is a smell that your nose should soon pickup. 'All tests failed usually implies Setup broke ' So assertions are not always needed. That said be pragmatic, look at your specific context and 'Add to taste.'

Answer 4

Instead of assertions in the setup to check the result, I used a simple test (a test method along the others, but positionned as first test method).

I have seen several advantages:

• The setup keeps short and focused, for readability.
• The assertions are run only once, which is more efficient.

---

Usage and discussion :

For example, I name the method testSetup().

To use it, when I have some test errors in that class, I know that if testSetup() has an error, I don't need to bother with the other errors, I need to fix this one first.

If someone is bothered by this, and wants to make this dependency explicit, the testSetup() could be called in the setup() method. But I don't think it matters. My point is that, in JUnit, you can already have something similar in the rest of your tests:

1. some tests that test local code,
2. and some tests that is calls more global code, which indirectly calls the same code as the previous test.

When you read the test result where both fail, you already have to take care of this dependency that is not in the test, but in the code being called. You have to fix the simple test first, and then rerun the global test to see if it still fails. This is the reason why I'm not bothered by the implicit dependency I explained before.

Answer 5

I use Java asserts, rather than JUnit ones, in the cases where something like this is necessary. e.g. when you use some other utility class to set up test data.:

byte[] pkt = pktFactory.makePacket(TIME, 12, "23, F2");
assert pkt.length == 15;

Failing has the implication 'system is not in a state to even try to run this test'.