What are "bad" attributes in unit tests?

Question

Hi,

I just read question that answered what were desirable features of unit tests, but what should be avoided? What makes a unit test "bad"?

What are the worst unit tests you've seen? (For example. I remember a developer telling me he once found a test suite with plenty of methods but completely devoid of any asserts).

I'm particularly interested in slightly more subtle and specific problems with unit tests e.g. suppose you have a test-suite that runs quickly with good coverage, what problems could it still have?

Answer 1

The best unit tests were simple to read and understand. Quick to execute. Tested specific functionality, well refactored and were maintained.

The worst were not the above.

Answer 2

Tests that are brittle usually have an unacceptable maintenance overhead which will evenutally lead to the tests not being updated, remaining in a broken state, and not being run since they are out of synch with the source code.

Brittle tests usually have dependencies on file system, registry keys, databases etc... These are fine with Integration and System tests but sometimes I see tests masquerading (spelling?) as unit tests with these properties and that's usually a problem.

Answer 3

In the spirit of CW, I knew this would come in handy one day.

Also you did ask for a specific one :) See the MAGIC block below

@Test
      public void testCheckForDuplicateCustomer() {
            //List<CustomerSearch> customerInfo = null;
            String customerName = null;
            boolean status = false;
            try {
                  status = custSearchService.checkForDuplicateCustomer(customerName);

/*************/ MAGIC BEGINS HERE
                      if(status){
                            assertEquals(true, status);
                      } else {
                            assertEquals(false, status);
                      }
                      /**************/ MAGIC ENDS HERE
                } catch (Exception e) {
                      //fail();
                }
          }

Answer 4

• Tests with external dependencies (DB, file, server, time...)

• Tests that depend on each other

• Tests that verify the implementation rather than the behavior

• Tests that are so slow that no one execute them

• Tests that test too much things

There is also the TDD anti-patterns.

Answer 5

For me readability is King when it comes to unit tests. If I can't read and understand the test in 2 seconds, there's probably something wrong. Any test that's more than 5 lines long had better have a pretty good excuse.

Sometimes people take the refactoring too far and I have to look at various helper classes or parent classes to find out what exactly is being tested. Always keep readability in mind when refactoring tests. sometimes it's better to leave a bit of duplication in there if it means the test is clearer.