How do I create flexible unit tests?

Question

We are currently using unit tests to test our project. We have the majority of functionality covered but I think our tests are too brittle.

I was wondering if there are any specific things we can be doing to make the unit tests more flexible so they don't break for the wrong reasons.

A couple answers have mentioned being careful of mocking too much... So what are legitimate reasons for mocking? I think that may be one of our main problems, but when your application is mostly a dynamic, database-driven site, how do you get away from mocking?

Answer 1

This is a somewhat simplistic answer, but shows the right mindset:

• A test should break if the behaviour changes in a way that you care about.
• A test should continue to work if the behaviour changes in a way that you don't care about.

So as far as is possible - without going hugely out of your way - make sure you're testing the "end result" of the method without caring how it got there. One thing to watch out for is mocking - it's incredibly useful, but can easily make your tests brittle.

Answer 2

+1 to Jon. Well put.

I've found a lot of value in structuring my tests in a more BDD style. That is... reject the fixture-per-class mindset, instead go for fixture-per-context.

I also found that RhinoMocks 3.5's AAA syntax is much nicer.

Those cover organization and clean/readable tests.

To make my tests less brittle I've started to pull back on mocking. Mock frameworks are crucial for stubbing out dependencies, but the more you mock the more the test knows about the implementation. If the implementation changes (but behavior doesn't) then your tests shouldn't break.

Answer 3

Also +1 to Jon.

In typical engineering fashion, the answer is always "it depends".

I'd suggest taking a look at the book "xUnit Test Patterns: Refactoring Test Code". (In this context, x={J,N} to cover both the Java and .NET worlds and isn't explicitly intendend for the new actually-called-xUnit framework.)

Just as design patterns have emerged in the OO world, so have patterns emerged in the TDD world. It's worth a look.

Answer 4

I found that when my tests have the following attributes they tend to be more brittle

1) Complex to set up the correct state in order to the test the actual logic. 2) Many expectation on mocks. 3) Poor readability of test code. 4) General poor system design.

To tackle these issues we try to do the following

1) Change the system design in order to ease the setup of tests, usually by applying the SRP and looking for responsibility leaks in our class.

2) using mocks without explicit expectations regarding the number or order of calls performed on the mock.

3) Treating test code as production code, performing code, design reviews etc.

Answer 5

So what are legitimate reasons for mocking? I think that may be one of our main problems, but when your application is mostly a dynamic, database-driven site, how do you get away from mocking?

Reasons for mocking an object comprise

• object is or uses an external resource such as database, network, file system
• object is a GUI
• object is not [yet] available
• object behavior is not deterministic
• object is expensive to setup