The value of high level unit tests and mock objects

Question

I am beginning to believe that unit testing high level, well-written code, which requires extensive use of mock objects, has little to no value. I am wondering if this assertion is correct, or am I missing something?

What do I mean by high level? These are the classes and functions near the top of the food chain. Their input and output tends to be user input, and user interface. Most of their work consists of taking user input and making a series of calls to lower-level entities. They often have little or no meaningful return values.

What do I mean by well-written? In this case, I am referring to code that is decoupled from its dependencies (using interfaces and dependency injection), and line by line is at a consistent level of abstraction. There's no tricky algorithms, and few conditionals.

I hate writing unit tests for this kind of code. The unit tests consist almost entirely of mock object setup. Line by line, the unit test read almost like a mirror image of the implementation. In fact, I write the unit tests by looking at the implementation. "First I assert this mock method is called, then I assert this mock method is called...", etc. I should be testing the behavior of the method, not that it's calling the right sequence of methods. Another thing: I have found that these tests are extremely fragile to refactoring. If a test is so brittle it utterly shatters and must be rewritten when the code under test is refactored, then hasn't one of the major benefits of unit testing been lost?

I don't want this post to be flagged as argumentative, or not a question. So I'll state my question directly: What is the correct way to unit test the kind of code I have described, or is it understood that not everything needs a unit test?

Answer 1

you are correct. not everything needs a unit test. :)

it's generally accepted that this type of code does not benefit from unit testing.

Answer 2

In my experience, the lower level your code is (short of being trivial), the more value unit tests are, relative to the effort required to write them. As you get higher up the food chain, tests become increasingly elaborate and more expensive.

Unit tests are critical because they tell you when you break something during refactoring.

Higher level tests have their own value, but then they are no longer called unit tests; they are called integration tests and acceptance tests. Integration tests are needed because they tell you how well the different software components work together.

Acceptance tests are what the customer signs off. Acceptance tests are typically written by other people (not the programmer) in order to provide a different perspective; programmers tend to write tests for what works, testers try to break it by testing what doesn't work.

Mocking is only useful for unit tests. For integration and acceptance tests, mocking is useless because it doesn't exercise the actual system components, such as the database and the communication infrastructure.

Answer 3

An aside

Just to touch on your bolded statement:

"I should be testing the behavior of the method, not that it's calling the right sequence of methods"

The behaviour of the object-under-test is the sequence of actions it takes. This is actually "behaviour" testing, whereas when you say "behaviour of the method", I think you mean stateful testing, as in, give it an input and verify the correct output.

I make this distinction because some BDD purists go so far as to argue that it is much more meaningful to test what your class should be calling on, rather than what the inputs and outputs are, because if you know fully how your system is behaving, then your inputs and outputs will be correct.

A response

That aside, I personally never write comprehensive tests for the UI layer. If you are using an MVVM, MVP or MVC pattern for your application, then at a "1-developer team" level, it's mind-numbing and counter-productive for me to do so. I can see the bugs in the UI, and yes, mocking behaviours at this level tends to be brittle. I'm much more concerned with making sure that my underlying domain and DAL layers are performing properly.

What is of value at the top level is an integration test. Got a web app? Instead of asserting that your controller methods are returning an ActionResult (test of little value), write an integration test that requests all the pages in your app and makes sure there are no 404's or 403's. Run it once on every deployment.

Final answer

I always follow the 80/20 rule with unit testing. To get that last 20% coverage at the high level you are talking about, is going to be 80% of your effort. For my personal and most of my work projects, this doesn't pay off.

In short, I agree. I would write integration tests, and ignore unit tests for the code you describe.

Answer 4

If doing TDD you should not write tests after implementation but rather the other way around. This way you'll also avoid the problem of making a test conform to the written code. You probably do have to test certain method calls within those units, but not their sequence (if it's not imperative to the domain problem - business process).

And sometimes it's perfectly feasible not to write a test for a certain method.

Answer 5

I think it is highly dependent on environment. If you are on relatively small team, and can maintain test integrity, then the more complex parts of your application should have unit tests. It is my experience that maintaining test integrity on large teams is quite difficult, as the tests are initially ok until they inevitably break...at which point they are either a) "fixed" in a way which completely negates their usefulness, or b) promptly commented out.

The main point of Mock testing seems to be so that managers can claim that the code-coverage metric is at Foo%....so everything must be working! The one exceptional case where they are possibly useful is when you need to test a class which is a huge pain to recreate authentically(testing an Action Class in Struts, for example).

I am a big believer in writing raw tests. Real code, with real objects. The code inside of methods will change over time, but the purpose and therefore the overall behaviour usually does not.

Answer 6

Well, you definitely have a code smell here. Are you using DRY enough? Do you have adequate UI abstractions?

Answer 7

In general, I consider testing this type of method/command to be ripe for the integration testing level. Specifically, I "unit test" for smaller, low level commands that (generally) don't have side effects. If I really want to unit test something that doesn't fit that model, the first thing I do is see if I can refactor/redesign to make it fit.

At the higher, integration (and/or system) testing level, I get into the testing of things that have side effects. I try to mock as little as possible (possibly only external resources) at this point. An example would be mocking the database layer to:

1. Record how it was called to get data
2. Return canned data Record how it was
3. Record how it was called to insert manipulated data