How to avoid false positives using a mockist approach in unit tests?

Question

Since the datastructure of my application domain is becoming pretty complex as of late, I started reading up on mock objects. Soon a simple question came to my mind, but the answer has proven to be quite the headache so far. So here goes:

We have a class 'Foo' with 'bar' as one of its methods:

class Foo {
    public String bar(int i){
        if(i == 1) return "arrr!";
    }
}

And we have a class Pirate calling Foo.bar(1); in one of its methods:

class Pirate {
    public String yell(){
        Foo foo = new Foo();
        return foo.bar(1);
    }

Now we mock the Foo class in the unit test of the Pirate class because Foo happens to have a plethora of other dependencies:

@Test
public void returnsPirateString() {
    Pirate blackBeard = new Pirate();
    Foo fooMock = mock(Foo.class);
    fooMock.expectAndReturn("bar",1,"arrr!"); //expects 'bar' function to be called once and returns "arrr!"
    assertEquals(blackBeard.yell(),"arrr!");
}

What happens now, is that if we refactor method bar to return null instead of "arrr!", our test will keep running happily while our program does not work the way we want it to. This may result in a possible debugging nightmare.

Using a mockist approach instead of the classical testing approach to unit testing, most of the time all "helper" objects get mocked, and only the tested object remains unmocked, so the previously stated problem can occur quite often as well.

What can be done to prevent this problem while mocking?

Answer 1

You should be testing the 'helper object' in isolation as well. Once both of those are covered and tested, then you can be sure that both interact with each other in the expected way.

Changing your 'helper object' is something that should be done with tests against that helper object to confirm it still behaves as expected.

If you are concerned about the specific runtime behavior of the combination of helper and primary class, then you should use integration tests, or some other test at a higher level, to assert the two work together as expected.

Answer 2

In your test, you are testing the yell() method of Pirate class which uses Foo. So you have to mock the behavior of Foo's bar method. To make sure your bar method is functioning correctly, you need another test case to test the bar method of Foo.

@Test
public void testBar() {
   //make sure bar retrun "arrr"!
}

Now if your bar method returns null, this test case will fail!

Answer 3

The test returnsPirateString is not a false positive - it's testing what happens when a Pirate's Foo instance returns 'arrr!'

In other words, when you're testing Pirate.yell, it doesn't matter what Foo.bar returns, unless it creates a special boundary condition (and you should probably already have a test that documents what yell does when Foo returns null).

Pirate.yell is not responsible for guaranteeing any particular return value for Foo.bar, so its unit tests should not expect any particular return values.You should even make a point of changing your test to use something other than the current return value of Foo.bar.