When to Expect and When to Stub?

Question

I use NMock2, and I've drafted the following NMock classes to represent some common mock framework concepts:

• Expect: this specifies what a mocked method should return and says that the call must occur or the test fails (when accompanied by a call to VerifyAllExpectationsHaveBeenMet()).

• Stub: this specifies what a mocked method should return but cannot cause a test to fail.

So which should I do when?

I'm starting to understand why behavior verification can be just as important as state verification. That is, I now see why we should do 'white box testing' in addition to 'black-box testing.' If I write automated unit tests for a Presenter in an MVP application, I often need to mock calls to my business layer in order to test the Presenter's methods. For example, I might want to test that the presenter's OnSave method calls the Business Layer's Save method, but only when a certain condition is met. After all, the presenter's job is do exactly that: it calls the business layer, it does not perform the save itself. Therefore, I use Expect to specify what that I want the presenter to call the business layer and conclude my unit test by checking that this was the case using VerifyAllExpectationsHaveBeenMet.

The problem is, now I feel like I just drank the Kool-Aid and can't see any legitimate use case for Stubbing interface calls. Often I would love to Stub these calls and omit the concluding VerifyAllExpectationsHaveBeenMet (especially because this places a tremendous burden on the test writer to account for all external calls) ... but I can't tell implementation details apart from important testable behavior anymore.

On top of that, when refactoring methods tested with Stub calls, you risk orphaning those calls, polluting your unit tests and ensuring that heads are scratched forevermore.

How do you decide when to Stub and when to Expect?

Answer 1

Well... IMHO it can't be simpler: if your test is about ensuring your Presenter will call Save, do an Expect. if your test is about ensuring your Presenter will handle exception gracefully if Save throws up, do a Stub.

For more details, check out this podcast by Hanselman and Osherove (author of The Art Of Unit Testing)

Answer 2

A lot of mocking frameworks are bringing the concepts of mocks & stubs closer & closer together to the point that they can be considered functionally almost the same. From a conceptual perspective however, I usually try to follow this convention:

• Mock: Only when you are explicitly trying to verify the behaviour of the object under test (i.e. your test is saying that this object must call that object).
• Stub: When you are trying to test some functionality/behaviour, but in order to get that working you need to rely on some external objects (i.e. your test is saying that this object must do something, but as a side effect, it may call that object)

This becomes clearer when you make sure that each of your unit tests only test one thing. Sure if you try to test everything in one test then you might as well Expect everything. But by only expecting the things that specific unit test is checking for, your code is much clearer because you can see at a glance what the purpose of the test is.

Another benefit of this is that you'll be slightly more insulated from change & get better error messages when a change causes a break. In other words if you subtley change some part of your implementation, your more likely to get only one test case breaking, which will show you exactly what's broken, rather than a whole suite of tests breaking & just creating noise.

Edit: It might be clearer based on a contrived example where a calculator object audits all additions to a database (in pseudo-code)...

public void CalculateShouldAddTwoNumbersCorrectly() {
    var auditDB = //Get mock object of Audit DB
    //Stub out the audit functionality...
    var calculator = new Calculator(auditDB);
    int result = calculator.Add(1, 2);
    //assert that result is 3
}

public void CalculateShouldAuditAddsToTheDatabase() {
    var auditDB = //Get mock object of Audit DB
    //Expect the audit functionality...
    var calculator = new Calculator(auditDB);
    int result = calculator.Add(1, 2);
    //verify that the audit was performed.
}

So in the first test case we're testing the functionality of the Add method & don't care whether an audit event occurs or not, but we happen to know that the calculator won't work with out an auditDB reference so we just stub it out to give us the minimum of functionality to get our specific test case working. In the second test we're specifically testing that when you do an Add, the audit event happens, so here we use expectations (notice that we don't even care what the result is, since that's not what we're testing).

Yes you could combine the two cases into one, & do expectations and assert that your result is 3, but then you're testing two cases in one unit test. This would make your tests more brittle (since there's a larger surface area of things that could change to break the test) and less clear (since when the merged test fails its not immediately obvious what the problem is.. is the addition not working, or is the audit not working?)