Mocks... and Verifiers?

Question

Hello, currently, I am looking deeper into testing techniques, even though I am not sure if I still reside in the unittest-land or left it into the land of integration tests already.

Let me elaborate a bit, Given two components A and B and A uses B, then we have a certain "upwards-contract" for B and a certain "downwards-contract" for A. Basically this means: If A uses B correctly and B behaves correctly, then both contracts will be fulfilled and things will work correctly.

I think mocks are a way to guarantee a subset of an upwards-contract that is required for a given testcase. For example, a database connection might have the upwards contract to retrieve data records if they have been inserted earlier. A database connection mock guarantees to return certain records, without requiring their insertion into the database.

However, I am currently wondering if there is a way to verify the downwards-contract as well. Given the example of the database connection, the downwards-contract might be: You must connect to the database and ensure the connection exists and works and enter correct SQL-Queries.

Does anyone do something like this? Is this worth the work for more complicated contracts? (For example, the database connection might require an SQL-parser in order to completely verify calls to the database layer)

Greetings, tetha

Answer 1

Does anyone do this?

Yes, I sometimes use mocks to verify "downward-contracts".

E.g. You can use the DB mock to check, if the correct credentials where used for the login. Especially, if You have interfaces to other subsystems, then You can mock them up and let the mockup check for usage violations.

E.g. if a subsystem requires an initialization call or some kind of registration, then Your mock-up of the subsystem interface could enforce this, too.

---

Is it worth the work?

It depends on how deep You want Your tests to be, let me give You some examples of different "deepness":

1. If You want to check the proper sequence of calls to an interface, then a simple state machine may be sufficient.
2. If You want to verify the proper usage of an interface language (SQL in Your example), You have to use a parser.
3. If You want to verify that it actually works with the real subsystem, then make an integration test (cannot be done with mockups).

Conclusion:

If appropriate, it should be done. However, You cannot expect a mockup to find each and every wrong use of the interface. E.g. a database mockup hardly detects, if there will be a deadlock caused by two concurrent transactions.

Answer 2

This is really the difference between mocks and stubs - mocks verify exactly that (or at least can do so - you can use mocks as stubs with most frameworks). Essentially, mocks allows you to do protocol testing rather than just "if you call X I'll give you Y". Every mocking framework I've used allows you to easily verify things like "all these calls were made" and "these calls happened in a particular order".

The more protocol you enforce between components, the more brittle the tests will be - sometimes that's entirely appropriate (e.g. "you must authenticate before you perform any other operations") but it's easy to end up with tests which have to be changed every time you change the implementation, even in a reasonable way.