unit tests in C++

Question

Is okay to break all dependencies using interfaces just to make a class testable? It involves significant overhead at runtime because of many virtual calls instead of plain method invocations.

How does test driven development work in real world C++ applications? I read Working Effectively With Legacy Code and fond it quite useful but do not get up to speed practising TDD.

If I do a refactoring it occurs very often that I have to completely rewite unit test because of massive logic changes. My code changes very often change the fundamental logic of the processing of data. I do not see a way to write unit tests which do not have to change in a large refactoring.

May be someone can point me to an open source c++ application wich is using TDD to learn by example.

Answer 1

I've routinely used macros, #if and other preprocessor tricks to "mock-out" dependencies for the purpose of unit testing in C and C++, exactly because with such macros I don't have to pay any run-time cost when the code is compiled for production rather than testing. Not elegant, but reasonably effective.

As for refactorings, they may well require changing the tests when they are so overwhelmingly large and intrustive as you describe. I don't find myself refactoring so drastically all that often, though.

Answer 2

The obvious answer would be to factor out dependencies using templates rather than interfaces. Of course that might hurt compile-times (depending on exactly how you implement it), but it should eliminate the runtime overhead at least. A slightly simpler solution might be to just rely on a set of typedefs, which can be swapped out with a few macros or similar.

Answer 3

As to your first question - it's rarely worthwhile to break things just for the sake of testing, although sometimes you might have to break things before making them better as part of your refactoring. The most important criteria for a software product is that it works, not that it's testable. Testable is only important as it helps you make a product that is more stable and functions better for your end-users.

A big part of test-driven development is selecting small, atomic parts of your code that aren't likely to change for unit testing. If you're having to rewrite a lot of unit tests because of massive logic changes, you might need to test at a finer-grained level, or re-design your code so that it's more stable. A stable design shouldn't change drastically over time, and testing won't help you avoid massive refactoring is that becomes required. However, if done right testing can make it so that when you refactor things you can be more confident that your refactoring was successful, assuming that there are some tests that don't need to be changed.

Answer 4

Update: See this question too.

I can only answer some parts here:

Is okay to break all dependencies using interfaces just to make a class testable? It involves significant overhead at runtime because of many virtual calls instead of plain method invocations.

If your performance will suffer too much because of it, no (benchmark!). If your development suffers too much, no (estimate extra effort). If it seems like it won't matter much and help in the long run and helps you with quality, yes.

You could always 'friend' your test classes, or a TestAccessor object through which your tests could investigate stuff within it. That avoids making everything dynamically dispatchable just for testing. (it does sound like quite a bit of work.)

Designing testable interfaces isn't easy. Sometimes you have to add some extra methods that access the innards just for testing. It makes you cringe a bit but it's good to have and more often than not those functions are useful in the actual application as well, sooner or later.

If I do a refactoring it occurs very often that I have to completely rewite unit test because of massive logic changes. My code changes very often change the fundamental logic of the processing of data. I do not see a way to write unit tests which do not have to change in a large refactoring.

Large refactorings by defintion change a lot, including the tests. Be happy you have them as they will test stuff after refactoring too.

If you spend more time refactoring than making new features, perhaps you should consider thinking a bit more before coding to find better interfaces that can withstand more changes. Also, writing unit-tests before interfaces are stable is a pain, no matter what you do.

The more code you have against an interface that changes much, the more code you will have to change each time. I think your problem lies there. I've managed to have sufficiently stable interfaces in most places, and refactor only parts now and then.

Hope it helps.

Answer 5

Is okay to break all dependencies using interfaces just to make a class testable? It involves significant overhead at runtime because of many virtual calls instead of plain method invocations.

I think it is OK to break the dependencies, since that will lead into better interfaces.

If I do a refactoring it occurs very often that I have to completely rewite unit test because of massive logic changes. My code changes very often change the fundamental logic of the processing of data. I do not see a way to write unit tests which do not have to change in a large refactoring.

You won't get ride of these large refactorings in any language since your tests should be expressing the real intent of your code. So if the logic changes, your tests must change.

Maybe you aren't really doing TDD, like:

1. Create a test that fails
2. Create the code to pass the test
3. Create another test that fails
4. Fix the code to pass both tests
5. Rinse and repeat until you think you have enough tests that show what your code should be doing

These steps say that you should be doing minor changes, and not big ones. If you stay with the latter, you can't escape big refactors. No language will save you from that, and C++ will be the worst of them because of the compile times, link times, bad error messages, etc. etc.

I'm actually working in a real world software written in C++ with a HUGE legacy code under it. We're using TDD and it is really helping evolve the design of the software.

Answer 6

If I do a refactoring it occurs very often that I have to completely rewrite unit test because of massive logic changes. ... I do not see a way to write unit tests which do not have to change in a large refactoring.

There are multiple layers of testing, and some of those layers won't break even after big logic changes. Unit test, on the other hand, are meant to test the internals of methods and objects, and will need to change more often than that. There's nothing wrong, necessarily. It's just how things are.

Is [it] okay to break all dependencies using interfaces just to make a class testable?

It's definitely OK to design classes to be more testable. That's part of the purpose of TDD, after all.

It involves significant overhead at runtime because of many virtual calls instead of plain method invocations.

Just about every company has some list of rules that all employees are supposed to follow. The clueless companies simply list every good quality they can think of ("our employees are efficient, responsible, ethical, and never cut corners"). More intelligent companies actually rank their priorities. If somebody comes up with an unethical way to be efficient, does the company do it? The best companies not only print up brochures saying how the priorities are ranked, but they also make sure management follows the ranking.

It is entirely possible for a program to be efficient and easily testable. However there are times when you need to choose which is more important. This is one of those times. I don't know how important efficiency is to you and your program, but you do. So "would you rather have a slow, well-tested program, or a fast program without total test coverage?"

Answer 7

It involves significant overhead at runtime because of many virtual calls instead of plain method invocations.

Remember that it is only a virtual call overhead if you access the method through a pointer (or ref.) to you interface or object. If you access the method through a concrete object in the stack, it will not have the virtual overhead, and it can even be inlined.

Also, never assume that this overhead is big before profiling your code. Almost always, a virtual call is worthless if you compare to what your method is doing. (Most of the penalty comes from not being possible to inline a one-line method, not from the extra indirection of the call).