Unit Testing - How to go about it?

Question

I want to unit test my program (in C) because I know of the benefits of doing so as well, as it shows where the problem is.

I also like to blackbox test, since it tells me if the program works (at least, for the tests).

At the moment, I am using Autotest (Which comes with Autoconf) in order to not add a dependency.

At this point, I wouldn't mind so much using a better framework, but the problem is I don't want to use a different framework for blackbox and unit tests. Would it be possible to run the blackbox tests from a unit test framework? The thing I really would like is good log output, saying exactly what went wrong and where.

My other option is to unit test with Autotest. The problem is there is no framework. I have written a small "test driver" that accepts the name of the function to test and arguments to pass to the function, and calls that function. The problem is that I am not sure what boundary to use between assertions and outputting the return value of the function (for logging purposes, since I like how Autotest will give me a diff). Since most functions return lists, it is quicker to prepare using the diff with expected output (expout using Autotest).

Answer 1

Would it be possible to run the blackbox tests from a unit test framework?

Yes, you could invoke the Autotest with system() from the unit tests and then assert on the returned value.

But I wouldn't recommend doing that as unit tests are executed very often, they shall be very fast i.e. measured in seconds, not in minutes.

Unit tests and integration tests (which you call blackbox tests) serve different purposes: unit tests validate that the units in the code (whatever this means, function or clusters of function) works as expected by the tests, while integration tests cover the program end-to-end, validating it as a whole.

So, typically unit tests are ran after every few changes in the code, especially if you apply TDD, while integration tests are executed when a capability has been added.

I would rather have a typical unit-test program(s), with assertions, and an integration suite that would invoke the unit-tests in addition to your blackbox tests.

The problem is that I am not sure what boundary to use between assertions and outputting the return value of the function (for logging purposes, since I like how Autotest will give me a diff).

With assertions there is nothing to output: either the expected and the actual values are equal and nothing happens, or they are different and the UT framework prints out an error message (expected is X, actual is Y). That is one let the computer do the job of testing.

With logging output diff, one has still to manually (visually) inspect the outcome of the diff (for instance: is there one item missing in the list or one extra item ...).

Since most functions return lists, it is quicker to prepare using the diff with expected output (expout using Autotest).

You might want to write a function that compares lists using assertions.

Answer 2

You may want to use CTest, which comes with CMake, a cross-platform make system with many backends: http://www.cmake.org/Wiki/CMake#CTest

PS: CMake is much more powerful than autotools too.