Unit testing an executable project

Question

Maybe I am not thinking about this correctly. I am starting my second project using unit tests. My first project I rolled my own, for this project I am trying out Boost::test.

My question is, what are the proper procedures for unit testing projects that compile into executables? It seems like everything I see out there is for libraries and dependencies. I want my exe project to be unit tested, but I do not want a bunch of unit test functions floating around in the binary, nor do I want to do

#ifdef _DEBUG
    BOOST_AUTO_TEST_CASE( my_func )
    {
    }
#endif

around all my tests.

I thought about creating a separate project for unit tests, but that does not really work for executable.. unless I want to do some fancy pre-build operation copying from my other project into the test project..

Any thoughts or ideas?

Answer 1

The very idea is to create a seperate project where you test seperate units of code for the expected behaviour regardless of building debug or not (some issues don't even show up in debug builds due to code differences).
You build it as a binary and run to see if your changes didn't break anything - often it is even set up as a automatic post-build action.

If you want to test your application from the outside - there are probably some testing frameworks you can use, depending on the area/framework/etc. of the application... But Boost.Test and all the other unit testing frameworks are not intended for testing executables.

Answer 2

Unit-testing an executable can be a great idea - but realize that it's a whole different monster than unit-testing code. Once you have an executable, there is no more C++ or Boost. It's just a program. You need to be able to run it and analyze/control your inputs in a different mechanism:

Input:

• arguments
• stdin
• environment variables (maybe)
• configuration files (maybe)

Output:

• return value
• stdout
• output files (maybe)

It's probably going to be easiest to run it inside a shell. bash will work wonders in a Linux environment (pipe to/from files, run sed/awk/grep on the output to analyze correctness). You could use Perl or Python and their own respective unit-testing frameworks, with the caveat that you have to wrap the invocation of your program in some other function: both languages support the notion of pipes from subprocesses, python with the subprocess module and Perl with its standard file-opening mechanism.

Whatever you do, you do not want to try to unit-test an entire executable from C++.

Answer 3

The project can be compiled as a library and this library linked, possibly statically, in two separate executables: the "project", which will be delivered, and the unit tests.

Now the problem seems to originate from your IDE, which one is it ? Does it allow to create two binaries for one project ?

Answer 4

Hi Charles,

i use cppunit to test my executables within an extra project which simply links to the *.obj files generated from the executables code. This way you don't have any test logic in the original code base and can write a seperate console or windows application for your tests.

Cheers Holger