How do you integrate a TDD approach with VisualStudio?

Question

I am interested in hearing about experiences using TDD and unit testing for C++ in general with Visual Studio 2005 (Professional)

First some background. We have a fairly large project and much of it has been developed on Linux using CppUnit for the unit tests. The project is divided into several libraries, each with their own set of tests. I have a simple script which compiles the library, compiles the test suite and then runs the tests. So after making changes in the code I just run "test" from a command line and the tests run.

Now, most of the developers are using Visual Studio 2005 on Windows for the development of this product. Of course they can still run the tests from a command line using nmake but involves extra steps and I would prefer to have a much more integrated solution.

So my question has two parts.

Firstly, what is the best way of laying out the code for tests on a large code base? Is it normal to create several test projects in a solution, one for each library?

Secondly are there any tools for integrating CppUnit tests into Visual Studio? With dependencies set up corectly running the test project should run the tests but currently the results still appear in a command window.

Answer 1

My team is currently using an system where we have an automated nightly build (that can also be run from the project build dashboard by anyone) that includes a VS2k5 "test" solution. The test solution holds all the unit test projects; one unit test project for every "unit" of code in the main project.

When the automated build runs, it builds the main solution, then the test solution, and finally runs all the executables produced by the test solution (a Perl script glues this together). The results of the compile as well as the test execution (EXIT _ SUCCESS, EXIT _ FAILURE) are used to update the project build dashboard.

That EXIT _ FAILURE trick can also be applied to a custom build step of the main project: if the unit test custom build step returns EXIT _ FAILURE, then the build itself fails.

Answer 2

I use the Boost Test framework. I tend to split my code into .lib files and will have a separate console-mode EXE test project for each. When the test project is built it makes use of the 'Post build stage' to launch itself, thus running the tests. You could make each test project a dependency of your main application so that each time it builds, all tests are run first, but this can be time-consuming. Instead I tend to run the test projects by hand as needed, but my automated nightly build system will run all test projects as a matter of course (I script this and if any tests fail, the build fails and I get an email notification).

More details here.

Answer 3

One of the projects at my company does exactly this. We use a unit test framework called CXXTest (http://cxxtest.sourceforge.net/guide.html). We really like this framework for C++ because it only requires that you write a header file that contains your unit tests. The .CPP files are created by a script (both Python and Perl scripts are provided).

We integrate with visual studio by providing a post build step that builds the unit tests (if they need building) and then executes them. The output (showing what passed and what failed) is displayed in the output window -- you never need to leave the IDE.

Answer 4

You can also use managed C++ to write unit tests in Visual Studio, using the unit testing framework that's built in.

Answer 5

• I find the following folder hierarchy useful. Create code and tests as the subfolders of ProjectFolder. Create 2 solutions code\Project.sln and tests\Tests.sln. Now for every class library or executable created, e.g. Customers.dll have a corresponding test dll. So code\Customers\Customers.csproj will have tests\Customers\TestCustomers.csproj, which references the former.
• Integrating CPPUnit into Visual Studio would be on the lines of choosing the right application in Project Properties.. 'Debug' settings. I think this page has what you need to have single key press test execution + reporting within the IDE.

Answer 6

Have a look at CUnitWin32. There's an example included as well.

Answer 7

Here is what I do:

• Create a test executable project, in your main solution, which uses source from only the unit, the unit's tests and the test framework.
• Make the test runner generate a text file on successful run of the tests, so visual studio can track dependencies.
• Add a project to launch your test runner and generate the test file. This means you now have two projects per test.
• Make the test runner a dependency of the library that incorporates the unit.

Personally, I don't think the test framework (Google Test, Boost test, CppUnit, etc) matters that much. Most are pretty much functionally equivalent.

I'm not entirely happy with the number of projects generated, but I consider this a Visual Studio GUI issue, in the sense that its actually quite useful to have these projects included like this for purposes such as debugging.

I tried using post build steps to run the tests but this unfortunately mean that the build was not interrupted after the first failure has passed.