Guidelines for writing a test suite

Question

What are the best practices/guidelines for writing test suite for C++ projects?

Answer 1

This is a very broad question. For unit testing and Test Driven Development (TDD), there is some useful (if somewhat platidinous in parts) guidance on this from Microsoft - you can overlook the Visual Studio-specific advice, if it does not apply.

If you are looking for guidance on system or performance testing, I would clarify your question. There is a decent broader rationale in the docs for Boost.Test.

There are several unit testing best practices to review before we close. Firstly, TDD is an invaluable practice. Of all the development methodologies available, TDD is probably the one that will most radically improve development for many years to come and the investment is minimal. Any QA engineer will tell you that developers can't write successful software without corresponding tests. With TDD, the practice is to write those tests before even writing the implementation and ideally, writing the tests so that they can run as part of an unattended build script. It takes discipline to begin this habit, but once it is established, coding without the TDD approach feels like driving without a seatbelt.

For the tests themselves there are some additional principals that will help with successful testing:

Avoid creating dependencies between tests such that tests need to run in a particular order. Each test should be autonomous.

Use test initialization code to verify that test cleanup executed successfully and re-run the cleanup before executing a test if it did not run.

Write tests before writing the any production code implementation.

Create one test class corresponding to each class within the production code. This simplifies the test organization and makes it easy to choose where to places each test.

Use Visual Studio to generate the initial test project. This will significantly reduce the number of steps needed when manually setting up a test project and associating it to the production project.

Avoid creating other machine dependent tests such as tests dependent on a particular directory path.

Create mock objects to test interfaces. Mock objects are implemented within a test project to verify that the API matches the required functionality.

Verify that all tests run successfully before moving on to creating a new test. That way you ensure that you fix code immediately upon breaking it.

Maximize the number of tests that can be run unattended. Make absolutely certain that there is no reasonable unattended testing solution before relying solely on manual testing.

Answer 2

TDD is certainly one set of bests practices. When retrofitting tests, I aim for two things code coverage and boundary condition coverage. Basically you should pick inputs to functions such that A) All code paths are tested and better if all permutations of all code paths are tested(sometimes that can be a large number of cases and not really be necessary if path differences are superficially different) and B) That all boundary conditions(those are conditions that cause variation in code path selection) are tested if your code has an if x > 5 in it you test with x = 5, and x = 6 to get both sides of the boundary.