Code coverage percentage not good

Question

Hello,

I have installed on my computer C++Test only with UnitTest license (only Unit Test license) as a Visual Studio 2005 plugin ( cpptest_7.2.11.35_win32_vs2005_plugin.exe ).

I have a sample similar to the following:

bool MyFunction(... parameters... )
{
    bool bRet = true;

        // do something
    if( some_condition )
    {
        // do something
        bRet = CallToAFunctionThatCanReturnBothTrueAndFalse....
    }
    else
    {
        bRet = false;
        // do something
    }

    if(bRet == false)
    {
        // do something
    }

    return bRet;
}

In my case after running the coverage tool I have the following results (for a function similar to the previously mentioned):

[LC=100 BC=100 PC=75 DC=100 SCC=100 MCDC=50 (%)]

I really don't understand why I don't have 100% coverage when it comes to PathCoverage (PC). Also if someone who has experience with C++Test Parasoft could explain the low MCDC coverage for me that would be great.

What should I do to increase coverage? as I'm out of ideas in this case. Directions to (some parts of) the documentation are welcome.

Thank you,

Iulian

Answer 1

I can't help with the specific tool you're using, but the general idea with path coverage is that each possible path through the code should be executed.

If you draw a flowchart through the program, branching at each if/break/continue, etc. you should see which paths your tests are taking through the program. To get 100% (which isn't totally necessary, nor does it assure a perfect test) your test will have to go down every branch of the code, executing every line.

Hope that helps.

Answer 2

You need at least two testcases to get 100% coverage. One where some_condition is true and one where it is not. If you have that you should get 100% coverage.

Though you should see 100% coverage as perfect. You would need 3 tests for that in this case so all combinations can be tested. Look up cyclomatic complexity to learn more about that.

Answer 3

This is a good reference on the various types of code coverage: http://www.bullseye.com/coverage.html.

MCDC: To improve MCDC coverage you'll need to look at some_condition. Assuming it's a complex boolean expression, you'll need to look at whether you're exercising the necessary combinations of values. Specifically, each boolean sub-expression needs to be exercised true and false.

Path: One of the things mentioned in the link above as being a disadvantage of path coverage is that many paths are impossible to exercise. That may be the case with your example.

Answer 4

There are four hypothetical paths through that function. Each if-clause doubles the number of paths. Each if-statement is a branch where you can go two different ways. So whenever your tool encounters an "if", it assumes the code can either take the "true" branch or the "false" branch. However, this is not always possible. Consider:

bool x = true;
if (x) {
    do_something();
}

The "false" branch of the if-statement is unreachable. This is an obvious example, but when you factor in several if-statements it becomes increasingly difficult to see whether a path is possible or not.

There are only three possible paths in your code. The path that takes the "false" branch in the first if statement and the "true" branch in the second is unreachable.

Your tool is not smart enough to realize that.

That being said, even if the tool is perfect, obtaining 100 % path coverage is probably unlikely in a real application. However, very low path coverage is a sure sign that your method has too high cyclomatic complexity.

Answer 5

Personally, I think it's bad form to start ANY function with

bool retCode = true;

You're making an explicit assumption that it'll succeed by default, and then fail under certain conditions.

Programmers coming after you will not make this same assumption.

Fail fast, fail early.

And as others have said, if you want to test failure cases, you have to code tests that fail.