views:

124

answers:

2

I have C++/CLI code and I'm using Visual Studio 2008 Team Suite Code Coverage.

The code header:

// Library.h

#pragma once

#include <string>

using namespace System;

namespace Library
{
    public ref class MyClass
    {
    public:
  static void MyFoo();
  static std::string Foo();
    };
}

The code implementation:

#include "Library.h"

using namespace Library;
using namespace System;

void MyClass::MyFoo()
{
 Foo();
}

std::string MyClass::Foo()
{
 return std::string();
}

I have a C# unit test, that calls MyClass.MyFoo():

[TestMethod]
public void TestMethod1()
{
    Library.MyClass.MyFoo();
}

For some reason, I don't get a full code coverage for MyClass. The Foo() method has 3 uncovered blocks and 5 covered blocks. The closing curly brackets (}) are marked in orange - partially covered. I have no idea why is it partially covered instead of fully covered, and this is my question.

MyClass Code Coverage Print Screen

UPDATE

Another Example:

Header:

// Library.h

#pragma once

using namespace System;

namespace Library
{
    struct MyStruct
    {
        int _number;
    };

    public ref class MyClass
    {
    public:
        static void MyFoo();
        static MyStruct* Foo();
    };
}

Implementation:

#include "Library.h"

using namespace Library;
using namespace System;

void MyClass::MyFoo()
{
    delete Foo();
}

MyStruct* MyClass::Foo()
{
    return new MyStruct();
}

I'm still getting the same missing coverage in Foo's return statement.

+2  A: 

You aren't covering the case where the function exits via exception instead of normally. Of course if you can't even construct a zero-length std::string then your program is probably too far gone for recovery, but determining that is beyond the scope of code coverage analysis.

EDIT: In order to improve coverage you can mock up a global operator new which fails based on some global flag (or more flexibly, fails on the Nth allocation) which you can set in your test case.

e.g.

int allocation_failure = 0;
void* operator new(size_t requestedbytes)
{
    if (allocation_failure) {
        if (!--allocation_failure) {
            throw std::bad_alloc();
        }
    }
    void* retval = malloc(requestedBytes);
    if (!retval) {
        throw std::bad_alloc();
    }
    return retval;
}

void operator delete(void* p)
{
    if (p) free(p);
}

Or you could conditionally fail allocations of a particular size, or the Nth allocation of a particular size, etc, to exercise all the possible paths through your code.

Ben Voigt
Look at the second example I gave. It doesn't use std::string.
brickner
Any use of `new` can fail by throwing an exception. Your test case isn't covering the exceptional path.
Ben Voigt
I was doubtful, but you were right.I've managed to create code similar to my 2nd example and fully cover it by making it throw an exception.I couldn't fully cover something that returns std::string, but it seems I'm having hard time making it throws exceptions in all possible places.Thank you!
brickner
A: 

[Full disclosure: I'm on the team that helps create the code coverage tools in VS]

Try collecting code coverage against a release build instead of a debug build.

The compiler can emit IL that is actually unreachable after certain optimizations/transformations have been applied. The easiest way to see if this is the case is to look at the IL with ildasm or Reflector. See here for an example.

Chris Schmich
Just out of curiosity (since you're here :-P), do you know (or can you say) whether Microsoft at any point intends to support testing and coverage analysis for native C++? We C++ developers frequently feel left out in the cold, with all the awesome new Visual Studio features being .NET-only (though, admittedly, the native code profiler is quite nice).
James McNellis
We've received lots of feedback about the need to better support native code with our tooling. I can't speak to the future of native testing (that's a different team), but code coverage has worked for native code since VS2005 (like the profiler). The experience is not nearly as polished as it is for managed, but the functionality is there. See http://blogs.msdn.com/ms_joc/archive/2005/10/04/477166.aspx for more info.
Chris Schmich
@Chris: Thanks a lot for the link; I had no idea that was an available feature. From the Visual Studio GUI, you'd never guess that native code coverage was supported. The lack of built-in highlighting support is disappointing, but coverage metrics + XML format output = a huge plus. Awesome.
James McNellis
I've tried compiling it in release and running the test. I'm still get partial coverage.I've tried using reflector, but it throws an exception for the unmanaged Foo() method.
brickner
Full disclosure: I'm on the team that creates test coverage tools at Semantic Designs. If the problem is that "type erasure" inserts code before the test coverage instrumentation is inserted, you'll end up with kind of kind of effect. Our tools operate on the unadorned source code (e.g., before type erasure), and so you shouldn't get this particular kind of artifact. (See http://www.semanticdesigns.com/Products/TestCoverage/CppTestCoverage.html).
Ira Baxter
How do I know if my problem is that "type erasure" inserts code before the test coverage instrumentation is inserted?
brickner
That's the point of doing the test coverage probe insertion *before* the compiler fiddles with the code.
Ira Baxter