views:

418

answers:

2

We have our TeamCity builds set up using a build chain, so that our unit tests and and integration tests can run in parallel when triggered by a commit:

  • Build Chain - dependant on:
    • Unit tests
    • Integration tests

I am looking for a way that we can combine/merge the coverage data generated by the unit and integration tests in the build chain, so that we can get a better picture of how much actual code is covered by the two combined.

The plan then is to be able to monitor changes in coverage of committed code, and perhaps failing builds if percentages fall!

A: 

I have set up the 'build chain' target so that the coverage files (*.em, *.ec) from the unit and integration targets are available to it.

I created an ant build file specifically for the build chain target (with help from the emma doco!):

<project name="coverage-merge" basedir="." default="all">
    <!-- directory that contains emma.jar and emma_ant.jar: -->
    <property name="emma.dir" value="${basedir}/lib"/>
    <property name="coverage.dir" location="${basedir}/coverage"/>

    <path id="emma.lib">
        <pathelement location="${emma.dir}/emma-teamcity-3.1.1.jar"/>
        <pathelement location="${emma.dir}/emma_ant-2.0.5312.jar"/>
    </path>

    <taskdef resource="emma_ant.properties" classpathref="emma.lib"/>

    <target name="all" depends="-report"/>

    <target name="-report">
        <emma>
            <report sourcepath="${src.dir}" sort="+block,+name,+method,+class" 
                    metrics="method:70,block:80,line:80,class:100">
                <infileset dir="${coverage.dir}" includes="**/*.em, **/*.ec"/>

                <!-- for every type of report desired, configure a nested
                     element; various report parameters
                     can be inherited from the parent <report>
                     and individually overridden for each report type:
                -->
                <txt outfile="${coverage.dir}/coverage.txt" depth="package" 
                        columns="class,method,block,line,name"/>
                <xml outfile="${coverage.dir}/coverage.xml" depth="package"/>
                <html outfile="${coverage.dir}/coverage.html" depth="method" 
                        columns="name,class,method,block,line"/>
            </report>
        </emma>
    </target>
</project>

...which merges all the coverage files into a single report!

The metrics parameter of report sets the highlight threshold for the html report, so that the percentages against packages and files that are lower than the threshold are highlighted in red.

Modifying the xml output will allow me to use something like andariel to run an xpath over the results, and then force the build to fail if thresholds are not met!

brass-kazoo
A: 

Most of the code coverage tools I've encountered do not seem to have a way to combine test results from different or overlapping subsystems. As you have pointed out, this is a very useful ability.

The SD Test Coverage tools do have this ability and are available for Java, C, C++, C#, PHP and COBOL. In fact, the SD test coverage tools can combine test coverage data from multiple languages into a single monolithic result, so that you can get an overview of test coverage for your multi-lingual applications. It is able to show the coverage on all the source langauges involved, as well as provide summary reports.

Ira Baxter