How can I make Cobertura/Emma play nice with an Eclipse application?

Question

My company is in the middle of upgrading our build system to use Buckminster (which has gone well). Naturally the managers would like it if we could automatically generate some of the metrics they use for code while we're at it - one of these metrics is coverage for the unit tests.

The previous build, which was PDE-based, resulted in a separate headless application which parses through all plugins in the context, looks inside them for test classes, tosses all test cases into a larger suite, and then runs it programmatically, via JUnitResultFormatter.startTestSuite(). In the process it adds some homegrown tests for particular metadata (copyright markings, etc) and tests for package dependency.

It seems like I should be able to just export the product for the test app, instrument it with the coverage tool of my choice, and then run it; however, it looks like both the coverage tools want to know the entire classpath they're dealing with. Is it possible to make either of these tools play nice without trawling my entire build workspace for jars and dependencies?

Answer 1

Managed to get everything working, with the help of an older SO question.

We started with this post, which provided us with an example coverage setup using EMMA. However, it appeared that we'd need to force a dependency on EMMA on every plugin we wanted to get coverage data for.

Some Google forensics got us to this book excerpt, which rather adequately covers the OSGi classloader hierarchy. By adding the osgi.parentClassloader=app line to the config.ini of the test running application, we could specify a classpath on the commandline. That classpath needed to include:

• Eclipse's startup.jar
• The Java SDK jars used by the application
• The jars for the coverage tool

Of course, we were running the tests via the <java jar="foo.jar"> Ant task, which does silently ignores any classpath information you provide and uses just the jar. After we got Emma working, we switched to Cobertura, and our final Ant script (slightly abridged and anonymized) looks like:

<target name="generate.coverage" depends="buckminster.init">
  <!-- Generate the classpath. -->
  <path id="cobertura.classpath">
    <fileset dir="${tasks.dir}/lib/cobertura">
      <include name="cobertura.jar" />
      <include name="**/*.jar" />
    </fileset>
  </path>
  <taskdef classpathref="cobertura.classpath" resource="tasks.properties" />

  <!-- Get ready to run the unit tests app, and delete old coverage data. -->
  <unzip src="${test-app.artifact}" dest="${output.dir}" overwrite="true" />
  <delete file="${output.dir}/cobertura.ser" />

  <!-- Instrument the jars in-place. Make sure to only capture what you want instrumented! -->
  <cobertura-instrument datafile="${output.dir}/cobertura.ser">
    <fileset dir="${output.dir}/test-app/plugins">
      <include name="**/*our.company_*.jar" />
    </fileset>
  </cobertura-instrument>

  <!-- Run the unit test application, by classname rather than by jar. -->
  <java fork="true" classname="org.eclipse.equinox.launcher.Main" logerror="true" failonerror="true" maxmemory="1G">
    <classpath>
      <pathelement location="${output.dir}/test-app/startup.jar" />
      <pathelement location="${tasks.dir}/lib/cobertura/cobertura.jar" />
      <fileset dir="${tasks.dir}/lib/cobertura">
        <include name="**/*.jar" />
      </fileset>
      <pathelement location="${java.class.path}" />
    </classpath>
    <sysproperty key="net.sourceforge.cobertura.datafile" file="${output.dir}/cobertura.ser" />            
    <arg value="-data" />
    <arg value="${output.dir}/test-app/workspace" />
    <arg value="--formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter:${log.dir}/test-results.xml" />
  </java>

  <!-- Generate the coverage reports. -->
  <cobertura-report format="html" datafile="${output.dir}/cobertura.ser" destdir="${output.dir}/cobertura-report">
    <fileset dir="${workspace.dir}/plugins">
      <include name="**/*.java" />
    </fileset>
  </cobertura-report>
</target>

Hopefully this helps someone out who's in the same spot we were in.