How to run test cases on a list of slightly different jars ?

Question

I have a list of jar files, all containing slightly different versions of the same system.

I want to execute a set of test cases on each jar file while being able to obtain the results for each jar (via RunListener or something like that).

I am a bit confused with the class loading problems that are implied.

How can I nicely do this ?

Answer 1

If the jars maintain the same interface, then all you have to do is run each test with a slighly different classpath - each time with jars from another version.

If you're using Eclipse JUnit integration, just create N run configurations, in each one in the classpath tab specify required jars.

If you want to do it programmatically, then I suggest you start with empty classpath and use URLClassLoader, giving it a different set of jars each time.

Something like this:

URLClassloader ucl = new URLClassLoader( list of jars from version1 );
TestCase tc = ucl.loadClass("Your Test Case").newInstance();
tc.runTest();

ucl = new URLClassLoader( list of jars from version2 );
TestCase tc = ucl.loadClass("Your Test Case").newInstance();
tc.runTest();

Answer 2

If you are using ant, the JUnit task takes a classpath.

<target name="test">
  <junit ...">
    <classpath>
      <pathelement path="${test.jar.path}" />
    </classpath>
    ...
  </junit>
</target>

You can use the antcall task to call your test target repeatedly with a differed value for the test.jar.path property.