Parameterized test case classes in JUnit 3.x

Question

I have a JUnit 3.x TestCase which I would like to be able to parameterize. I'd like to parametrize the entire TestCase (including the fixture). However, the TestSuite.addTestSuite() method does not allow be to pass a TestCase object, just a class:

   TestSuite suite = new TestSuite("suite");
   suite.addTestSuite(MyTestCase.class);

I would like to be able to pass a parameter (a string) to the MyTestCase instance which is created when the test runs. As it is now, I have to have a separate class for each parameter value.

I tried passing it an anynomous subclass:

   MyTestCase testCase = new MyTestCase() {
       String getOption() {
           return "some value";
       }
   }

   suite.addTestSuite(testCase.getClass());

However, this fails with the assertion:

   ... MyTestSuite$1 has no public constructor TestCase(String name) or TestCase()`

Any ideas? Am I attacking the problem the wrong way?

Answer 1

If this is Java 5 or higher, you might want to consider switching to JUnit 4, which has support for parameterized test cases built in.

Answer 2

Rather than create a parameterized test case for the multiple/different backends you want to test against, I would look into making my test cases abstract. Each new implementation of your API would need to supply an implementing TestCase class.

If you currently have a test method that looks something like

public void testSomething() {
   API myAPI = new BlahAPI();
   assertNotNull(myAPI.something());
}

just add an abstract method to the TestCase that returns the specific API object to use.

public abstract class AbstractTestCase extends TestCase {
    public abstract API getAPIToTest();

    public void testSomething() {
       API myAPI = getAPIToTest();
       assertNotNull(myAPI.something());
    }

    public void testSomethingElse() {
       API myAPI = getAPIToTest();
       assertNotNull(myAPI.somethingElse());
    }
}

Then the TestCase for the new implementation you want to test only has to implement your AbstractTestCase and supply the concrete implementation of the API class:

public class ImplementationXTestCase extends AbstractTestCase{

    public API getAPIToTest() {
        return new ImplementationX();
    }
}

Then all of the test methods that test the API in the abstract class are run automatically.

Answer 3

Ok, here is a quick mock-up of how JUnit 4 runs parameterized tests, but done in JUnit 3.8.2.

Basically I'm subclassing and badly hijacking the TestSuite class to populate the list of tests according to the cross-product of testMethods and parameters.

Unfortunately I've had to copy a couple of helper methods from TestSuite itself, and a few details are not perfect, such as the names of the tests in the IDE being the same across parameter sets (JUnit 4.x appends [0], [1], ...).

Nevertheless, this seems to run fine in the text and AWT TestRunners that ship with JUnit as well as in Eclipse.

Here is the ParameterizedTestSuite, and further down a (silly) example of a parameterized test using it.

(final note : I've written this with Java 5 in mind, it should be trivial to adapt to 1.4 if needed)

ParameterizedTestSuite.java:

package junit.parameterized;

import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Collection;

import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;

public class ParameterizedTestSuite extends TestSuite {

 public ParameterizedTestSuite(
   final Class<? extends TestCase> testCaseClass,
   final Collection<Object[]> parameters) {

  setName(testCaseClass.getName());

  final Constructor<?>[] constructors = testCaseClass.getConstructors();
  if (constructors.length != 1) {
   addTest(warning(testCaseClass.getName()
     + " must have a single public constructor."));
   return;
  }

  final Collection<String> names = getTestMethods(testCaseClass);

  final Constructor<?> constructor = constructors[0];
  final Collection<TestCase> testCaseInstances = new ArrayList<TestCase>();
  try {
   for (final Object[] objects : parameters) {
    for (final String name : names) {
     TestCase testCase = (TestCase) constructor.newInstance(objects);
     testCase.setName(name);
     testCaseInstances.add(testCase);
    }
   }
  } catch (IllegalArgumentException e) {
   addConstructionException(e);
   return;
  } catch (InstantiationException e) {
   addConstructionException(e);
   return;
  } catch (IllegalAccessException e) {
   addConstructionException(e);
   return;
  } catch (InvocationTargetException e) {
   addConstructionException(e);
   return;
  }


  for (final TestCase testCase : testCaseInstances) {
   addTest(testCase);
  }  
 }
 private Collection<String> getTestMethods(
   final Class<? extends TestCase> testCaseClass) {
  Class<?> superClass= testCaseClass;
  final Collection<String> names= new ArrayList<String>();
  while (Test.class.isAssignableFrom(superClass)) {
   Method[] methods= superClass.getDeclaredMethods();
   for (int i= 0; i < methods.length; i++) {
    addTestMethod(methods[i], names, testCaseClass);
   }
   superClass = superClass.getSuperclass();
  }
  return names;
 }
 private void addTestMethod(Method m, Collection<String> names, Class<?> theClass) {
  String name= m.getName();
  if (names.contains(name))
   return;
  if (! isPublicTestMethod(m)) {
   if (isTestMethod(m))
    addTest(warning("Test method isn't public: "+m.getName()));
   return;
  }
  names.add(name);
 }

 private boolean isPublicTestMethod(Method m) {
  return isTestMethod(m) && Modifier.isPublic(m.getModifiers());
  }

 private boolean isTestMethod(Method m) {
  String name= m.getName();
  Class<?>[] parameters= m.getParameterTypes();
  Class<?> returnType= m.getReturnType();
  return parameters.length == 0 && name.startsWith("test") && returnType.equals(Void.TYPE);
  }

 private void addConstructionException(Exception e) {
  addTest(warning("Instantiation of a testCase failed "
    + e.getClass().getName() + " " + e.getMessage()));
 }

}

ParameterizedTest.java:

package junit.parameterized;
import java.util.Arrays;
import java.util.Collection;

import junit.framework.Test;
import junit.framework.TestCase;
import junit.parameterized.ParameterizedTestSuite;


public class ParameterizedTest extends TestCase {

 private final int value;
 private int evilState;

 public static Collection<Object[]> parameters() {
  return Arrays.asList(
    new Object[] { 1 },
    new Object[] { 2 },
    new Object[] { -2 }
    );
 }

 public ParameterizedTest(final int value) {
  this.value = value;
 }

 public void testMathPow() {
  final int square = value * value;
  final int powSquare = (int) Math.pow(value, 2) + evilState;
  assertEquals(square, powSquare);
  evilState++;
 }

 public void testIntDiv() {
  final int div = value / value;
  assertEquals(1, div);
 }

 public static Test suite() {
  return new ParameterizedTestSuite(ParameterizedTest.class, parameters());
 }
}

Note: the evilState variable is just here to show that all test instances are different as they should be, and that there is no shared state between them.