Is it possible to programmatically generate JUnit test cases and suites?

Question

I have to write a very large test suite for a complex set of business rules that are currently captured in several tabular forms (e.g., if parameters X Y Z are such and such, the value should be between V1 and V2). Each rule has a name and its own semantics.

My end goal is to have a test suite, organized into sub test suites, with a test case to each rule.

One option is to actually hard code all these rules as tests. That is ugly, time consuming, and inflexible.

Another is to write a Python script that would read the rule files and generate Java classes with the unit tests. I'd rather avoid this if I can. Another variation would be to use Jython.

Ideally, however I would like to have a test suite that would read the files, and would then define sub-suites and tests within them. Each of these tests might be initialized with certain values taken from the table files, run fixed entry points in our system, and then call some validator function on the results based on the expected value.

Is there a reasonable way to pull this off using only Java?

Update: I may have somewhat simplified our kind of rules. Some of them are indeed tabular (excel style), others are more fuzzy. The general question though remains as I'm probably not the first person to have this problem.

Answer 1

Have you considered using FIT [http://en.wikipedia.org/wiki/Framework%5Ffor%5FIntegrated%5FTest%5D for that?

You seem to have the tables already ready, and "business rules" sounds like "business people write them using excel".

FIT is a system for checking tests based on tables with input->expected output mappings, and a open source java library for running those tests is available.

Answer 2

Within JUnit 4 you will want to look at the Parameterized runner. It was created for the purpose you describe (data driven tests). It won't organize them into suites however.

In Junit 3 you can create TestSuites and Tests programatically. The answer is in Junit Recipes, which I can expand if you need it (remember that JUnit 4 can run Junit 3 tests).

Answer 3

We tried FIT and decided to go with Concordion. The main advantages of this library are:

• the tests can be checked in alongside the code base (into a Subversion repository, for example)
• they are executed by a standard JUnit runner

Answer 4

I wrote something very similar using JUnit. I had a large number of test cases (30 pages) in an XML file. Instead of trying to generate different tests, I did it all in a single test, which worked just fine.

My test looked something like this:

void setup() { 
  cases = read in the xml file
}

void test_fn_works() {
  for case in cases {
    assert(case.expected_result, fn(case.inputs), 
        'Case ' + case.inputs + ' should yield ' + case.expected_result);

  }
}

With Ruby, I did exactly what you are saying-- generating tests on the fly. Doing this in Java, though, is complex, and I don't think it is worth it since there is another, quite reasonable approach.

Hope this helps.