Is there a JUnit TestRunner for running groups of tests?

Question

I am currently using JUnit 4 and have a need to divide my tests into groups that can be run selectively in any combination. I know TestNG has a feature to annotate tests to assign them to groups, but I can't migrate to TestNG right now. It seems this could easily be accomplished in JUnit with some custom annotations and a custom JUnit TestRunner. I've checked both the JUnit docs and searched the web but couldn't find such a thing. Is anyone aware of such a TestRunner?

Update: Thanks for your replies regarding test suites. I should have addressed these in my original question. Here we go: I don't want to use test suites because they would require me to manually create and manage them, which means touching ALL my tests and arranging them into suites manually (too much work and a maintenance nightmare). All I need to do is run all unit tests, except a few that are really integration tests. So I want to annotate these, and run all others. At other times, I want to just run the integration tests. I also have the need to put a unit test into multiple groups, which is not possible with suites. Hope this helps to clear things up.

Update 2: If JUnit doesn't have this OOB, I'm looking for an Open Source library that adds this to JUnit (annotations + custom JUnit Test Runner).

Answer 1

You can create suites, although that puts all the configuration in the suite, and not in annotations.

Answer 2

JUnit 3 allows you to create test-suites which can be run like any other test. Doesn't JUnit 4 have a similar concept?

Answer 3

If you're using ANT or maven, you can control which tests are run by filtering the tests by name. A bit awkward, but it might work for you.

Answer 4

No, there is no similar concept to TestNG groups, unfortunately. It was planned for JUnit4, but for some unclear reason, it was dropped from the planning.

Answer 5

JUnit has no such runner at the moment. Addressing the underlying issue, the need to get reasonable assurance from a test suite in a limited amount of time, is our highest development priority for the next release. In the meantime, implementing a Filter that works through annotations seems like it wouldn't be a big project, although I'm biased.

Answer 6

Check out Spring's SpringJUnit4ClassRunner. I've used it to optionally run tests based on a System property, using the IfProfileValue annotation.

This:

@IfProfileValue(name="test-groups", values={"unit-tests", "integration-tests"})
  public void testWhichRunsForUnitOrIntegrationTestGroups() {
      // ...
 }

Will run if the System property 'test-groups' is set to either 'unit-tests' or 'integration-tests'.

Update: JUnitExt has @Category and @Prerequisite annotations and looks like it should do what you need. However, I've never used it myself, so I can't vouch for it.

Answer 7

First, you are addressing two problems - unit tests (often in the same package as the unit under test) and integration tests. I usually keep my integration tests in a separate package, something like com.example.project.tests. In eclipse, my projects look like:

project/
  src/
    com.example.project/
  tsrc/
    com.example.project/
    com.example.project.tests/

Right-clicking on a package and selecting 'run' runs the tests in the package; doing the same on the source folder runs all the tests.

You can acheive a similar effect, although you expressed a disinterest in it, by using the Suite runner. However, this violates DRY - you have to keep copies of the test names up to date in the suite classes. However, you can easily put the same test in multiple suites.

@RunWith(Suite.class)
@Suite.SuiteClasses( { 
    TestAlpha.class, 
    TestBeta.class })
public class GreekLetterUnitTests {
}

Of course, I really should be keeping these things automated. A good method for doing that is to use the Ant task.

<target name="tests.unit">
  <junit>
    <batchtest>
      <fileset dir="tsrc">
        <include name="**/Test*.java"/>
        <exclude name="**/tests/*.java"/>
      </fileset>
    </batchtest>
  </junit>
</target>
<target name="tests.integration">
  <junit>
    <batchtest>
      <fileset dir="tsrc">
        <include name="**/tests/Test*.java"/>
      </fileset>
    </batchtest>
  </junit>
</target>

Answer 8

TestNG has my vote. It's annotation based, can run as Groups, single Tests, etc, can be linked into Maven, and can run all JUnit tests as part of it's test runs.

I highly recommend it over JUnit.

Answer 9

my advice is simply ditch JUnit and use TestNG. Once you get used to TestNG, Junit looks like Stone Age.