How can I distinguish between "run all tests" and "run just this test"?

Question

I have tests which run very slow, say a test which exports a large database to test the exporting code for memory leaks. I'd like to have these tests in my usual test suite but they should be ignored unless one of these conditions is fulfilled:

1. The test is running on a CI server
2. The user has selected this test in the IDE and runs it

The solution for #1 is simple: Use a property and skip the test if it's not set.

But how could I implement #2?

[EDIT] This is Java specific. I'm using JUnit 4 with Eclipse.

Answer 1

I have tests which run very slow, say a test which exports a large database to test the exporting code for memory leaks.

While useful, these are not unit tests, as denoted by the unit-test tag.

A test is not a unit test if:

• It talks to the database
• It communicates across the network
• It touches the file system
• It can't run at the same time as any of your other unit tests
• You have to do special things to your environment (such as editing config files) to run it.

Tests that do these things aren't bad and are certainly worth writing. They can even be written in using unit test framework. However, it is important to be able to separate them from true unit tests so that you can keep a set of tests that you can run fast whenever you make changes, particularly in a TDD cycle where you want to go from failing test to passing test as quickly as possible.

If you are using a Makefile you could have a check target for true unit-tests and a livecheck target for these slow, system tests.

How to run each test individually will depend on your framework.

Answer 2

We use Maven for that, which integrates nicely with Eclipse. We use JUnit for

• unit tests, which are prefixed or suffixed simply with Test and are run on every full build
• integration tests, which are prefixed IntegrationTest. Those are only run when a special Maven profile is activated via a command line switch (-PintegrationTests).

The complete set of integration tests is run three times a day on the CI server or by a developer who explicitly specifies the above-mentioned profile (mvn test -PintegrationTests). A single integration test can of course be run by programmers in their IDE at any time.

This is the relevant part from pom.xml:

<profiles>
    <profile>
        <id>integrationTests</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                           <includes>
                                <include>**/Test*.java</include>
                                <include>**/*Test.java</include>
                                <include>**/IntegrationTest*.java</include>
                         </includes>
                      </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>