Surefire is not picking up Junit 4 tests

Question

For some reason I cannot get Maven 2 Surefire plugin to execute JUnit 4 test class.

public class SimpleTest {
  @org.junit.Test
  public void simple() {
     System.out.println("foo");
  }
}

However if I change this class to be JUnit-3 like, such as

public class SimpleTest extends junit.framework.TestCase {
  public void testBar() {
     System.out.println("bar");
  }

  @org.junit.Test
  public void simple() {
     System.out.println("foo");
  }
}

then it gets executed. Here's what I've done:

• verified Maven version: Apache Maven 2.2.1 (r801777; 2009-08-06 20:16:01+0100)
• verified Surefire version: followed this advice
• verified Surefire version: checked Surefire jars in my ~/.m2/repository/org/apache/maven/surefire -- all of them are either version 2.4.2 or 2.4.3
• done a mvn dependency:tree | grep junit to ensure I only depend on junit version 4.7

The module I am having this problem at doesn't have JUnit 3 tests.

Is there anything else I am missing?

Answer 1

Have you configured your maven-compile-plugin for the correct compiler level, like:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <source>1.5</source>
    <target>1.5</target>
  </configuration>
<plugin>

Otherwise maven will have trouble with annotations

Answer 2

The verification that you've done are good, especially checking that you are using version 2.3+ of the surefire plugin (by default, you'll get version 2.4.3 with maven 2.1 super POM so this should be ok) and checking that you not pulling the junit-3.8.1.jar dependency transitively.

Now, just to validated that this is not a "global problem" (I don't think so TBH), could you create a project from scratch, for example by running:

mvn archetype:create -DgroupId=com.mycompany.app -DartifactId=maven-junit4-testcase

Then update the junit dependency:

<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.7</version>
  <scope>test</scope>
</dependency>

And configure the compiler level for 1.5+

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <source>1.5</source>
    <target>1.5</target>
  </configuration>
</plugin>

Finally put your SimpleTest.java next to AppTest.java and run mvn test.

If running mvn test works fine for that project (and I'm expecting it to run without problem), could you please update your question with the POM configuration you're using (from the project having troubles)?

Answer 3

I don't know what you mean by "can't execute," but does it help to explicitly set the includes used by the maven-surefire-plugin?

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.4.3</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
        </includes>
    </configuration>
</plugin>

Also, does running maven with the -X flag provide any useful information?

Answer 4

I'm running into the same issue. Performing the shell project exercise does indeed execute the junit4 tests. But my project still doesn't execute the tests unless they extend "TestCase". I did notice executing surefire:test caused the downloading for junit-3.8.1. Not sure why. Below is my pom and some output from the build. Any ideas greatly appreciated!

<?xml version="1.0"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany.service</groupId>
<artifactId>LocationServiceIntTest</artifactId>
<version>10.01</version>
<packaging>jar</packaging>
<name>Location Service Integration Test</name>
<repositories>
    <repository>
        <id>central</id>
        <url>XXXXX</url>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>
    <repository>
        <id>3rdp-releases</id>
        <url>XXXXX
        </url>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>
    <repository>
        <id>snapshots</id>
        <url>XXXXX</url>
        <releases>
            <enabled>true</enabled>
        </releases>
    </repository>
</repositories>
<pluginRepositories>
    <pluginRepository>
        <id>central</id>
        <url>XXXX</url>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </pluginRepository>
    <pluginRepository>
        <id>snapshots</id>
        <url>XXXXX</url>
        <releases>
            <enabled>false</enabled>
        </releases>
    </pluginRepository>
</pluginRepositories>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.axis2</groupId>
            <artifactId>axis2-wsdl2code-maven-plugin</artifactId>
            <version>1.5.1</version>
            <executions>
                <execution>
                    <goals>
                        <goal>wsdl2code</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <packageName>com.mycompany.service</packageName>
                <wsdlFile>${basedir}/../ServiceClient/src/main/resources/LocationService.wsdl
                </wsdlFile>
                <databindingName>adb</databindingName>
                <unpackClasses>true</unpackClasses>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.5</source>
                <target>1.5</target>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.4.3</version>
            <configuration>
                <includes>
                    <include>**/*Test.java</include>
                </includes>
            </configuration>
        </plugin>
    </plugins>
</build>
<dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.4</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.unitils</groupId>
        <artifactId>unitils-core</artifactId>
        <version>3.1</version>
        <type>pom</type>
    </dependency>
    <dependency>
        <groupId>org.unitils</groupId>
        <artifactId>unitils-testng</artifactId>
        <version>3.1</version>
        <type>pom</type>
    </dependency>
    <dependency>
        <groupId>org.apache.axis2</groupId>
        <artifactId>axis2</artifactId>
        <version>1.5.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.ws.commons.axiom</groupId>
        <artifactId>axiom-impl</artifactId>
        <version>1.2.8</version>
    </dependency>
    <dependency>
        <groupId>org.apache.ws.commons.axiom</groupId>
        <artifactId>axiom-dom</artifactId>
        <version>1.2.8</version>
    </dependency>
    <dependency>
        <groupId>axis</groupId>
        <artifactId>axis-wsdl4j</artifactId>
        <version>1.5.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.rampart</groupId>
        <artifactId>rampart-core</artifactId>
        <version>1.4</version>
    </dependency>
    <dependency>
        <groupId>org.apache.rampart</groupId>
        <artifactId>rampart</artifactId>
        <version>1.4</version>
    </dependency>
    <dependency>
        <groupId>org.apache.rampart</groupId>
        <artifactId>rahas</artifactId>
        <version>1.4</version>
    </dependency>
    <dependency>
        <groupId>org.apache.geronimo.specs</groupId>
        <artifactId>geronimo-jaxws_2.1_spec</artifactId>
        <version>1.0</version>
    </dependency>
    <dependency>
        <groupId>commons-lang</groupId>
        <artifactId>commons-lang</artifactId>
        <version>2.4</version>
    </dependency>
</dependencies>
<reporting>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.4</version>
        </plugin>
    </plugins>
</reporting>

$ mvn -version
Apache Maven 2.2.1 (r801777; 2009-08-06 13:16:01-0600)
Java version: 1.6.0_17
Java home: c:\Program Files\Java\jdk1.6.0_17\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows xp" version: "5.1" arch: "x86" Family: "windows"


[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test {execution: default-test}]
Downloading: http://XXXXXX/artifactory/repo/junit/junit/3.8.1/junit-3.8.1.jar
118K downloaded  (junit-3.8.1.jar)
[INFO] Surefire report directory: c:\dev\LocationService\IntegrationTest\target\
surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.625 sec

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1 minute 4 seconds
[INFO] Finished at: Thu Jan 07 14:30:18 MST 2010
[INFO] Final Memory: 31M/127M
[INFO] ------------------------------------------------------------------------

Answer 5

mvn -X helped me to reveal the following:

...
[INFO] [surefire:test {execution: default-test}]
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG]   org.apache.maven.surefire:surefire-booter:jar:2.4.3:runtime (selected for runtime)
[DEBUG]     org.apache.maven.surefire:surefire-api:jar:2.4.3:runtime (selected for runtime)
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/apache/maven/surefire/surefire-booter/2.4.3/surefire-booter-2.4.3.jar
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/apache/maven/surefire/surefire-api/2.4.3/surefire-api-2.4.3.jar
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG]   org.testng:testng:jar:jdk15:5.8:test (selected for test)
[DEBUG]     junit:junit:jar:3.8.1:test (selected for test)
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/testng/testng/5.8/testng-5.8-jdk15.jar
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG] Retrieving parent-POM: org.apache.maven.surefire:surefire-providers:pom:2.4.3 for project: null:surefire-testng:jar:null from the repository.
[DEBUG] Adding managed dependencies for unknown:surefire-testng
[DEBUG]   org.apache.maven.surefire:surefire-api:jar:2.4.3
[DEBUG]   org.apache.maven.surefire:surefire-booter:jar:2.4.3
[DEBUG]   org.codehaus.plexus:plexus-utils:jar:1.5.1
[DEBUG]   org.apache.maven.surefire:surefire-testng:jar:2.4.3:test (selected for test)
[DEBUG]     org.apache.maven:maven-artifact:jar:2.0:test (selected for test)
[DEBUG]       org.codehaus.plexus:plexus-utils:jar:1.0.4:test (selected for test)
[DEBUG]     junit:junit:jar:3.8.1:test (selected for test)
[DEBUG]     org.testng:testng:jar:jdk15:5.7:test (selected for test)
[DEBUG]     org.apache.maven.surefire:surefire-api:jar:2.4.3:test (selected for test)
...
[DEBUG] Test Classpath :
...
[DEBUG]   /home/mindas/.m2/repository/junit/junit/4.7/junit-4.7.jar

So it seems that the problem was coming from testng jar requiring JUnit v3.8.1. Even though Test Classpath was set to depend on JUnit 4, it was too late.

testng dependency was located in my POM:

<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>5.8</version>
  <scope>test</scope>
  <classifier>jdk15</classifier>
</dependency>

Immediately after I have commented it out, tests started to execute.

Lessons learned:

• mvn dependency:tree is not always enough, mvn -X is a friend.
• surefire is not made for developer heaven (I have realized this while looking at project JIRA reports). This is especially true as there are no other alternatives if you use Maven.

Thanks everybody for your help. Unfortunately there is no way to split answer points between Pascal and Kaleb, but Kaleb's advice to use mvn -X helped me to get on the right track so correct answer points go to him.