Surefire is not picking up Junit 4 tests

Question

For some reason I cannot get Maven 2 Surefire plugin to execute JUnit 4 test class.

public class SimpleTest {
  @org.junit.Test
  public void simple() {
     System.out.println("foo");
  }
}

However if I change this class to be JUnit-3 like, such as

public class SimpleTest extends junit.framework.TestCase {
  public void testBar() {
     System.out.println("bar");
  }

  @org.junit.Test
  public void simple() {
     System.out.println("foo");
  }
}

then it gets executed. Here's what I've done:

• verified Maven version: Apache Maven 2.2.1 (r801777; 2009-08-06 20:16:01+0100)
• verified Surefire version: followed this advice
• verified Surefire version: checked Surefire jars in my ~/.m2/repository/org/apache/maven/surefire -- all of them are either version 2.4.2 or 2.4.3
• done a mvn dependency:tree | grep junit to ensure I only depend on junit version 4.7

The module I am having this problem at doesn't have JUnit 3 tests.

Is there anything else I am missing?

Epidemiology

Could you post your POM? That would be easier. – Partridgeberry

I can't post entire POM for both legal reasons and length - it's a big project, with hundreds of lines just in the POM. I could probably post relevant part(s), but not sure what exactly can be relevant. – Epidemiology

Well, at least the maven-surefire-plugin configuration (if you have one or if you are inheriting one). Can you say if this problem is specific to one module? Does it happen outside the project? Are you inheriting from a corporate POM? – Tupler

Can you provide detail on what the symptoms of "can't execute" are? – Sanguineous

Answer 1

mvn -X helped me to reveal the following:

...
[INFO] [surefire:test {execution: default-test}]
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG]   org.apache.maven.surefire:surefire-booter:jar:2.4.3:runtime (selected for runtime)
[DEBUG]     org.apache.maven.surefire:surefire-api:jar:2.4.3:runtime (selected for runtime)
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/apache/maven/surefire/surefire-booter/2.4.3/surefire-booter-2.4.3.jar
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/apache/maven/surefire/surefire-api/2.4.3/surefire-api-2.4.3.jar
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG]   org.testng:testng:jar:jdk15:5.8:test (selected for test)
[DEBUG]     junit:junit:jar:3.8.1:test (selected for test)
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/testng/testng/5.8/testng-5.8-jdk15.jar
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG] Retrieving parent-POM: org.apache.maven.surefire:surefire-providers:pom:2.4.3 for project: null:surefire-testng:jar:null from the repository.
[DEBUG] Adding managed dependencies for unknown:surefire-testng
[DEBUG]   org.apache.maven.surefire:surefire-api:jar:2.4.3
[DEBUG]   org.apache.maven.surefire:surefire-booter:jar:2.4.3
[DEBUG]   org.codehaus.plexus:plexus-utils:jar:1.5.1
[DEBUG]   org.apache.maven.surefire:surefire-testng:jar:2.4.3:test (selected for test)
[DEBUG]     org.apache.maven:maven-artifact:jar:2.0:test (selected for test)
[DEBUG]       org.codehaus.plexus:plexus-utils:jar:1.0.4:test (selected for test)
[DEBUG]     junit:junit:jar:3.8.1:test (selected for test)
[DEBUG]     org.testng:testng:jar:jdk15:5.7:test (selected for test)
[DEBUG]     org.apache.maven.surefire:surefire-api:jar:2.4.3:test (selected for test)
...
[DEBUG] Test Classpath :
...
[DEBUG]   /home/mindas/.m2/repository/junit/junit/4.7/junit-4.7.jar

So it seems that the problem was coming from testng jar requiring JUnit v3.8.1. Even though Test Classpath was set to depend on JUnit 4, it was too late.

testng dependency was located in my POM:

<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>5.8</version>
  <scope>test</scope>
  <classifier>jdk15</classifier>
</dependency>

Immediately after I have commented it out, tests started to execute.

Lessons learned:

• mvn dependency:tree is not always enough, mvn -X is a friend.
• surefire is not made for developer heaven (I have realized this while looking at project JIRA reports). This is especially true as there are no other alternatives if you use Maven.

Thanks everybody for your help. Unfortunately there is no way to split answer points between Pascal and Kaleb, but Kaleb's advice to use mvn -X helped me to get on the right track so correct answer points go to him.

Epidemiology

Actually, you should accept your own answer as it is providing the whole solution to your problem:) – Tupler

Answer 2

The Surefire plugin figures out which JUnit provider should be used based upon the classpath. If there are multiple JUnit versions on the classpath, you can either correct the classpath to have only one JUnit version on the classpath (as discussed above), or you can explicitly specify which provider you want to use. For example, specifying the following in your (parent) POM forces using the newest provider (e.g., "surefire-junit47"):

[...]
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.8</version>
  <dependencies>
    <!-- Force using the latest JUnit 47 provider -->
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.8</version>
    </dependency>
  </dependencies>
[...]

Note however that Surefire 2.7 changed the way it's determining which unit test classes are run. The new behavior when using Surefire 2.7 (or later) with JUnit 4 is that any test without a @Test annotation will be skipped automatically. This may be great if you just have JUnit 4 unit tests, but if you have a combination of JUnit 3 and 4 unit tests, using the "surefire-junit47" provider will not work correctly. In such cases, its best to explicitly choose the "surefire-junit4" provider:

[...]
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.8</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <!-- Use the older JUnit 4 provider -->
      <artifactId>surefire-junit4</artifactId>
      <version>2.8</version>
    </dependency>
  </dependencies>
[...]

Industrial

Answer 3

I don't know what you mean by "can't execute," but does it help to explicitly set the includes used by the maven-surefire-plugin?

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.4.3</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
        </includes>
    </configuration>
</plugin>

Also, does running maven with the -X flag provide any useful information?

Sanguineous

Thanks for your tip to use -X, this helped me to reveal the problem (see my own answer) – Epidemiology

Answer 4

One more possible cause can be this bug (closed with "Won't fix"): https://issues.apache.org/jira/browse/SUREFIRE-587

Short summary: Tests extending TestCase (but not using annotations) will not be picked up if their name does not end with "Test".

Becalmed

No, this wasn't the case for me (see my own answer for the real cause). Anyway, thanks for the tip - might be helpful for other people. – Epidemiology

Yeah, I saw your solution. I had the same question, and for me the solution was this bug. Thats why i shared it here. – Mellophone

Thanks! I spent several hours debugging because of this "feature". – Cheddite

Updated link for SUREFIRE-587 : issues.apache.org/jira/browse/SUREFIRE-587 Thanks for the hint Gábor. – Capitally

Answer 5

In case you have JUnit 5 (import org.junit.jupiter.api.Test;) in your dependencies and you're trying to run some JUnit4 (import org.junit.Test;) tests, make sure you include following dependency

<dependency>
  <groupId>org.junit.vintage</groupId>
  <artifactId>junit-vintage-engine</artifactId>
  <scope>test</scope>
</dependency>

Tinge

Noted: I upgrade my service from 2.3.5.RELEASE to 2.6.7 - Add dependency junit 4.13.2 and junit-vintage-engine - Swagger 2 change to use springdoc-openapi-ui 1.6.8 - management.info.env.enabled=true for path /info – Encrata

This helped me. We have a confusing mix of junit4 and 5 tests. – Sheathing

Answer 6

For some poor soul out there who is wondering why Maven doesn't pick up JUnit tests.

I have both JUnit and TestNG as dependecies. But I want failsafe to run my functional tests using TestNG and surefire to run my unit tests using JUnit.

However, I found that surefire was attempting to run my unit tests using TestNG and found nothing to run. My JUnit tests were skipped.

Later I came across this Maven issue and configured surefire to run only "JUnit" tests like this:

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <properties>
      <property>
        <name>junit</name>
        <value>true</value>
      </property>
    </properties>
  </configuration>
</plugin>

Hope this helps someone.

Aspen

This particular solution helped me !! – Odysseus

Answer 7

For the benefit of Googlers, when I had this issue it was because I'd included a PowerMock dependency that pulled in TestNG, causing no [TestNG] tests to be detected by SureFire.

I used the m2eclipse "Dependency Hierarchy" tab of the POM editor to find the dependency and right-clicked to generate an exclusion (see XML below).

For completeness (and for those not using m2eclipse) here's the XML that excludes the dependency - I only came across this feature of Maven by seeing these tags generated automatically:

<dependency>
  <groupId>org.powermock</groupId>
  <artifactId>powermock-mockito-release-full</artifactId>
  <version>1.4.9</version>
  <classifier>full</classifier>
  <exclusions>
    <exclusion>
      <artifactId>powermock-module-testng</artifactId>
      <groupId>org.powermock</groupId>
    </exclusion>
  </exclusions>
</dependency>

(In my case, excluding "powermock-module-testng" was sufficient, but you could exclude TestNG directly if it's coming in from somewhere else.)

Hennahane

Thank you so much for this tip. I had the same issue and it was because of PowerMock. Again, big help as without this post I would have been looking in the wrong direction for some time. – Pargeting

Answer 8

1.) Include following surefire plugin in pom.xml

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
</configuration>
</plugin>

2.)By default surefire plugin picks Test class from package :- src/test/java/....

So go to Build Path and include test folder in classpath like below :

3.) Go to --> Run As -->Maven Test

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.081 
s - in com.bnym.dcm.api.controller.AccountControllerTest
[INFO] Running com.bnym.dcm.api.controller.DCMApiControllerTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - 
in com.bnym.dcm.api.controller.DCMApiControllerTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ----------------------------------------------------------------------
--
[INFO] BUILD SUCCESS

Clarence

Answer 9

One small change helped me Funnily!!!

I changed the class name from MyClassTest to TestMyClass, I got this idea after veryfing my parent POM.xml contains below line

<test.include.pattern> **/Test*.java <test.include.pattern/>

Remark

Answer 10

The verification that you've done are good, especially checking that you are using version 2.3+ of the surefire plugin (by default, you'll get version 2.4.3 with maven 2.1 super POM so this should be ok) and checking that you not pulling the junit-3.8.1.jar dependency transitively.

Now, just to validated that this is not a "global problem" (I don't think so TBH), could you create a project from scratch, for example by running:

mvn archetype:create -DgroupId=com.mycompany.app -DartifactId=maven-junit4-testcase

Then update the junit dependency:

<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.7</version>
  <scope>test</scope>
</dependency>

And configure the compiler level for 1.5+

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <source>1.5</source>
    <target>1.5</target>
  </configuration>
</plugin>

Finally put your SimpleTest.java next to AppTest.java and run mvn test.

If running mvn test works fine for that project (and I'm expecting it to run without problem), could you please update your question with the POM configuration you're using (from the project having troubles)?

Tupler

I did what you suggested and indeed both tests (AppTest and my test were executed). Will try to dig this deeper tomorrow and/or post the relevant POM bits. Thanks a lot Pascal! Your method could benefit others while debugging same problem, too. Btw, can you modify your plugin mvn code - you forgot to close plugin tag at the end. – Epidemiology

Debugging shows it pulls junit-3.8.1.jar transitively. How do I override it. – Brittan

Answer 11

Had a similar issue when trying to run integration tests. I had an old version of the surefire plugin which was trying to run TestNG and not jUnit. I changed the version number in the pom to 2.20 and it worked.

Ejaculate

Answer 12

If you're using Maven, the filename matters.

By default, test filenames need to either start or end with "Test" or "Tests", or they will be silently ignored when searching for tests.

https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html

The

Answer 13

Have you configured your maven-compile-plugin for the correct compiler level, like:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <source>1.5</source>
    <target>1.5</target>
  </configuration>
</plugin>

Otherwise maven will have trouble with annotations

Partridgeberry

Yes, all modules have these settings. Thanks for your input, though! – Epidemiology

The test would fail without 1.5+ compiler level, but it would be picked up. – Tupler

I know that it's just wild guessing - I never had trouble with Junit4 tests in maven... – Partridgeberry

Me neither. I'm really wondering what could cause this weird behavior. – Tupler