How to run JUnit tests by category in Maven?

Question

Using JUnit 4.8 and the new @Category annotations, is there a way to choose a subset of categories to run with Maven's Surefire plugin?

For example I have:

@Test
public void a() {
}

@Category(SlowTests.class)
@Test
public void b() {
}

And I'd like to run all non-slow tests as in: (note that the -Dtest.categories was made up by me...).

mvn test -Dtest.categories=!SlowTests // run non-slow tests
mvn test -Dtest.categories=SlowTests // run only slow tests
mvn test -Dtest.categories=SlowTests,FastTests // run only slow tests and fast tests
mvn test // run all tests, including non-categorized

So the point is that I don't want to have to create test suites (Maven just picks up all unit tests in the project which is very convenient) and I'd like Maven to be able to pick the tests by category. I think I just made up the -Dtest.categories, so I was wondering if there's a similar facility I can use?

Wheatworm

See Mixing testng and junit for more details. From maven-surefire-plugin version 2.11 categories are supported – Chute

It is not directly related, but this is a way to compute 'Tests by Category' counter. – Recrudescence

Answer 1

Maven has since been updated and can use categories.

An example from the Surefire documentation:

<plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.11</version>
      <configuration>
        <groups>com.mycompany.SlowTests</groups>
      </configuration>
</plugin>

This will run any class with the annotation @Category(com.mycompany.SlowTests.class)

Fae

Version 2.11 didn't work for me, because Surefire kept ignoring the groups I specified. Upgrading to surefire plugin version 2.12.3 did the trick – Educationist

How does one do multiple configurations for different categories though? Ie the OP wanted to be able to specify a category on the command line, but if the category is specified in the POM, how do you specify one on the command line? – Flavio

Attention! Version 2.11 ignores groups! Provided example under maven.apache.org/surefire/maven-surefire-plugin/examples/… is still incorrect! – Terchie

There is also excludedGroups option, see: maven.apache.org/surefire/maven-surefire-plugin/… – Tonsure

Do i have to use groups tag - if i want want to achieve running specific test at my pom file ? – Explanatory

Answer 2

Based on this blog post - and simplifying - add this to your pom.xml:

<profiles>
    <profile>
        <id>SlowTests</id>
        <properties>
            <testcase.groups>com.example.SlowTests</testcase.groups>
        </properties>
    </profile>
    <profile>
        <id>FastTests</id>
        <properties>
            <testcase.groups>com.example.FastTests</testcase.groups>
        </properties>
    </profile>
</profiles>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.13</version>
            <dependencies>
                <dependency>
                    <groupId>org.apache.maven.surefire</groupId>
                    <artifactId>surefire-junit47</artifactId>
                    <version>2.13</version>
                </dependency>
            </dependencies>
            <configuration>
                <groups>${testcase.groups}</groups>
            </configuration>
        </plugin>
    </plugins>
</build>

then at the command line

mvn install -P SlowTests
mvn install -P FastTests
mvn install -P FastTests,SlowTests

Authoritarian

Version 2.13 of surefire plugin didn't work for me. I got this error: "groups/excludedGroups require TestNG or JUnit48+ on project test classpath", although I am using Junit 4.11. Downgrading surefire plugin to 2.12.2 resolved the error. – Barra

This worked for me, using version 2.17 of Surefire. I had to change the org.apache.maven.surefire:surefire-junit47:2.13 dependency to org.apache.maven.surefire:common-junit48:2.17. – Felony

Nice. with maven-surefire-plugin:2.2.22 you don't need the surefire-junit47 dependency. Of course one can run mvn test -P SlowTests – Angwantibo

Answer 3

I had a similar case where I want to run all test EXCEPT a given category (for instance, because I have hundreds of legacy uncategorized tests, and I can't / don't want to modify each of them)

The maven surefire plugin allows to exclude categories, for instance:

<profiles>
    <profile>
        <id>NonSlowTests</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <excludedGroups>my.category.SlowTest</excludedGroups>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Desma

Hot would you set to run only a specific Group ? – Explanatory

@Francy just run with the corresponding profile, from this example: mvn test -P NonSlowTests – Desma

Can you exclude multiple groups? – Frankly

I guess you can, coma separated, given it's named "excludedGroups" – Desma

Answer 4

You can use

mvn test -Dgroups="com.myapp.FastTests, com.myapp.SlowTests"

But ensure that you configure properly the maven surefire plugin

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.11</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.12.2</version>
    </dependency>
  </dependencies>
</plugin>

See docs in: https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html

Likeness

Answer 5

I lost lot of time on this error "groups/excludedGroups require TestNG or JUnit48+ on project test classpath" because I thought I was using a bad version of junit, or a bad version of the surefire plugin, or a combination that does not fit.

It was none of that: in my project I had a "config" module that was built before the module I wanted to test. This module had no junit dependency -> it had no junit on the classpath...

This mistake may help others...

Gimcrack

Answer 6

For the use-case where you want to ignore certain tests by default and then conditionally run all tests from the command line, this setup will work on JUnit 4.9.

First create the marker interface:

public interface IntegrationTest {}

Annotate tests to be excluded:

@Category(IntegrationTest.class)
public class MyIntegrationTest() {}

Add plugins and profiles to pom.xml:

<plugins>
  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.2</version>
    <!-- Read JUnit categories to be excluded from Maven property -->
    <configuration>
      <excludedGroups>${test.excluded.groups}</excludedGroups>
    </configuration>
  </plugin>
</plugins>
  
<profiles>
  <profile>
    <id>Default</id>
    <!-- Set profile to be active by default -->
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <properties>
      <test.excluded.groups>com.example.IntegrationTest</test.excluded.groups>
    </properties>
  </profile>
  <profile>
    <id>AllTest</id>
    <!-- Set groups property to blank value which will match nothing -->
    <properties>
      <test.excluded.groups></test.excluded.groups>
    </properties>
  </profile>
</profiles>

When running tests as usual from command line, integration test will be excluded:

mvn test

All tests including integration tests can be activated with the corresponding profile:

mvn test -P AllTest

Kerrin

Answer 7

I was having issue running test with categories ex: @Category(com.mycompany.SlowTests.class) when running the test via mvn test -Dgroups=com.mycompany.SlowTests.class

I discovered that tests in a classes without the word Test in their name would not run. After adding the word Test to the class the the tests in the class ran.

Webbed

Answer 8

Not exactly the same thing but using surefire plugin, test classes can be chosen based on file name. You are not using Junit Categories though.

An example for running just DAO tests.

<executions>
  <execution>
     <id>test-dao</id>
        <phase>test</phase>
          <goals>
             <goal>test</goal>
        </goals>
          <configuration>
             <excludes>
                <exclude>none</exclude>
            </excludes>
            <includes>                  
                <include>**/com/proy/core/dao/**/*Test.java</include>
            </includes>
        </configuration>
  </execution>

http://maven.apache.org/plugins/maven-surefire-plugin/examples/inclusion-exclusion.html

Intercolumniation

Answer 9

Junit 5 allows you to use the @Tag annotation. More info about that here: https://www.baeldung.com/junit-filtering-tests

I find it looks a little cleaner:

@Tag("SlowTests")
@Test
public void b() {
}

Vickers