Skipping tests in some modules in Maven

Question

I would like my Maven builds to run most unit tests. But there are unit tests in one project which are slower and I'd like to generally exclude them; and occasionally turn them on.

Question: How do I do this?

I know about -Dmaven.test.skip=true, but that turns off all unit tests.

I also know about skipping integration tests, described here. But I do not have integration tests, just unit tests, and I don't have any explicit calls to the maven-surefire-plugin. (I am using Maven 2 with the Eclipse-Maven plugin).

Cordwainer

Answer 1

What about skipping tests only in this module ?

In the pom.xml of this module:

<project>
  [...]
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.4.2</version>
        <configuration>
          <skipTests>true</skipTests>
        </configuration>
      </plugin>
    </plugins>
  </build>
  [...]
</project>

Eventually, you can create a profile that will disable the tests (still the pom.xml of the module) :

<project>
  [...]
  <profiles>
    <profile>
      <id>noTest</id>
      <activation>
        <property>
          <name>noTest</name>
          <value>true</value>
        </property>
      </activation>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.4.2</version>
            <configuration>
              <skipTests>true</skipTests>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>
  [...]
</project>

With the latter solution, if you run mvn clean package, it will run all tests. If you run mvn clean package -DnoTest=true, it will not run the tests for this module.

Ammeter

Thanks that worked. The first code snippet skips the test; I may later use your further suggestion to define another profile. My confusion was in the fact that my pom was invoking surefire implicitly. There was no mention of the surefire plugin in my pom.xml. Nonetheless, the code to configure the surefire plugin correctly did so. – Cordwainer

For others, another option is to skip the whole module until you want to build and verify it: https://mcmap.net/q/162355/-how-to-exclude-a-module-from-a-maven-reactor-build – Response

And here is how to create maven profile that will run tests only in case it's activated: #34334757 – Waterbuck

How do you do this for all but a few modules in the project? – Anserine

Can someone explain to me what is the use of noTest here? – Octarchy

Answer 2

I think this is easier, and also has the benefit of working for non-surefire tests (in my case, FlexUnitTests)

<profile>
   <id>noTest</id>
    <properties>
       <maven.test.skip>true</maven.test.skip>
    </properties>
 </profile>

Boling

This won't skip testing here, with or without -DnoTest=true – Stainless

Thats cause you should use -PnoTest instead of -DnoTest=true – Hello

Setting maven.test.skip to true also has the effect of skipping test compilation, where as skipTests still compiles the tests but doesn't run them. – Finical

Answer 3

If you have a large multi-module project and you would like to skip tests only in certain modules without the need to change each of the module pom.xml file with custom configuration and profiling, you could add the following to the parent pom.xml file:

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <version>1.12</version>
            <executions>
                <execution>
                    <id>regex-property</id>
                    <goals>
                        <goal>regex-property</goal>
                    </goals>
                    <configuration>
                        <name>maven.test.skip</name>
                        <value>${project.artifactId}</value>
                        <regex>(module1)|(module3)</regex>
                        <replacement>true</replacement>
                        <failIfNoMatch>false</failIfNoMatch>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>
<modules>
    <module>module1</module>
    <module>module2</module>
    <module>module3</module>
</modules>

Thanks to the build-helper-maven-plugin you would actually dynamically check whether you are in a certain module or not during the build, via the project.artifactId property (pointing at each artifactId module during the build), the regex would then seek matching for certain values (the module names for which you want to skip tests) and populated the maven.test.skip property accordingly (setting it to true).

In this case, tests will be skipped for module1 and module3 while running properly for module2, that is, as expressed by the regex.

The advantage of this approach is to have it dynamic and centralized (in the parent pom.xml) hence better for maintenance: you could add or remove modules at any time simply by changing the simple regex above.

Obviously, if this is not the default behavior of the build (recommended case), you could always wrap the snippet above in a maven profile.

---

You could also go further and have dynamic behavior based on your input:

<properties>
    <test.regex>none</test.regex>
</properties>

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <version>1.12</version>
            <executions>
                <execution>
                    <id>regex-property</id>
                    <goals>
                        <goal>regex-property</goal>
                    </goals>
                    <configuration>
                        <name>maven.test.skip</name>
                        <value>${project.artifactId}</value>
                        <regex>${test.regex}</regex>
                        <replacement>true</replacement>
                        <failIfNoMatch>false</failIfNoMatch>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Here we are actually replacing the regex value with a property, test.regex, with default value to none (or whatever would not match any module name or, also, the default skipping matchings required).

Then from command line we could have

mvn clean test -Dtest.regex="(module1)" > will skip tests only for module1
mvn clean test -Dtest.regex="(module1)|(module2)" > will skip tests on module1 and module2
mvn clean test -Dtest.regex="(module1)|(module2)|(module3)" > will skip the three module tests
mvn clean test -Dtest.regex=".+" > will skip all module tests
mvn clean test > would not skip anything (or fall back on default behavior)

That is, then at runtime you decide, without any need to change the pom.xml file or activating any profile.

Aweless

The only way I managed to execute this was by calling the validate phase explicitly. This didn't work for me when using another phase, like package or install. Looked promising though. – Ratify

I can't this to work in any case, unfortunately. When the regex doesn't match, it sets maven.test.skip to none instead of leaving it alone, and whether it matches or doesn't match maven-surefire-plugin completely ignores it. – Mestas

Answer 4

Using Surefire Plugin 2.19 you can simply exclude the tests you don't want using regular expressions:

mvn '-Dtest=!%regex[.*excludedString.*]' test

The above command will exclude all the tests that contain excludedString.

NB1 If double quotation mark(") is used instead of apostrophe(') the command will not be interpreted properly and will produce unexpected results. (Tested using bash 3.2.57)

NB2 Particular attention should be paid to projects in which multiple version of the surefire plugin is used. Versions of surefire older than 2.19 will not execute any tests because they do not support regular expressions.

Version management(it might be a good idea to add this in the parent pom file):

<build>
  <pluginManagement>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.19.1</version>
      </plugin>
    </plugins>
  </pluginManagement>
</build>

Examples of build commands that skip tests: https://artbcode.wordpress.com/2016/11/28/how-to-skip-a-subset-of-the-unit-tests/

Coastwise

Any idea how to skip multiple patterns? – Clariceclarie

-Dtests doesn't play nicely with surefire it seems. The following causes ALL my tests to get skipped: mvn '-Dtest=!%regex[.*DeviceLogServiceTest.*]' --quiet install -Dsurefire.useFile=false -Dsurefire.printSummary=false -Dmaven.test.skip=false – Clariceclarie

I'm using mvn install "-Dtest=!%regex[.*excludedString.*]" -Dsurefire.failIfNoSpecifiedTests=false to avoid testing modules with Java packages name containing excludedString. – Darsie

Answer 5

I had a slightly different need from this question that may prove helpful. I wanted to exclude from the command line a few different tests from different packages, so a single wildcard would not do it.

I found in the Maven Failsafe documentation rules for exclusions that you can specify a comma-separated list of either regex or wildcard exclusions: https://maven.apache.org/surefire/maven-failsafe-plugin/examples/inclusion-exclusion.html

So my pomfile looked like this:

<excludes>
    <exclude>${exclude.slow.tests}</exclude>
</excludes>

and my command line included this:

mvn install "-Dexclude.slow.tests=**/SlowTest1.java, **/package/ofslowtests/*.java, **/OtherSlowTest.java"

For me the key ingredient was getting a bunch of tests into one maven property in a single exclude statement.

Divergence

Answer 6

For maven-surefire-plugin I would suggest a bit different approach with environment variables or properties.

In my case I need to use parameter testSourceDirectory. But this parameter is not supported by profiles, so I cannot use them. And using profiles means too many new lines of code. It makes pom.xml more complicated.

Environment variable:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <skipTests>${env.SKIP_UNIT_TESTS}</skipTests>
    </configuration>
</plugin>

Set it to true if you need to skip tests for the module. If it's not set, parameter skipTests will be ignored, so tests will run.

Properties:

<properties>
    <skipUnitTests>false</skipUnitTests>
</properties>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <skipTests>${skipUnitTests}</skipTests>
            </configuration>
        </plugin>
    </plugins>
</build>

To skip tests, run mvn with -DskipUnitTests=true

Erinaceous