maven-surefire-plugin include/exclude precedence

Question

When using the maven-surefire-plugin and both includes and excludes, which order are they processed in? Furthermore, if you have 3 sets of tests, the first being the base set, the second and third being special cases, can you use profiles to further include/exclude? How will the profile include/exclude settings be merged? For example, I would like to do something like this:

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.12.2</version>
        <configuration>
          <excludes>
            <exclude>/org/mycompany/dataset/test/ExtractProd*.java</exclude> <!-- requires special network connectivity -->
            <exclude>/org/mycompany/dataset/test/LargeDataset*.java</exclude> <!-- requires lengthy processing -->
          </excludes>
        </configuration>
      </plugin>
    </plugins>
  </build>

  <profiles>
    <profile>
      <id>connectedToProdNetwork</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <includes>
                <include>/org/mycompany/dataset/test/ExtractProd*.java</include>
              </includes>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>
    <profile>
      <id>runForAsLongAsYouNeed</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <includes>
                <include>/org/mycompany/dataset/test/LargeDataset*.java</include>
              </includes>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>

And then be able to run like this:

mvn package -P connectedToProdNetwork

or

mvn package -P runForAsLongAsYouNeed

or

mvn package -P connectedToProdNetwork,runForAsLongAsYouNeed

---- UPDATE -----

Using mvn help:effective-pom -P [profileA] I was able to determine that if i specify a single profile, the resulting effective pom will be:

        <configuration>
          <includes>
            <include>[includeFromProfileA]</include>
          </includes>
          <excludes>
            <exclude>/org/mycompany/dataset/test/ExtractProd*.java</exclude> <!-- requires special network connectivity -->
            <exclude>/org/mycompany/dataset/test/LargeDataset*.java</exclude> <!-- requires lengthy processing -->
          </excludes>
        </configuration>

And if I supply more than one profile, mvn help:effective-pom -P [profileA],[profileB]:

        <configuration>
          <includes>
            <include>[includeFromProfileAOrBSeeminglyArbitraryChoice]</include>
          </includes>
          <excludes>
            <exclude>/org/mycompany/dataset/test/ExtractProd*.java</exclude> <!-- requires special network connectivity -->
            <exclude>/org/mycompany/dataset/test/LargeDataset*.java</exclude> <!-- requires lengthy processing -->
          </excludes>
        </configuration>

And lastly, if I add the attribute combine.children="append" to the <includes> element of the profile configurations, and supply both profiles, mvn help:effective-pom -P [profileA],[profileB]:

        <configuration>
          <includes combine.children="append">
            <include>[includeFromProfileA]</include>
            <include>[includeFromProfileB]</include>
          </includes>
          <excludes>
            <exclude>/org/mycompany/dataset/test/ExtractProd*.java</exclude> <!-- requires special network connectivity -->
            <exclude>/org/mycompany/dataset/test/LargeDataset*.java</exclude> <!-- requires lengthy processing -->
          </excludes>
        </configuration>

However, now that each file is specified as both an <include> and an <exclude>, what happens?

---- UPDATE 2 ----

Actually running a build with this configuration:

<configuration>
  <includes>
    <include>**/TestA.java</include>
  </includes>
  <excludes>
    <exclude>**/TestA.java</exclude>
  </excludes>
</configuration>

Does NOT run TestA, so it appears an <exclude> will overpower an <include>. Note that for completeness sake, I did reverse the order and put <excludes> before <includes> but the behavior did not change. If anyone can find somewhere short of the source code where this behavior is outlined, I would be happy to give them the answer...

Titulary

Exclude overrides includes because typically people include a larger set than they need and just need some taken out. This generally makes for a shorter list and less work. – Saboteur

@Steven, yeah, that matches what i experienced during testing. Do you know of anywhere this is officially stated so that I would know they wont just change this behavior in the future? Anyway, thanks man. – Titulary

I think the experimental approach is often the best one to be sure :) - still: the pom reference gives some hint on the order: maven.apache.org/guides/introduction/… - for ex. inherited executions run first. It is often quite common that plugins use the excludes / includes in the order of the pom. So you can include something special while excluding all or include all and exclude something special. In doubt the order in the pom will be followed (in most cases and plugins) – Impersonal

Answer 1

I couldn't find official documentation about surefire plugin, but indeed the exclude-override-include is a common approach and is also applied by Maven in other similar contexts, like for resources.

The only official an related info (I found) comes from the official Maven POM Reference documentation, here:

includes: A set of files patterns which specify the files to include as resources under that specified directory, using * as a wildcard.

excludes: The same structure as includes, but specifies which files to ignore. In conflicts between include and exclude, exclude wins.

NOTE: I added the final bold formatting on the interesting statement.

So more than probably the same approach is used across official maven plugins (in general, all the plugins having the org.apache.maven.plugins groupId and maven- prefix as artifactId).

Rutty

Great finding... not sure i should mark it as the answer as it is not technically definitive, but definitely userful information. – Titulary

oh this is a pity, I wish you could specify include to be the winner – Tampon

Answer 2

Have you tried using JUnit categories?

Using that approach, you could give tests a number of different categories, and exclude/include them using that, rather than the class name. This would be a more extensible approach.

Henceforward