Running Cucumber tests with Junit Categories via Maven

Question

I have a maven project with multiple modules and one common parent module. In this project there are some unit tests run with Junit together with surefire, as well as BDD Cucumber integration tests. I would like to run two separate jobs, one to run all the unit tests, and one for running the BDD/Integration tests. In order to do that, I have annotated my BDD runner classes with a Junit category annotation like so:

@RunWith(Cucumber.class)
@CucumberOptions(
                tags = { "@ATagToBeRun", "~@ATagNotToBeRun","~@ToBeImplemented" },
                dryRun = false, strict = true, 
                features = "src/test/resources/cucumber/testing",
                glue = { "com.some.company.test",
                        "com.some.company.another.test"})
@Category(value = IntegrationTest.class)
public class FlowExecutionPojoTest {
}

I created a Maven profile in the parent pom which uses the maven-surefire-plugin feature that is intended to filter tests base on groups. Here is my maven configuration:

     <dependencies>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>1.2.4</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>1.2.4</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-spring</artifactId>
            <version>1.2.4</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-jvm-deps</artifactId>
            <version>1.0.5</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <profile>
        <id>testJewels</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <modules>
            <module>../my-module</module>
        </modules>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.19.1</version>
                    <configuration>
                        <groups>com.some.company.IntegrationTest</groups>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>

What I expected was that when I run mvn test -PtestJewels the classes annotated with the category contained in the <groups> tag should be executed. In actuality none of the annotated classes executed.

An interesting point to note is that when I used the maven-surefire-plugin version 2.18 this worked, but from version 2.18.1 and on it did not. According to the documentation at the bottom of the page, there was a change in version 2.18.1 regarding categories being inherited but in my case they are being in

Repertory

Can you share your pom dependencies? Did you include cucumber-java, cucumber-junit packages? – Twayblade

@Twayblade sure thing, added it to the question – Repertory

Answer 1

I discovered that there is actually a pull request on the cucumber-jvm repository to fix this specific problem. The problem is caused as a result of the fact that when Junit filters the test runner classes based on the @Category annotation, it also checks all the child classes as well. In the case of a cucumber test that runs .feature files, all the classes that represent the .feature files are checked as well (cucumber converts each .feature file in an instance of the FeatureRunner class). Being that the annotation does not exist on the FeatureRunner class, the tests are filtered out and not run. (Prior to version 2.18.1 the maven-surefire-plugin did not check child classes for annotations and hence this was not a problem.

A workaround that works for the specific setup that I described above, where all the BDD tests are run in a specific module, was to override the <groups> configuration in the child module, like so:

 <build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <groups combine.self="override"></groups>
            </configuration>
        </plugin>
    </plugins>
</build>

This will obviously not work in a different setup, and it is therefore rather unfortunate that the cucumber team has yet to merge the PR that I mentioned above.

Repertory