I came up with another solution to rerun just failed test using maven & cucumber.
1) Record test failures using a RunNotifier
public class RerunningCucumber extends Cucumber {
private final String className;
@SuppressWarnings("rawtypes")
public RerunningCucumber(Class clazz) throws InitializationError, IOException {
super(clazz);
className = clazz.getSimpleName();
}
@Override
public void run(RunNotifier notifier) {
notifier.addListener(new RunListener(){
public void testFailure(Failure failure) throws Exception {
Throwable error = failure.getException();
if (error instanceof AssertionError){
//Nothing. This is a normal failure. Continue
return;
}
//No! A wild exception has appeared!
//Let's run this test again.
RerunningCucumber.addFile(className);
}
});
super.run(notifier);
}
private static final String filename = "target/rerun.properties";
private static final Set<String> addedClasses = new HashSet<String>();
public static synchronized void addFile(String className) throws IOException{
//First find the file
if (addedClasses.contains(className)){
return;
}
File file = new File(filename);
if (!file.exists()){
//Need to create the file
PrintWriter writer = new PrintWriter(file, "UTF-8");
writer.print("retryclasses=**/"+className+".class");
writer.close();
}
else {
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(file, true)));
out.print(",**/"+className+".class");
out.close();
}
addedClasses.add(className);
}
}
2) Use custom class as a runner for the cucumber tests.
This will run the tests, and whenever there is a failure, output the failed class to a file. Trick is to keep features short and create a lot of test classes to avoid repeating tests.
@RunWith(RerunningCucumber.class)
@CucumberOptions(features = {"classpath:features/testFeature.feature}, format = {
"html:target/cucumber-html-report/testFeature.html",
"json:target/cucumber-json-report/testFeature.json"},
tags = {"@testFeature"})
public class RunTestFeature {
}
3) Add a Rerun profile to maven.
This does three things: 1) it loads the failed classes into memory, 2) cleans JUST the failed classes properties file, and 3) reruns ONLY the failed tests as loaded from the properties file:
<profile>
<id>retry</id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<!-- Associate the read-project-properties goal with the initialize
phase, to read the properties file. -->
<execution>
<phase>pre-clean</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>target/rerun.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-clean-plugin</artifactId>
<version>2.6.1</version>
<configuration>
<filesets>
<fileset>
<directory>target</directory>
<includes>
<include>rerun.properties</include>
</includes>
</fileset>
</filesets>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target>
<echo>Retrying the following classes: "${retryclasses}"</echo>
</target>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<includes>
<include>${retryclasses}</include>
</includes>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
4) Usage
First test run:
mvn clean test
Next test runs:
mvn clean test -Pretry
mvn clean test -Pretry
mvn clean test -Pretry
...
You can repeat as many times as you want until there are no errors.
