maven surefire: when are we forced to set reuseForks=false?

Question

In my project when I set reuseForks=true then i have to increase the forkCount to number of test classes. Otherwise, It is throwing illegalargument exception. Also, If I set reuseForks=false then it also work fine.

Currently I have following configuration because number of test classes are less than 10.

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.18.1</version>
            <configuration>
                <reuseForks>true</reuseForks>
                <forkCount>10</forkCount>
            </configuration>
        </plugin>

How can I keep reuseFork=true and forkCount=1.

EDIT: StackTrace on reuseFork=true and forkCount=1

    checkForReturnEventsPresent on checkForReturnEventsPresent(com.eras.senders.OMSReturnEventDataSenderTest)(com.eras.senders.OMSReturnEventDataSenderTest)  Time elapsed: 0.014 sec  <<< FAILURE!
java.lang.IllegalArgumentException: null
    at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
    at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:212)
    at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:108)
    at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)

Clackmannan

Can you please add the full error or stack trace for the IllegalArgumentException that you saw? I cannot reproduce this locally. – Mandeville

@ChrisNauroth I have added the stacktrace on reuseFork=true and forkCount=1. – Clackmannan

Answer 1

In my project when I set reuseForks=true then i have to increase the forkCount to number of test classes. Otherwise, It is throwing illegalargument exception.

From the stack trace, it appears something inside your code is causing the exception to be thrown, but only when forks are reused. This implies that the different tests are somehow not isolated from each other, and therefore running one after the other within the same test process violates some assumptions. For example, perhaps one test initializes some global state like a singleton, and then that global state isn't correct for the next test run within that process.

Also, If I set reuseForks=false then it also work fine.

By setting reuseForks to false, any problems related to mutating global state like this get bypassed. The process gets torn down at the end of the test run, and a new process with fresh state gets started for the next test run.

At this point, the path forward is highly dependent on the specifics of your codebase and how its tests are implemented. I see 2 options:

1. Debug your tests looking for problems of global state like I described. Attaching a breakpoint right before this exception would probably give you a strong hint.
2. If you already know that the codebase relies on some global state, and it isn't feasible to change that code quickly, then just stick with reuseForks set to false. In many cases, the extra process teardown and startup won't cause a noticeable performance impact for your overall test run.

Mandeville

Even If I have single test case, I have to keep forkCount=2. – Clackmannan