Maven runs unit tests twice with clover plugin

Question

I have this plugin code in my pom.xml. If I remove this plugin then Maven won't run unit tests twice. I just wanted to know which part of this plugin makes the unit tests run twice.

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <version>3.0.4</version>
    <configuration>
        <licenseLocation>/location/to/clover.license</licenseLocation>
        <generateXml>true</generateXml>
        <generateHtml>true</generateHtml>
    </configuration>
    <executions>
        <execution>
            <phase>generate-sources</phase>
            <goals>
                <goal>instrument</goal>
            </goals>
        </execution>
        <execution>
            <id>main</id>
            <phase>verify</phase>
            <goals>
                <goal>instrument</goal>
                <goal>aggregate</goal>
                <goal>clover</goal>
            </goals>
        </execution>
        <execution>
            <id>site</id>
            <phase>pre-site</phase>
            <goals>
                <goal>instrument</goal>
                <goal>aggregate</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Fujio

Answer 1

The clover plugin forks a life cycle up to the test phase using the instrumented classpath.

Your standard life cycle will run the tests (typically using surefire in the test phase, but there could be multiple executions or other test plugins invoking tests in that phase)

Then clover comes along and asks for them to be run again.

Now for my standard warning:

• Code coverage modifies the byte code that is executed in ways that are legal under the JVM specification.
• Such transformations can result in the addition of synchronisation points while updating the coverage map.
• Such additional synchronisation points can restrict the reordering of operations that the JVM is able to do.
• There are other changes that are not threading related, but rather complex to explain.

If you only ever run your test cases with coverage turned on, you cannot trust a 100% pass rate as coverage can mask bugs

Conversely, if you never run with coverage turned on, any change in JVM architecture can highlight new bugs that you didn't know.

• Test passes with and without coverage turned on => probably ok

• Test passes with coverage turned on, fails when off => beware, bug here

• Test passes with coverage turned off, fails when on => if test is not timing critical then bug here, but if tests run on production environment bug may only show when -server fully optimises the hot paths

• Test fails with and without coverage turned on => bug

"Stephen's golden rule: Just run the damn tests twice"

Disperse

Is this specific to Clover? Or do all instrumentation tools have to do this? – Hernia

Not specific to Clover. My advice is to run the tests twice, but people can and do ignore this advice (at their own peril). JavaCoCo is perhaps the easiest to configure for running the tests once only... with all the risks I mention – Disperse

Answer 2

The clover2:instrument goal you're using works in such way that it forks a parallel build life cycle. Thanks to this all artifacts produced during a build (classes, jars etc) are put in /target/clover directory or have -clover.jar suffix. This ensures that you have a separation between your "normal" code (i.e. not instrumented) and the instrumented one.

If you don't need such separation (for instance you run"mvn test" and not "mvn deploy"), then I suggest using clover2:setup instead of clover2:instrument.

http://docs.atlassian.com/maven-clover2-plugin/latest/setup-mojo.html

Baum