Running integration tests with Cobertura Maven plugin

Question

I am having trouble getting the Cobertura plugin to run integration tests in Maven. The closest answer to this question I have found is http://jira.codehaus.org/browse/MCOBERTURA-86. However, the issue remains an open bug. I tried the configuration suggested by Stevo on 03/Apr/09, it didn't work.

My POM

<reporting>
    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>cobertura-maven-plugin</artifactId>
            <version>2.3-SNAPSHOT</version>
            <reportSets>
            <reportSet>
                <reports>
                    <report>cobertura-integration</report>
                </reports>
            </reportSet>
            </reportSets>               
        </plugin>   
    </plugins>
</reporting>

which is by the way exactly the same as the configuration fragment provided by Stevo.

Quillan

How do you run your ITs? Using a separate module? Using maven-failsafe-plugin? In a separate profile? – Rowenarowland

where have you placed your integration tests (i.e. src/it/java or src/test/java) and how do try to execute the tests ? – Workout

Answer 1

From my opinion, the cobertura maven plugin has two big disadvantages. It has no report only goal, all unit tests will run beside surefire again. It creates the code coverage for unit tests only .

I am using the JaCoCo maven plugin now. JaCoCo reuses the surefire and/or failsafe reports to create a code coverage from unit and/or integration test. Furthermore JaCoCo has a good Jenkins integration. Here is an example where JaCoCo uses surefire unit tests and failsafe integration tests.

    <build>
    <plugins>
        <!-- Unit tests -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.16</version>
        </plugin>

        <!-- Integration tests -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.16</version>
            <executions>
                <execution>
                    <phase>integration-test</phase>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

        <!--
            The JaCoCo plugin generates a report about the test coverage. In contrast to the cobertura plugin
            JaCoCo can be configured to generate code coverage for integration tests. Another advantage of JaCoCo
            is that it reports only, cobertura executes all unit tests again (beside the surefire plugin).
        -->
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <version>0.6.4.201312101107</version>
            <executions>
                <execution>
                    <id>jacoco-prepare-agent</id>
                    <goals>
                        <goal>prepare-agent</goal>
                    </goals>
                </execution>
                <execution>
                    <id>jacoco-prepare-agent-integration</id>
                    <goals>
                        <goal>prepare-agent-integration</goal>
                    </goals>
                </execution>
                <execution>
                    <id>jacoco-report</id>
                    <goals>
                        <goal>report</goal>
                    </goals>
                </execution>
                <execution>
                    <id>jacoco-integration</id>
                    <goals>
                        <goal>report-integration</goal>
                    </goals>
                </execution>
                <execution>
                    <id>jacoco-check</id>
                    <goals>
                        <goal>check</goal>
                    </goals>
                    <configuration>
                        <rules />
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Trappist

Nice tip. After a bit of looking I also figured out how to create a code coverage report that combines the output of the unit and integration tests with JaCoCo, using the merge goal. – Busk

I have not figured out how to make JaCoco ignore trivials. Do you know how? – Yolandoyolane

Use ant with maven-antrun-plugin, see my answer. Also, JaCoCo's JVM agents are not always a possibility. Stopping the JVM (I think this is the only way to get the coverage reported) is not always what you want or can do. Cobertura has a coberturaFlush.war to flush the coverage data and keep the JVM running. – Grilse

Answer 2

Try to configure a execution phase for that plugin like

 <build>
   <plugins>
     <plugin>
       <groupId>org.codehaus.mojo</groupId>
         <artifactId>cobertura-maven-plugin</artifactId>
         <version>2.5.1</version>
         <configuration>
           <formats>
             <format>xml</format>
           </formats>
         </configuration>
         <executions>
           <execution>
             <id>cobertura-check</id>
             <phase>verify</phase>
             <goals>
               <goal>cobertura</goal>
             </goals>
           </execution>
         </executions>
       </plugin>

That way it works like a charm for me.

Sherer

Answer 3

After some research found a working config listed from http://jira.codehaus.org/browse/MCOBERTURA-86 Make sure to invoke this with

 mvn clean deploy -PbuildWithIntegrationTestCoverage

        <profile>
        <!-- Build with IntegrationTestcoverage => instrument classes with cobertura before integrationtests starts. -->
        <id>buildWithIntegrationTestCoverage</id>
        <activation>
            <property>
                <name>buildWithIntegrationTestCoverage</name>
                <value>true</value>
            </property>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>cobertura-maven-plugin</artifactId>
                    <version>2.5.2</version>
                    <executions>
                        <execution>
                            <id>instrument-classes</id>
                            <phase>package</phase>
                            <goals>
                                <goal>instrument</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
                <!-- Add cobertura as dependency to the jetty-plugin (required for instrumented classes) -->
                <plugin>
                    <groupId>org.mortbay.jetty</groupId>
                    <artifactId>maven-jetty-plugin</artifactId>
                    <dependencies>
                        <dependency>
                            <groupId>org.codehaus.mojo</groupId>
                            <artifactId>cobertura-maven-plugin</artifactId>
                            <version>2.5.2</version>
                            <type>jar</type>
                        </dependency>
                    </dependencies>
                </plugin>
            </plugins>
        </build>
    </profile>

Vicechancellor

Answer 4

For anyone stumbling on this question through Google - cobertura-maven-plugin started supporting integration tests in version 2.7, which was published in 2015.

Quoting their website:

Up to version 2.6 there were only one report available: cobertura, which gave you a coverage report for your unit tests. Since there was only one, you didn't have to configure it in any way.

Starting with version 2.7 a new report was added: cobertura-integration-test, which gives you a coverage report for your integration tests. [..]

Both reports are enabled by default. If you want the old behaviour with only a coverage report for your unit tests, you need to configure the plugin ...

If you, like I did, run cobertura report using mvn cobertura:cobertura, you'll need to do mvn cobertura:cobertura-integration-test to get integration tests as well. You can check the details from its manual page.

Piston

Answer 5

I found that using the maven-antrun-plugin for multi-module projects including integration and UI tests is the best solution. I used this post to get the Ant goals down and went from there. We now have code coverage reports per module and merged one containing coverage from all tests.

Grilse

Answer 6

I got the integration tests working with Maven 3.0.3 and cobertura-maven-plugin 2.5.1 by configuring the maven-failsafe-plugin to run during the test phase:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.12</version>
    <executions>
        <execution>
            <phase>test</phase>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Calculation

This defeats the purpose of having the failsafe plugin: if you're running failsafe in the test phase, you might as well just have your tests as regular tests (picked up by surefire), which run on the test phase. – Kilderkin