How do I get Emma or Cobertura, with Maven, to report coverage on source code in other modules?

Question

I have a multi-module Maven setup with Java code.

My unit tests, in one of the modules, exercise code in multiple modules. Naturally, the modules have inter-dependencies, and code in all relevant modules is compiled as needed in advance of test execution.

So: How can I get a report on the coverage of the entire codebase?

---

Note: I am not asking how to combine the results of coverage for tests in multiple modules. I am asking how to get coverage for tests in a single module using instrumented code from multiple modules. Anyone interested in the former might refer to these other questions, and the recommendations by Crowne for Maven Dashboard and Sonar.

I succeeded in getting a full coverage report using pure Ant. [EDIT:] I instrumented all jars from the development-runtime directory into a temporary directory; prepended the temporary directory to the classpath; then ran tests from Ant with batch-test.

Ant can be run from Maven, but the challenge here is seamless integration (i.e., feeding all the classpath and sourcepath elements from Maven to Ant automatically), which is why I did not use Maven's facilities for this purpose.

There are also other questions about integration tests. However, by default, each project's report by default only reports coverage on code in the same project, whereas my tests exercise code in multiple projects.

This article in Spanish might be relevant. Here is another Seam-specific article.

---

Nucleon

Answer 1

This recent blog post by Thomas Sundberg contains a method that partially solves the issue by using ant for the cobertura calls, instead of using the maven cobertura plugin.

It relies on the following basic approach with specialised pom.xml and build.xml files :

Start with a typical maven compile on the parent pom, which will compile all classes in the child modules.

mvn clean compile # maven-compile-plugin called for compiling

Then instrument all of the module classes:

ant instrument # cobertura called for instrumentation

Then call the maven-surefire-plugin called for testing using the instrumented classes, with cobertura as a test dependency

mvn test 

Then use a custom report call to pull in all of the results from different modules:

ant report # cobertura called for reporting

The key elements of the ant build.xml file are to instrument all modules separately and then to report on all of the modules after merging the results. This function needs to be called for each module in his example:

<target name="instrumentAModule">
    <property name="classes.dir" value="target/classes"/>
    <cobertura-instrument todir="./${module}/${classes.dir}">
        <fileset dir="./${module}/target/classes">
            <include name="**/*.class"/>
        </fileset>
    </cobertura-instrument>
</target>

Then after the testing is complete, the reporting phase first merges all results from all of the different directories are merged into a new .ser file (called sum.ser in his example)

<target name="report" depends="merge">
    <property name="src.dir" value="src/main/java/"/>
    <cobertura-report datafile="sum.ser"
                      format="html"
                      destdir="./target/report">
        <!-- Add all modules that should be included below -->
        <!-- fileset dir="./MODULE_NAME_TO_REPLACE/${src.dir}"/ -->
        <fileset dir="./product/${src.dir}"/>
    </cobertura-report>
</target>

<target name="merge">
    <cobertura-merge datafile="sum.ser">
        <fileset dir=".">
            <include name="**/cobertura.ser"/>
        </fileset>
    </cobertura-merge>
</target>

It may be possible to integrate the ant components into maven using the antrun plugin, but I am not familiar enough with the phases/lifecycles to know where to put the different calls.

This is very useful for me, as I write abstract test classes in my api modules and then provide them with an implementation in my lib modules. So far both cobertura and emma have been unable to handle this design so my code coverage is typically 0 or in the single digits.

Gynaecology

Answer 2

Never tried, but this may be a way to accomplish it:

• In each module, just before the install phase, let cobertura instrument the jar files and install the instrumented jar files (!) into the local Maven repository
• In the tests-module, Maven will use the artifact dependencies from the local Maven repository to run the tests. These instrumented classes should now appear in the datafile, e.g. cobertura.ser
• Run the cobertura-report generation as usual from within the tests-module of your project, e.g. mvn site

See cobertura documentation on how to manually invoke cobertura to instrument external JAR files in-place:

... You can also pass in jar files to be instrumented using standard ant filesets. Cobertura will extract each class from the jar and instrument it. If 'todir' was not specified then the original jar will be overwritten with an instrumented version ...

The pom.xml's build plugins may look like this - you may want to add a profile or use classifiers to distinguish between the final jar file and the instrumented jar file if you don't want to overwrite them in your local repo. Then, in the tests module, you just need to define the dependencies to your other modules using the classifiers.

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-antrun-plugin</artifactId>
            <executions>
                <execution>
                    <id>cobertura-inplace-instrumentation</id>
                    <phase>package</phase>
                    <configuration>
                        <tasks>
                            <taskdef classpathref="maven.plugin.classpath" resource="tasks.properties" />
                            <cobertura-instrument
                                datafile="${project.build.directory}/cobertura-nop.ser">
                                <fileset dir="${project.build.directory}">
                                    <include name="${project.build.finalName}.${project.packaging}" />
                                </fileset>
                            </cobertura-instrument>
                        </tasks>
                    </configuration>
                    <goals>
                        <goal>run</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>net.sourceforge.cobertura</groupId>
                    <artifactId>cobertura</artifactId>
                    <version>1.9.4.1</version>
                </dependency>
            </dependencies>
        </plugin>

Emirate

Answer 3

Generally, reports pertain to their specific module, however they can be aggregated,
two approaches are:

• maven-dashboard-plugin
• sonar

I would recommend that you try using sonar to do the report aggregation for you.

See their public instance "nemo", to see the impressive capabilities that are offered.

Mac

Thank you. However, I am not asking how to combine the results of coverage for tests in multiple modules. Rather, I am asking how to get coverage for tests in a single module using instrumented code from multiple modules. – Nucleon

Answer 4

I doubt if would be possible since the coverage information is obtained by cobertura/emma by instrumenting the compiled classes. While this would work for classes in the specified project, it is doubtful if these tools will instrument dependant libraries.

A look at the maven cobertura plugin usage also does not seem to indicate any such possibility.

Jailbird

All my modules are under my control. All are compiled and instrumented, based on the dependency tree, when I run Maven and the coverage tool. – Nucleon

@Joshua Fox. I guess you need to check with cobertura developers or look at cobertura source. – Jailbird

Answer 5

I found this pretty straightforward (although I did it a while ago, and may be rusty with details...

My core project contains all of the modules. I use Cobertura to measure my test coverage. I am using Hudson as a Continuous Integration engine, and have the Cobertura plugin for Hudson.

It has been working a treat for a while now.

Good Luck !

Cleopatracleopatre