How to get Cobertura to fail M2 build for low code coverage

Question

I'm trying to configure my WAR project build to fail if the line or branch coverage is below given thresholds. I've been using the configuration provided on page 455 of the excellent book Java Power Tools, but with no success. Here's the relevant snippet of my project's Maven 2 POM:

<build>
...
<plugins>
  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>cobertura-maven-plugin</artifactId>
    <version>2.2</version>
    <configuration>
      <check>
        <!-- Per-class thresholds -->
        <lineRate>80</lineRate>
        <branchRate>80</branchRate>
        <!-- Project-wide thresholds -->
        <totalLineRate>90</totalLineRate>
        <totalBranchRate>90</totalBranchRate>
      </check>
      <executions>
        <execution>
          <goals>
            <goal>clean</goal>
            <goal>check</goal>
          </goals>
        </execution>
        <execution>
          <id>coverage-tests</id>
          <!-- The "verify" phase occurs just before "install" -->
          <phase>verify</phase>
          <goals>
            <goal>clean</goal>
            <goal>check</goal>
          </goals>
        </execution>
      </executions>
      <instrumentation>
        <excludes>
      <exclude>au/**/*Constants.*</exclude>
        </excludes>
        <ignores>
      <ignore>au/**/*Constants.*</ignore>
        </ignores>
      </instrumentation>
    </configuration>
  </plugin>
  ...
</plugins>
...
</build>

As I say, the coverage report works fine, the problem is that the "install" goal isn't failing as it should if the line or branch coverage is below my specified thresholds. Does anyone have this working, and if so, what does your POM look like and which version of Cobertura and Maven are you using? I'm using Maven 2.0.9 and Cobertura 2.2.

I've tried Googling and reading the Cobertura docs, but no luck (the latter are sparse to say the least).

Replay

Answer 1

To my knowledge, if the <haltOnFailure> element is set to true and any of the specified checks fails, then Cobertura will cause the build to fail which is what you're asking for. But actually, this element defaults to true if you do not specify it so you don't have to add it to your configuration checks. Failing the build below any coverage threshold is (or at least should be) the default behavior.

EDIT: I did some further testing and haltOnFailure seems to be working as expected on my environment (Maven 2.2.1. and versions 2.3, 2.2, 2.1 of the plugin i.e. versions 1.9.2, 1.9, 1.8 of cobertura on Linux). I'm updating this answer with the result below.

Actually, I've added an <execution> element to my pom. I may be misinterpreting the part of cobertura:check's documentation that says it "Binds by default to the lifecycle phase: verify" but, without the <execution> element, cobertura:check wasn't triggered during the verify phase of my build. Below the setup I've use for the cobertura-maven-plugin:

<project>
  ...
  <build>
    ...
    <plugins>
      ...
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>cobertura-maven-plugin</artifactId>
        <version>2.3</version>
        <configuration>
          <check>
            <!--<haltOnFailure>true</haltOnFailure>--><!-- optional -->
            <!-- Per-class thresholds -->
            <lineRate>80</lineRate>
            <branchRate>80</branchRate>
            <!-- Project-wide thresholds -->
            <totalLineRate>90</totalLineRate>
            <totalBranchRate>90</totalBranchRate>
          </check>
        </configuration>
        <executions>
          <execution>
            <phase>verify</phase>
            <goals>
              <!--<goal>clean</goal>--><!-- works if uncommented -->
              <goal>check</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

I get the following result when running mvn clean install on a freshly generated maven project (with mvn archetype:create) patched with the plugin configuration mentioned above:

$ mvn archetype:create -DgroupId=com.mycompany.samples -DartifactId=cobertura-haltonfailure-testcase
...
$ mvn clean install
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building cobertura-haltonfailure-testcase
[INFO]    task-segment: [clean, install]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting directory /home/pascal/Projects/cobertura-haltonfailure-testcase/target
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/pascal/Projects/cobertura-haltonfailure-testcase/src/main/resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Compiling 1 source file to /home/pascal/Projects/cobertura-haltonfailure-testcase/target/classes
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/pascal/Projects/cobertura-haltonfailure-testcase/src/test/resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Compiling 1 source file to /home/pascal/Projects/cobertura-haltonfailure-testcase/target/test-classes
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: /home/pascal/Projects/cobertura-haltonfailure-testcase/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.mycompany.samples.AppTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.09 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] [jar:jar {execution: default-jar}]
[INFO] Building jar: /home/pascal/Projects/cobertura-haltonfailure-testcase/target/cobertura-haltonfailure-testcase-1.0-SNAPSHOT.jar
[INFO] Preparing cobertura:check
[WARNING] Removing: check from forked lifecycle, to prevent recursive invocation.
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/pascal/Projects/cobertura-haltonfailure-testcase/src/main/resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [cobertura:instrument {execution: default}]
[INFO] Cobertura 1.9.2 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Instrumenting 1 file to /home/pascal/Projects/cobertura-haltonfailure-testcase/target/generated-classes/cobertura
Cobertura: Saved information on 1 classes.
Instrument time: 337ms

[INFO] Instrumentation was successful.
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/pascal/Projects/cobertura-haltonfailure-testcase/src/test/resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: /home/pascal/Projects/cobertura-haltonfailure-testcase/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.mycompany.samples.AppTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.098 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] [cobertura:check {execution: default}]
[INFO] Cobertura 1.9.2 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Cobertura: Loaded information on 1 classes.

[ERROR] com.mycompany.samples.App failed check. Line coverage rate of 0.0% is below 80.0%
Project failed check. Total line coverage rate of 0.0% is below 90.0%

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Coverage check failed. See messages above.
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18 seconds
[INFO] Finished at: Sat Oct 24 21:00:39 CEST 2009
[INFO] Final Memory: 17M/70M
[INFO] ------------------------------------------------------------------------
$ 

I didn't test with maven 2.0.9, but on my machine, haltOnFailure generates a BUILD ERROR and halt the build. I don't see any differences with your plugin configuration, I can't reproduce the behavior you describe.

Confiture

I replicated your success using your config, so now I just have to integrate it into my real projects. BTW, there are several differences between my config and yours, e.g. my "executions" tag is a child of my "configuration" tag, whereas yours is a sibling (as in the book; I don't know why I deviated from that). – Replay

Glad to know. And indeed, configuration are different which is actually a good thing :) I don't know how I missed that. – Confiture

I know this is a very old answer, but I just thought to point out that in the latest documentation, it is specifically asked to set it to "true" if you want it to fail. mojohaus.org/cobertura-maven-plugin/check-mojo.html – Kerouac

Answer 2

Add the following to the <check/> configuration.

<haltOnFailure>true</haltOnFailure>

Naturalize

Thanks for answering, but I tried that and it didn't work. From what I've read about that setting, it determines whether the build halts upon a test failure, not whether the build fails (or halts) because of low coverage. Is it working for you, or were you just suggesting something to try? – Replay

Answer 3

mvn clean install -Dcobertura.skip=true

Saladin