Why does Cobertura fail to report assert branch path was covered?

Question

In Cobertura, I can not get it to report that the conditional path of an assert statement was taken. Is this a known limitation?

I have a JUnit test that expects and AssertionError to be thrown, and it passes correctly. The problem is that Cobertura reports that the assert branch was not covered.

---

After more investigation, I see that part of the branch coverage is being detected. The line is question is:

assert data != null;

and Cobertura reports the coverages as:

Conditional coverage 75% (3/4) [each condition 50%, 100%].

What are the different branch conditions Cobertura is expecting?

Schacker

Answer 1

I was able to get 100% coverage by running JUnit twice; once with assertions enabled, and once with assertions disabled.

Schacker

Could you publish the relevant POM configuration (to run JUnit tests twice)? – Fining

Answer 2

I bumped into the same issue, so I spent a bit of time to reverse engineer the answer, donating it to Stack Overflow.

For each Java assert-statement, Cobertura keeps track of two conditions:

1. Whether a given assert statement was executed with assertion checking enabled or disabled.
2. Whether the predicate actually evaluates to true or to false.

Thus, a total of four outcomes are possible. The information provided for a given line in a HTML report consists of

• the outcome for condition 1 (0-2 possibilities taken out of 2, addressing execution with checking enabled or disabled),
• and the outcome for condition 2 (0-2 possibilities taken out of 2: assertion pass or fail).
• the overall outcome (0-4 out of 4),

Typical scenario's are:

• Running Cobertura once, with assertion checking disabled. You'll get:
  Enabled/Disabled: 50% (disabled); Passed/Failed: 0% (not reached); Hence overall 25%.
  Cobertura will report this as

  Conditional coverage 25% (1/4) [each condition 50%, 0%]

• Running Cobertura once, with assertion checking enabled. Typically your assertions are always true, hence you get:
  Enabled/Disabled: 50% (enabled); Passed/Failed: 50% (always true); Hence overall: 50%.

• Running Cobertura twice, once with assertion checking enabled, and once without. Assuming assertions are always true, we get:
  Enabled/Disabled: 100% (both enabled and disabled); Passed/Failed: 50% (always true); Hence overall 75%.

Then, if we add test cases that ensure that a given assertion fails at least once, and passes at least once, we get all numbers at 100%.

Note, however, that if you use assertions in the style of design by contract, you will generally not even be able to make them fail, see the answer to another Stack Overflow question, Cobertura coverage and the assert keyword.

Finally: while theses numbers are explainable, I am not sure that they are very useful. My preference would be to be able to omit assertion-related coverage from the overall reports. Clover can do this, but I'm not aware of an open source coverage analysis tool with this nice feature.

Boycie

Wow! Maybe a javac hack that replaces all assertion statements with noop might help to get the coverage fixed. See scg.unibe.ch/staff/adriankuhn/javacompiler/forceassertions on how to get that done. Yet, eventually Cobertura should be fixed to ignore assertions. Patch, anyone? – Missi

Noops to rescue coverage reports -- nice idea! – Boycie

If you are unit testing your classes, it should not be difficult to get the asserts to fail. I guess it depends what level you are unit testing at. Also, I would caution against converting asserts to no-ops, since you might end up hiding a bug in your assert logic. – Schacker