Execution data for class does not match + Jacoco

Question

I'm using Jacoco to find the code coverage of unit tests with ANT but the report isn't generated and I get this error sequence:

[jacoco:report] Loading execution data file C:\JUnit\apache-ant-1.10.1\jacoco.ex
ec

[jacoco:report] Writing bundle 'Test' with 566 classes

[jacoco:report] Classes in bundle 'Test' do no match with execution data. For report generation the same class files must be used as at runtime.

[jacoco:report] Execution data for class com/!!/AccountDetails does not match.

[jacoco:report] Execution data for class com/!!/DataExtractorHelper does not match.

[jacoco:report] Execution data for class com/!!/WelcomeLetter does not match.

[jacoco:report] Execution data for class com/!!/WelcomeLetterABCD does not match.

I've read these answers but none seemed to help me solve the problem.

jacoco code coverage report generator showing error : "Classes in bundle 'Code Coverage Report' do no match with execution data"

jacoco: For report generation the same class files must be used as at runtime

I compiled all classes on Eclipse and used the ANT build tool to perform code coverage on these classes. I do make use of some external jars due to some dependencies and they've been compiled on jdk 1.8.0_101 and I'm using jdk 1.8.0_111 (I tried solving this error by using jdk 1.8.0_101 but I got the same errors)

It has been mentioned that the class ID might change in Eclipse vs Oracle JDK compilation. Therefore I also checked this case by compiling some basic classes on Eclipse and used the jdk + ANT to find the code coverage. It worked in this case. There is no compilation taking place in the code coverage task. The .class files just need to be checked for coverage.

All the classes mentioned in the errors have been compiled on eclipse before testing for the code coverage.

I've tried using the offline instrumentation tool as a workaround to the persistence framework being used but it still gives me these errors. All the classes mentioned above in the errors are present in the Instrumented classes folder. ${dest.dir}

This is my build.xml at the moment.

<target name="instrument">
    <delete dir="${dest.dir}"/>
    <mkdir dir="${dest.dir}"/>
    <jacoco:instrument destdir="${dest.dir}">
        <fileset file="D:/NEON/HW/!!/module/!!/bin" includes="**/*.class"/>
        <fileset file="D:/NEON/HW/!!/testprojects/!!/bin" includes="**/*.class"/>
    </jacoco:instrument>
</target>


<target name="cov-test" depends="instrument">
    <mkdir dir="${report.dir}"/>    
    <jacoco:coverage>
        <junit fork="true" forkmode="once" showoutput="true" printsummary="on" enabletestlistenerevents="true">

            <classpath>
                <path refid="ALL.jars"/>
                <path refid="classpath"/>
                <pathelement location="C:/JUnit/jacoco-0.7.9/lib/jacocoagent.jar"/>
                <pathelement location="C:/JUnit/JARS/!!/config/"/>
                <pathelement path="C:/JUnit/apache-ant-1.10.1/InstrClasses"/>
            </classpath>

            <sysproperty key="jacoco-agent.destfile" file="jacoco.exec"/>



            <test name="Fully qualified classname"/>

            <formatter type="plain"/>
            <formatter type="plain" usefile="false" />
            <batchtest fork="yes" todir="${report.dir}">
                <fileset dir="${src.dir}" includes="Fully qualified classname.java"/>
            </batchtest>
        </junit>
    </jacoco:coverage>
</target>


<target name="cov-report" depends="cov-test">
    <jacoco:report>
        <executiondata>
            <file file="jacoco.exec" />
        </executiondata>
        <structure name="Test">
            <classfiles>
                <fileset dir="D:/NEON/HW/!!/module/!!/bin"/>
                <fileset dir="D:/NEON/HW/!!/testprojects/!!/bin"/>
            </classfiles>
            <sourcefiles>
                <fileset dir="D:/NEON/HW/!!/module/!!/src"/>
                <fileset dir="D:/NEON/HW/!!/testprojects/!!/src"/>
            </sourcefiles>      
        </structure>
        <csv destfile="${report.dir}/report.csv" />
    </jacoco:report>

</target>

Questions: 1.Will there be any difference in the bytecode generated by compilers based on jdk 1.8.0_101 and jdk 1.8.0_111? Can incremental updates change bytecode? Or is the difference only significant during major version updates?

2.Why am I still getting this error even after offline instrumentation has been implemented? Am I missing out on any declaration in the code? I've tried to keep the format of the code as similar to that of the example provided in the jacoco documentation here.

3. I've also noticed that the number of classes being instrumented(614) differs from the number of classes being added to the Test bundle (566) when both contain the same classpaths. Could that be of any consequence?

Scintilla

Hi, I rolled back your edit because it's a completely different question and invalidates the answer you received. You should ask a new question, ideally including a MCVE – Margarita

Answer 1

Will there be any difference in the bytecode generated by compilers based on jdk 1.8.0_101 and jdk 1.8.0_111? Can incremental updates change bytecode? Or is the difference only significant during major version updates?

Yes - in general any different versions (without exceptions) of compilers can generate different bits of bytecode.

Why am I still getting this error even after offline instrumentation has been implemented? Am I missing out on any declaration in the code?

The message itself is already perfectly explained in SO question to which you refer: jacoco code coverage report generator showing error : "Classes in bundle 'Code Coverage Report' do no match with execution data"

More precise answers require a Minimal, Complete, and Verifiable example (https://stackoverflow.com/help/mcve) to be provided by you, and unfortunately in my opinion just excerpt of build.xml with some comments is not Complete and Verifiable example. And not clear which role Eclipse is playing here in addition to JDK. By the way Eclipse has and uses its own Eclipse Java Compiler.

I've also noticed that the number of classes being instrumented(614) differs from the number of classes being added to the Test bundle (566) when both contain the same classpaths. Could that be of any consequence?

Yes - consequence is that what was instrumented is not the same as what was analyzed for generation of report. Which also correlates with messages about mismatch.

Credent

I just checked the classes not being added to the Test bundle and it turns out that they are all interfaces. Since interfaces aren't considered for code coverage, (jacoco.org/jacoco/trunk/doc/faq.html) Could this difference in the number of instrumented classes(614) and classes in the test bundle(566) now really matter? This wouldn't be the cause of the error "Execution data mismatch" right? – Scintilla

I have access to the .class as well as the src files(whose code-cov I want). These .class files have been compiled on Eclipse working on the jdk 1.8.0_111 & I'm running the jacoco coverage on these .class files. There are also some jars in the classpath which contain some basic dependencies for the code.These jars contain only the .class files which have been compiled on Eclipse ( jdk 1.8.0_101) Is this mismatch a result because the classes have been compiled on eclipse whereas the jacoco coverage test is being run via the oracle jdk(on cmd)? – Scintilla

@Scintilla If you accidentally mix compilation by Eclipse with compilation by JDK, then classes won't match. If not, then you should be safe, but why not try compiling everything with JDK to check? During instrumentation number of classes on disk is shown, during generation of report number of classes in report is shown, so yes - interfaces without executable code do not show up in second case. – Credent

Yeah it worked after I compiled it all via JDK. Although now I'm trying to run multiple junit tests together to find their coverage. I've been able to run all junit tests within a single folder. How can I run junits test classes from different folders? I'm updating the question for a more clearer picture. – Scintilla

@Scintilla Question about JUnit is completely unrelated to initial topic of "missmatch" and unrelated to JaCoCo. StackOverflow posts should contain one well defined question and not endless discussion - meta.stackexchange.com/questions/222735/… Moreover was already said that question lacks a Minimal, Complete, and Verifiable example (stackoverflow.com/help/mcve). Since you didn't upvoted the answer that addresses initial question, not even talking about marking it as accepted, I'm going to stop this discussion. Thank you for your understanding. – Credent

@Scintilla it seems that Godin has answered your original question. Your update is a completely different question. And it invalidates existing answers, so it's unfair to answerers. You should remove that update and ask a new question instead, ideally including a MCVE as Godin suggested. – Margarita

Answer 2

2.Why am I still getting this error even after offline instrumentation has been implemented? Am I missing out on any declaration in the code? I've tried to keep the format of the code as similar to that of the example provided in the jacoco documentation here.

I've noticed this when the class under test is added to PrepareForTest annotation in it's test class.

E.g. in following situation, Foo.java will have 0 coverage with error Execution data for class Foo does not match..

@PrepareForTest({ Foo.class })
public class FooTest {

  @Test
  public void test1() {
    Foo f = new Foo();
    String result = f.doSomething();
    assertEquals(expectedResult, result);
  }

}

Cuneal

Answer 3

Basically any difference between the bytecodes that tests were run against vs the bytecodes jacoco picks up to calculate coverage with, would cause this. some examples are minifying and obfuscation, using different jvms for different steps and any type of bytecode level manipulation.

in my case firebase was doing some bytecode manipulation. there is a ticket filed here.

you will need to downgrade to a version of firebase + AGP that does not have this issue.

Importation

omg you absolutely saved my life!!!! I was losing my mind already not knowing what was causing it for me, and it was firebase perf too... thank you!!!! – Halvorsen