Testng, Emma, Cobertura, coverage and JDK 7 result in ClassFormatError and VerifyError

Question

I have switched to the newest JDK 7 and I am having problems with running testng unit test on byte code that is fiddled by emma coverage tool. None of my test cases are run correctly and for most of them I am receiving such errors.

 java.lang.ClassFormatError: Illegal local variable table length 10 in method measurement.meter.AbstractSerialPortMeter.<init>(Lmeasurement/meter/SerialPort;)V at measurement.meter.Elc3133aTest.setUp(Elc3133aTest.java:42)

I have found an article here JSR 292 Goodness Fast Code Coverage Tool Less 10k, which is saying that "JSR 292 introduces a new bytecode instruction invokedynamic but also several new kind of constant pool constants. Which means that most of the tools that parse bytecodes like ASM, BCEL, findbugs or EMMA will need to be updated to be java 7 compatible."

Checked Emma homepage, but it looks like it has not been updated for a long long time.

Has anybody solved a similar problem?

I have also tried with Cobertura. It looks to work a bit better but I am getting a lot of exceptions of type VerifyError.

java.lang.VerifyError: Expecting a stackmap frame at branch target 85 in method measurement.meter.AbstractSerialPortMeter.close()V at offset 26
at measurement.meter.AbstractSerialPortMeterTest.setUp(AbstractSerialPortMeterTest.java:27)

Eastman

I don't think anyone will move to Java7 anytime soon as it's yet unknown how many bugs and security flaws it has, so I suggest you to do the same. At the same time, as you mentioned, EMMA hasn't been updated in a long time, long enough to consider it almost a dead project. I would suggest you to try Cobertura in a few months as it also has some issues with Java7, but unlike EMMA, there's a minimal ongoing development (last change was 2 months ago) – Dionysiac

Thanks for the hint. I will try Cobertura. Going back to JDK 6 is not an option as my target is exactly to check how JDK 7 performs. :-) – Eastman

I have the exact same problem running EMMA from within Eclipse 3.7.1 on a Java7 project. – Hoad

Yep, it's been some time since I wrote the question but still neither Cobertura nor Emma got updated. This is a bit frustrating, but well, we cannot do much about it I guess. – Eastman

It would be great if cobertura-dev could give us some clue about when java 7 would be supported. – Abc

Not sure about cobertura but the eclemma guys seem to have created a Java 7 compliant coverage engine of their own called JaCoCo eclemma.org/jacoco/trunk/index.html – Anatropous

just tested it. JaCoCo does not throw errors with Java 7 syntax, but it does not "understand" try-with-resources statments, resultung in seemingly two lines of uncovered code per statment. – Clemmieclemmons

The website of Cobertura announced that they support Java7 now. – Haversack

Answer 1

I had the same issue. Fortunately beta works with JDK 7.
Update site link: http://download.eclipselab.org/eclemma/beta/2.0.0/update/
This link should be used in Eclipse:

Help -> Install new software... -> Add...

Rest should be easy ;)

Goofball

After clicking on the link you have provided I got the xml response Access Denied. However by the content of the link I can tell this is an emma Eclipse plugin. Do you run it as an Ant task as well? The Ant task is the thing I need most. – Eastman

Ok, already figured it out. I should use it in the Eclipse "Install New Software..." – Eastman

The hint you had given me, led me to the JaCoCo: eclemma.org/jacoco/trunk/index.html which for me works perfectly and covers all my requirements. Therefore I am accepting this answer. – Eastman

Answer 2

I had same problem using maven cobertura plugin. All tests failed when run from cobertura:report. But all tests succeeded when run directly from surefire plugin. As some of you already said the problem is that coberture byte code instrumentation of is not compatible with JDK7.

You can see here http://vikashazrati.wordpress.com/2011/10/09/quicktip-verifyerror-with-jdk-7/ that the exception is related to the "new type checker with StackMapTable attributes" (see: -X:+UseSplitVerifier JVM option in http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html).

So my solution is to configure surefire-plugin to always execute the tests with JVM arg "-XX:-UseSplitVerifier. It works well with and without cobertura instrumentation.

My surefire configuration in maven:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12</version>
    <configuration>
        <argLine>-XX:-UseSplitVerifier</argLine>
    </configuration>
</plugin>

Heman

Added this to "VM Arguments" in Eclipse "Coverage Configurations...". Works well. – Bookshelf

Tks Pedro; Works perfectly for me. Same configuration, same issue. Same solution. – Intertwist

@Bookshelf You are genius! It works perfectly in Eclipse. Why bother all the time with Surefire, if you can see all the results in your code editor in Eclipse? You should have made your post as a top-level one, because it is really, really helpful. Also it worth mentioning that before using Cobertura in Kepler you must follow these steps: #18630641 – Dudleyduds

@nucleo Thanks for the kind words. I'm glad my note assisted you. I honestly don't even remember writing the comment or the what I was doing at the time. The whole thing was probably one of those get-through-it / fix-stupid-java-crap things one deals with and promptly forgets ; didn't seem worth more than a quick comment at the time. – Bookshelf

@Pedro Ballesteros With above configuration it fixed my issue last week. But again today it comes up. – Mastoiditis

Answer 3

I got Gradle 1.0M9, Java 7 and EMMA 2.1 working with the patches suggested here: using the jvm argument.

Details here... http://marcellodesales.wordpress.com/2012/04/03/running-emma-code-test-coverage-with-java-7-and-gradle-1-0m9/?preview=true&preview_id=179&preview_nonce=261e892908

configurations{
  emma
}

dependencies {
  // EMMS Code Coverage
  emma "emma:emma:2.1.5320"
  emma "emma:emma_ant:2.1.5320"
  ...
  testCompile group: 'junit', name: 'junit', version: '4.9'
}

test {
    // add EMMA related JVM args to our tests
    jvmArgs "-XX:-UseSplitVerifier", "-Demma.coverage.out.file=$buildDir/tmp/emma/metadata.emma", "-Demma.coverage.out.merge=true"

    doFirst {
       println "Instrumenting the classes at " + sourceSets.main.output.classesDir.absolutePath
       // define the custom EMMA ant tasks
       ant.taskdef( resource:"emma_ant.properties", classpath: configurations.emma.asPath)

       ant.path(id:"run.classpath") {
          pathelement(location:sourceSets.main.output.classesDir.absolutePath)
       }
       def emmaInstDir = new File(sourceSets.main.output.classesDir.parentFile.parentFile, "tmp/emma/instr")
       emmaInstDir.mkdirs()
       println "Creating $emmaInstDir to instrument from " +       sourceSets.main.output.classesDir.absolutePath
       // instruct our compiled classes and store them at $buildDir/tmp/emma/instr
       ant.emma(enabled: 'true', verbosity:'info'){
          instr(merge:"true", destdir: emmaInstDir.absolutePath, instrpathref:"run.classpath",
                metadatafile: new File(emmaInstDir, '/metadata.emma').absolutePath) {
             instrpath {
             fileset(dir:sourceSets.main.output.classesDir.absolutePath, includes:"**/*.class")
             }
          }
       }
       setClasspath(files("$buildDir/tmp/emma/instr") + configurations.emma +    getClasspath())
    }

    // The report should be generated directly after the tests are done.
    // We create three types (txt, html, xml) of reports here. Running your build script now should
    // result in output like that:
    doLast {
       def srcDir = sourceSets.main.java.srcDirs.toArray()[0]
       println "Creating test coverage reports for classes " + srcDir
       def emmaInstDir = new File(sourceSets.main.output.classesDir.parentFile.parentFile, "tmp/emma")
       ant.emma(enabled:"true"){
          new File("$buildDir/reports/emma").mkdirs()
          report(sourcepath: srcDir){
             fileset(dir: emmaInstDir.absolutePath){
                include(name:"**/*.emma")
             }
             txt(outfile:"$buildDir/reports/emma/coverage.txt")
             html(outfile:"$buildDir/reports/emma/coverage.html")
             xml(outfile:"$buildDir/reports/emma/coverage.xml")
          }
       }
       println "Test coverage reports available at $buildDir/reports/emma."
       println "txt: $buildDir/reports/emma/coverage.txt"
       println "Test $buildDir/reports/emma/coverage.html"
       println "Test $buildDir/reports/emma/coverage.xml"
    }
}

Running "gradle test" gives the following:

marcello@hawaii:/u1/development/workspaces/open-source/interviews/vmware$ gradle test
:compileJava
:processResources UP-TO-DATE
:classes
:compileTestJava
:processTestResources UP-TO-DATE
:testClasses
:test
Instrumenting the classes at /u1/development/workspaces/open-source/interviews/vmware/build/classes/main
Creating /u1/development/workspaces/open-source/interviews/vmware/build/tmp/emma/instr to instrument from /u1/development/workspaces/open-source/interviews/vmware/build/classes/main
Creating test coverage reports for classes /u1/development/workspaces/open-source/interviews/vmware/src/main/java
Test coverage reports available at /u1/development/workspaces/open-source/interviews/vmware/build/reports/emma.
txt: /u1/development/workspaces/open-source/interviews/vmware/build/reports/emma/coverage.txt
Test /u1/development/workspaces/open-source/interviews/vmware/build/reports/emma/coverage.html
Test /u1/development/workspaces/open-source/interviews/vmware/build/reports/emma/coverage.xml

BUILD SUCCESSFUL

Cleptomania

The jvmArgs "-XX:-UseSplitVerifier" did it for me. Thanks! – Madel

Answer 4

Emma works if you do not use new language features (like try-with-resources, etc.). You can use Java 7 using new libraries (Paths, DirectoryStream, etc.). I know this would be not a solution to your problem, but if you want to only check "how JDK 7 performs" it may work...

Epp

After switching to Java 7 I did a refactoring to my old code, so indeed going back is not an option, but thanks for the hint. Because of the fact that my code complies to the newest version of Java I was not able to check if it still does work with Java 6 syntax. – Eastman

This doesn't seem to be true. I'm having the same problem and I haven't made any source changes. – Criminate

Answer 5

I was having this problem. Upgrading to 2.0.1.201112281951 using the Eclipse marketplace worked for me.

Bertero

Answer 6

IntelliJ IDEA 11's internal coverage tool works fine for my project using try-with-resources, diamond operator but we are not using invokedynamic. I think the coverage tool is not included in the community edition, ultimate only.

I have yet to try jacoco - it's where most of emma's former developers seem to have gone.

Derron

JaCoCo looks perfectly and in comparison to other tools it does not use instrumentation like byte code instrumentation (Emma, Cobertura) nor source code instrumentation (Clover). I do not know the details but it looks like it reuses the debug information and works as Java agent, which simplifies lots of task during the build. – Eastman

In fact JaCoCo performs byte code instrumentation, but in contrast with Emma and Cobertura - on-the-fly. – Ambition

Answer 7

The Java 8+ equivalent of the answer by Pedro Ballesteros is:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.12.4</version>
  <configuration>
    <argLine>-noverify</argLine>
  </configuration>
</plugin>

(Tweak the version number to match the version of Surefire you are using.)

Siffre