JMH not working in Eclipse (as Maven project) - No benchmarks to run

Question

I want to start have a look at JMH and I'm failing to run benchmarks due some reasons. Let me explain what I tried:

1. Setup a maven project in Eclipse

2. Define pom.xml like:

3. Downloaded some official JMH examples. As example I choosed which is pretty simple and a good place to start: http://hg.openjdk.java.net/code-tools/jmh/file/0c58dc4fcf17/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_01_HelloWorld.java

4. Right mouse click > Run As > Java Application

However this produces the output:

Exception in thread "main" No benchmarks to run; check the include/exclude regexps.
at org.openjdk.jmh.runner.Runner.run(Runner.java:155)
at org.openjdk.jmh.samples.JMHSample_01_HelloWorld.main(JMHSample_01_HelloWorld.java:90)

I have googled and it seems the above example should work, but it is not the case for me. I also tried to solve the issue by reading but this seems not working for me:

• "No matching benchmarks" when running JMH from main in eclipse

• I tried to move the generated class manually to /META-INF/MicroBenchmarks however this produces following error:

  Exception in thread "main" java.lang.IllegalStateException: Mismatched format for the line: JMHSample_01_HelloWorld.class
  at org.openjdk.jmh.runner.BenchmarkRecord.<init>(BenchmarkRecord.java:92)
  at org.openjdk.jmh.runner.MicroBenchmarkList.find(MicroBenchmarkList.java:133)
  at org.openjdk.jmh.runner.Runner.run(Runner.java:150)
  at JMHSample_01_HelloWorld.main(JMHSample_01_HelloWorld.java:80)
  

It seems like the JMH should produce some valid lines in /META-INF/MicroBenchmarks - meaning in this folder there should no generated java class files, correct?

Can anyone help me to find the mistake? Thank you.

Ginter

You missed a step where JMH annotation processor is applied to your benchmark. Better to start from generating a new project from archetype using the example command given on the JMH home page – Disoperation

This post in jmh-dev provides additional information about using annotation processors in post-0.5 JMH. – Disoperation

Thanks a lot! The annotation processor was missing! :-) . What I did is just: 1) Create new maven project using Eclipse > File > New Maven Project 2) Use default Workspace location 3) search for group id "org.openjdk.jmh" 4) select "jmh-java-benchmark-archetype" 5) enter your group id and artifact id (e.g. jmh-test) 6) clean maven by right mouse click on project > Run As > Maven Clean 7) install maven by right mouse click on project > Run As > Maven Install 8) Import the samples from JMH websites 9) go to JMHSample_01_HelloWorld and run it as Java Application Hope this helps others... – Ginter

Can @OlegEstekhin provide the answer as "Answer", and can pitschr accept it then? Otherwise the question feels unanswered in the search list :) Thanks. – Icosahedron

@OlegEstekhin it would be good if this were better documented. Sometimes, you don't want to create a new maven project; and then its tricky to get this black magic working right. – Contusion

I think it is unfortunate that Maven archetype approach is needed: esp. since upgrading from 0.x version of jmh is now rather difficult, at least first time around. And many/most tutorials still use version 0.2 or 0.4. At least need for specific dependency (provided by top answer here, thanks!) should be documented by jmh docs as well. – Kwok

Answer 1

The benchmark "source code", in addition to be compiled as usual, needs to be also processed by the JMH annotation processor. Prior to JMH 0.5 the annotation processor was part of the main JMH artifact, so a single dependency to jmh-core was sufficient. In JMH 0.5 the annotation processing functionality was extracted into separate artifact to facilitate support for other languages.

In order the get the annotation processor back in "old" Java-based JMH projects an additional dependency to the processor artifact should be declared:

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>${jmh.version}</version>
    <!-- 
    the processor artifact is required only during compilation and 
    does not need to be transitive, hence provided scope
    -->
    <scope>provided</scope> 
</dependency>

For new JMH projects the easiest way is to generate a new project using the proper language-specific archetype (besides Java, there are Scala, Groovy and Kotlin archetypes), as described on the JMH home page.

mvn archetype:generate \
      -DinteractiveMode=false \
      -DarchetypeGroupId=org.openjdk.jmh \
      -DarchetypeArtifactId=jmh-java-benchmark-archetype \
      -DgroupId=my.benchmark.group \
      -DartifactId=MyBenchMarkArtifact \
      -Dversion=1.0

Disoperation

Answer 2

I got the same error in eclipse console on my jmh+maven+eclipse project though having all the required dependencies and plugins.

Tried mvn clean install from terminal instead of embedded maven in eclipse, got below error

ERROR: org.openjdk.jmh.runner.RunnerException: ERROR: Unable to acquire the JMH lock (/tmp/jmh.lock): already taken by another JMH instance, exiting. Use -Djmh.ignoreLock=true to forcefully continue.
    at org.openjdk.jmh.runner.Runner.run(Runner.java:202)
    at org.openjdk.jmh.Main.main(Main.java:71)

as recommended above added -Djmh.ignoreLock=true as VM arguments in eclipse Java application launcher, it worked

Petrifaction

Answer 3

I got the same error because I had changed the name of the package containing my @Benchmark class after running it previously. A "clean and build" was sufficient to get it working again.

Sapphera

Answer 4

You may face this error in case you have refactored the name of the class which includes @Benchmark,

You will have to change the name in the main function as well.

OptionsBuilder opts = new OptionsBuilder();
opts.include("<new_class_name>");

Yehudit