"No matching benchmarks" when running JMH from main in eclipse

Question

I wanted to try out the new feature of JMH by running it as Java Application in eclipse. I imported and built jmh-samples project. Compiled classes ended in /jmh-samples/target/generated-sources/annotations, there are several JARs in /target/ and running microbenchmarks.jar from command line works as usual.

However when I execute main I always get

No matching benchmarks. Miss-spelled regexp?

Any ideas? I am using version 0.3

Lagos

Face the same issue with my gradle project, have uploaded sample project https://github.com/twoVersatile/jmhsample for future users. – Linkoski

Answer 1

Okay, so looks like by default jmh looks for generated classes under META-INF/Microbenchmarks, which maven build puts under root of the project. However root of the eclipse project is not on the classpath, so executing it in IDE results in "no benchmarks found".

I got it running following way:

1. mvn clean package (using external maven installation, not embded in eclipse)
2. Right-click on jmh-samples project, select "Build Path -> Use as a source folder"
3. You can now run any of the benchmarks from jmh-samples as Java Application in eclipse

On the downside you get like 1000+ "errors" in Problems view, since eclipse gets confused with auto-generated files, but oh well, at least it works.

Lagos

I added both jmh-samples/target/generated-sources/annotations and jmh-samples/target/classes as source folders and it works for me without any errors. There's no problem with auto-generated files; they're correct Java sources (with thousands of warnings but no errors) and actually the thing that gets run in the benchmark. – Homy

target/classes are on classpath by default and adding jmh-samples/target/generated-sources/annotations doesn't give you anything, because this is not the location where org.openjdk.jmh.Main looks for benchmark classes. Check the value of org.openjdk.jmh.runner.MicroBenchmarkList.MICROBENCHMARK_LIST – Lagos

" doesn't give you anything" - except for without it I get "MicroBenchmark does not match a class" caused by ClassNotFoundException. Agreed, that it's not needed for the MICROBENCHMARK_LIST. – Homy

Well, you might be right. Somehow I was running it without it, but with all the problems above. And the configuration you had didn't work. Now to prove my point, i deleted and erased this project, re-imported and re-built it, and suddenly it works as intended... – Lagos

Answer 2

jmh-dev@ is a better way to communicate this with the developers.

Few things to try:

1. Hijacking Main is probably not a good idea. Use Java API instead, like this sample.
2. Use -v extra to debug the pattern matching: either the filter regexp is incorrect, or there are no benchmarks to run.
3. If the regexp is incorrect, fix it.
4. If there are no benchmarks to match against, then there is a chance resources are not generated and/or picked up properly. Make sure target/classes/ is also available on classpath.

Allpowerful

I am not doing anything with main, but running the exact piece of code you linked (JMHSample_01_HelloWorld.java)... target/classes is on classpath. However when debugging jmh-core code i can see it is trying to get classes from META-INF/Microbenchmarks which is under project root, not inside target/classes... – Lagos

Answer 3

I was also facing the same problem, and I followed the tutorial here. That solved the issue.

Below are the steps I took:

• I used the code from the tutorial AS-IS to understand how it works.

• Then I just did mvn clean and install

• I saw all set of classes being created in target -> annotations -> <package path> -> generated
• Then I ran the BenchmarkRunner main() class and this worked.

Giveandtake

A link to a solution is welcome, but please ensure your answer is useful without it: add context around the link so your fellow users will have some idea what it is and why it’s there, then quote the most relevant part of the page you're linking to in case the target page is unavailable. Answers that are little more than a link may be deleted. – Brickwork

Thank you @Das_Geek. I updated my answers to add more. Hope this adds a bit more relevance. – Giveandtake

No prob! Thanks for taking the time to improve your answer. – Brickwork