How to use Java 12's Microbenchmark Suite?

Question

According to JEP 230: Microbenchmark Suite, there exists a microbenchmark suite built-in to Java 12. The JEP explains that it's basically JMH, but without needing to explicitly depend on it using Maven/Gradle. However, it doesn't specify how to go about accessing the classes/annotations that belong to the suite to perform a benchmark.

My questions are:

• Is there a specific Java module that I need to require in my module-info.java to be able to use this suite?
• What package(s) are the classes/annotations of this suite located in?
• Are there any major differences between this suite and JMH?

Gagne

As far as I understood, this is not a tool for you to create application benchmarks. This is for creating benchmarks for the JDK code, which are not even included in a regular JDK build. As stated on the linked page, “The benchmark suite is not required for regular developer JDK builds; it will be a separate build target.” – Rilda

Answer 1

Your interpretation is incorrect. The JEP says:

Add a basic suite of microbenchmarks to the JDK source code, and make it easy for developers to run existing microbenchmarks and create new ones.

i.e. this is not necessarily something that makes it into a JDK distribution, just something that is added to the source code repository to make it easier to run benchmarks on JDK code. Though, to be fair, the fact that it's listed as one of the JDK 12 'features' seems a tad misleading.

The benchmarks can be run by using the OpenJDK build system. Once you have cloned the OpenJDK source code from https://github.com/openjdk/jdk (or another repository that includes the JEP), you can run benchmarks e.g. by using:

make test TEST="micro:java.lang.reflect"

Benchmarks are located in the \test\micro\ directory. See also the documentation: https://github.com/openjdk/jdk/blob/master/doc/testing.md#microbenchmarks

Also, this requires you to specify JMH and it's dependencies when generating a build configuration:

bash configure --with-jmh="/path/to/jmh/jars"

The specified path should point to a directory containing the JMH jars. Required jars are: commons-math3, jmh-core, jmh-generator-annprocess, and jopt-simple.

Vergne

It’s not the first time that something is listed as feature of a new Java version that is merely an internal implementation detail, like “code cleanup” or “finally get all the source code repositories into one”, etc. Likewise, the list does not make a difference between general Java features and features of the reference implementation aka HotSpot/OpenJDK. – Rilda

@Rilda Yeah, the things listed on the JDK 12 page seem to be a dump of the JEPs with fixVersion 12 from JBS: bugs.openjdk.java.net/browse/… – Vergne