JMH State classes and shared vs unshared states

Question

I'm new to jmh and to understanding what happens behind threads and so on.

So, I started reading and got stuck on the @State annotation and shared vs unshared states.

I read this example : http://hg.openjdk.java.net/code-tools/jmh/file/ecd9e76155fe/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_03_States.java and have few questions about it.

First question, what is the exact role of state classes? to hold parameters? let's say I want to benchmark a program that encrypts a key in 2 different ways. Should i keep the key (a String object) in a state class which annotated with a specific state? or just keep the String object on the benchmark class? An explanation about this would be great.

Second question, why in the example above the unshared state class performance was much better than the shared one? How does the multithreaded state changes it?

I feel really obscured since i'm new to this thing and couldn't find an "explain me like i'm 5" examples for jmh and it's options.

Rivalee

Answer 1

You can consider @State objects as the part of your benchmark that you need to run it without that the time for its creation should be considered as a part of your measured time.

Let us say that you want to measure the time it takes to compute:

@Benchmark
int benchmark() {
  int foo = 1, bar = 1;
  return foo + bar;
}

Unfortunately for you, the JIT compiler is too smart to let you do this and will fold the method to simply return 2. This is of course not what you want to measure. Using state, you can escape these values and let JMH take care of not letting the JIT fold its values. You would initialize values in a @Setup method.

As another use case, you can check that your benchmark did what you expected. This is possible by validating state in a @TearDown method.

Mayenne

So, does it really matter if I keep my variables in the benchmark class and initialize them with @Setup or creating an inner class which keeps the variables and initializes them. Or it's just 2 ways to do so. – Rivalee

The first is just a convenience for the latter if this is easier for you. – Mayenne

Answer 2

First question, what is the exact role of state classes? to hold parameters? let's say I want to benchmark a program that encrypts a key in 2 different ways. Should i keep the key (a String object) in a state class which annotated with a specific state? or just keep the String object on the benchmark class? An explanation about this would be great.

The benchmark class is also a state class. See JMHSample_04_DefaultState.java.

Second question, why in the example above the unshared state class performance was much better than the shared one? How does the multithreaded state changes it?

This is an issue of “modern” processors and not of JMH. Each core on the processor has its own L1 (and maybe L2) cache. They usually don't access the RAM directly. If multiple threads are constantly writing to the same area of memory, the processor is constantly busy to synchronize the data between all cores. You don't actually have to access the same variable to get this effect. See JMHSample_22_FalseSharing.java.

Uncovenanted

Answer 3

The two main benefits of using a state class are:

• You guarantee the processes for data preparations are not measured in the benchmark.
• You can clearly define the scope of the state objects, therefore having higher control over and knowledge of what is truly being benchmarked.

This article explains the State class and state objects with more detail: https://www.oracle.com/technical-resources/articles/java/architect-benchmarking.html

Anecdotage