Control number of operation per iteration JMH

Question

My current setup:

    public void launchBenchmark() throws Exception {
    Options opt = new OptionsBuilder()
            .include(this.getClass().getName())
            .mode(Mode.Throughput) //Calculate number of operations in a time unit.
            .mode(Mode.AverageTime) //Calculate an average running time per operation
            .timeUnit(TimeUnit.MILLISECONDS)
            .warmupIterations(1)
            .measurementIterations(30)
            .threads(Runtime.getRuntime().availableProcessors())
            .forks(1)
            .shouldFailOnError(true)
            .shouldDoGC(true)
            .build();

    new Runner(opt).run();
}

How can I know/control (if possible) the number of operations is performed per benchmark ?

And is it important to set warmUp time and measurementIteration time?

Thank you.

Monogenetic

Answer 1

You cannot control the number of operations per iteration. The whole point of JMH is to correctly measure that number.

You can configure the warmup using the annotation:

@Warmup(iterations = 10, time = 500, timeUnit = MILLISECONDS)

And the measurement by:

@Measurement(iterations = 200, time = 200, timeUnit = MILLISECONDS)

Just set the appropriate values for your use case

Herron

ahh I see. Do you have a good suggestion on the iteration time? e.g. if I know each operation will cost x amount of time. Then how much time i should set for each iteration ? – Monogenetic

It depends. With default settings the JIT uses C1 to compile to native after 10k iterations, and uses C2 after 30k. So you should set the number of iterations and the iteration time large enough to make sure the JIT has compiled everything using the C2 compiler, otherwise your benchmark will report incorrect statistics. If you set those values too high it will be just a waste of time, but no other negative effect. – Herron

Also the iteration time must be large enough to compensate for some jitter, but it again depends on how long the operation takes. For instance if a singl eoperation takes 100ms, then you would want aniteration time of several seconds at least, but if it takes a nanosecond -> then 100ms would be more than enough – Herron

"You cannot control the number of operations per iteration. The whole point of JMH is to correctly measure that number." What if I am measuring the time taken per operation? In this case does it make sense to be able to control the number of operations per iteration? – Lawyer