JMH throughput vs average results
Asked Answered
G

0

6

Throughput mode gives me 21 ops/ms. So, I'd thought on average one op takes 1/21 ms. Average mode however gives me the score of 0.937 ms/op. How to interpret the results?

Throughput mode:

Result "testMethod":
  21.752 ±(99.9%) 0.081 ops/ms [Average]
  (min, avg, max) = (20.681, 21.752, 22.529), stdev = 0.344
  CI (99.9%): [21.671, 21.834] (assumes normal distribution)


# Run complete. Total time: 00:06:47

Benchmark                Mode  Cnt   Score   Error   Units
MyBenchmark.testMethod  thrpt  200  21.752 ± 0.081  ops/ms

Average mode:

Result "testMethod":
  0.937 ±(99.9%) 0.004 ms/op [Average]
  (min, avg, max) = (0.890, 0.937, 0.979), stdev = 0.019
  CI (99.9%): [0.932, 0.941] (assumes normal distribution)


# Run complete. Total time: 00:06:47

Benchmark               Mode  Cnt  Score   Error  Units
MyBenchmark.testMethod  avgt  200  0.937 ± 0.004  ms/op
Gussy answered 9/5, 2017 at 10:55 Comment(3)
Ok, that's because I've used -t MAX (20 threads). Removing threading option makes results consistent. Still, multithreaded version is not clear to meGussy
I'm not sure why this is surprising. 20 threads working at the same time on an operation that takes 0.937 ms/op on a single thread, or 1.07 ops/ms/thread, makes 20 * 1.07 ops/ms total, which is roughly 21.Brag
Agree. I was confused that throughput is given for all threads but avg per thread. Would be nice to have more verbose summaryGussy

© 2022 - 2024 — McMap. All rights reserved.