We are working on a latency sensitive application and have been microbenchmarking all kinds of methods (using jmh). After microbenchmarking a lookup method and being satisfied with the results, I implemented the final version, only to find that the final version was 3 times slower than what I had just benchmarked.
The culprit was that the implemented method was returning an enum object instead of an int. Here is a simplified version of the benchmark code:
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@State(Scope.Thread)
public class ReturnEnumObjectVersusPrimitiveBenchmark {
enum Category {
CATEGORY1,
CATEGORY2,
}
@Param( {"3", "2", "1" })
String value;
int param;
@Setup
public void setUp() {
param = Integer.parseInt(value);
}
@Benchmark
public int benchmarkReturnOrdinal() {
if (param < 2) {
return Category.CATEGORY1.ordinal();
}
return Category.CATEGORY2.ordinal();
}
@Benchmark
public Category benchmarkReturnReference() {
if (param < 2) {
return Category.CATEGORY1;
}
return Category.CATEGORY2;
}
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder().include(ReturnEnumObjectVersusPrimitiveBenchmark.class.getName()).warmupIterations(5)
.measurementIterations(4).forks(1).build();
new Runner(opt).run();
}
}
The benchmark results for above:
# VM invoker: C:\Program Files\Java\jdk1.7.0_40\jre\bin\java.exe
# VM options: -Dfile.encoding=UTF-8
Benchmark (value) Mode Samples Score Error Units
benchmarkReturnOrdinal 3 thrpt 4 1059.898 ± 71.749 ops/us
benchmarkReturnOrdinal 2 thrpt 4 1051.122 ± 61.238 ops/us
benchmarkReturnOrdinal 1 thrpt 4 1064.067 ± 90.057 ops/us
benchmarkReturnReference 3 thrpt 4 353.197 ± 25.946 ops/us
benchmarkReturnReference 2 thrpt 4 350.902 ± 19.487 ops/us
benchmarkReturnReference 1 thrpt 4 339.578 ± 144.093 ops/us
Just changing the return type of the function changed the performance by a factor of almost 3.
I thought that the sole difference between returning an enum object versus an integer is that one returns a 64 bit value (reference) and the other returns a 32 bit value. One of my colleagues was guessing that returning the enum added additional overhead because of the need to track the reference for potential GC. (But given that enum objects are static final references, it seems strange that it would need to do that).
What is the explanation for the performance difference?
UPDATE
I shared the maven project here so that anyone can clone it and run the benchmark. If anyone has the time/interest, it would be helpful to see if others can replicate the same results. (I've replicated on 2 different machines, Windows 64 and Linux 64, both using flavors of Oracle Java 1.7 JVMs). @ZhekaKozlov says he did not see any difference between the methods.
To run: (after cloning repository)
mvn clean install
java -jar .\target\microbenchmarks.jar function.ReturnEnumObjectVersusPrimitiveBenchmark -i 5 -wi 5 -f 1
.