I attended PJUG last night and they had a talk about microbenchmarking in Java. The talk started with what might be the most important things to keep in mind for any benchmarking code.
- avoid dead code elimination by always return output or use blackhole
- An example is that if the code you are benchmarking doesn’t do anything, the optimizer might interpret it as dead code and never translate that into bytecode at all. However, if the result in the dead code is being return, then optimizer won’t be able to eliminate the code away.
- avoid constant folding by reading input from state objects
- If at some point inside the code for benchmarking where a function is taking a constant as argument and return the same value every time , then the optimizer again will interpret the return value as a constant as well even though you might be benchmarking the execution of the function. In this case, move the constant out and make it into a field in your class so that the function argument becomes a variable.
Other interesting topic were monomorphic, bimorphic and megamorphic, and concurrency issue with jmh. The full presentation slides is here.