Java - performance of object pool vs new object instantiation

Question

I'm currently trying to build some code execution optimization for a contest, and was looking at the ObjectPool pattern to favor object reuse instead of new object instantiation.

I've put together a small project (and the only test class) to investigate some of the things I see and don't understand.

What I'm doing:

• compare the creation of very simple objects for 5 000 000 iterations using both the new() and Pool.get() operations
• play around three axes, running all tests with and without:
  • a "warmup" that runs the loop once before doing the measurements
  • assigning the newly creating object to a local variable and using it for some computation
  • using fixed vs random parameters as arguments

The results I have are: Figures are for new instantiation vs with object pool for 5 000 000 iterations without_warmup_without_new_object_use_with_random_parameters: 417 vs 457 without_warmup_without_new_object_use_with_fixed_parameters: 11 vs 84 without_warmup_with_new_object_use_with_random_parameters: 515 vs 493 without_warmup_with_new_object_use_with_fixed_parameters: 64 vs 90 with_warmup_without_new_object_use_with_random_parameters: 284 vs 419 with_warmup_without_new_object_use_with_fixed_parameters: 8 vs 55 with_warmup_with_new_object_use_with_random_parameters: 410 vs 397 with_warmup_with_new_object_use_with_fixed_parameters: 69 vs 82

What I notice from that:

• Using fixed parameters has a huge impact when instantiating a new object without reusing it. My guess was that the compiler was doing some kind of optimization and found that there was no side-effects and would remove the object instantiation altogether, but comparing the perfs with an empty loop shows that something still happens
• Using fixed parameters has a significant impact (though less pronounced) for the speed of new Object(), making it faster than the object pool version in some cases
• The object pool is faster in the "real life" scenarios (ie reuse the new objects and use somewhat random params), but not in most of them, which also hints at a compiler optimization.

What I'm looking for here is to understand these results, and get pointers to docs / books that I could read to get a good knowledge of what happens behind the scenes in these cases.

Thanks!

Meingoldas

You seem to be mainly benchmarking your benchmarks. Use a proven benchmarking framework like JMH. You have already seen how sneaky the JIT can be. – Tohubohu

Also, the differences you witness between what you think of as "test with random parameters" and what you think of as "test with fixed parameters" can be attributed in full to the cost of invoking random.nextDouble() rather than to any compiler / JIT optimizations that you are hypothesizing. – Lanfri

In any case, this is an exercise in futility: of course your pool will seem to perform faster than memory allocation, but then your pool is too simplistic, it does not really work, (fill it up and you start reusing objects previously allocated,) and it does not free anything. By the time you add to it enough code to fix it, it is almost guaranteed that it will be performing worse than memory allocation. – Lanfri

Thanks for the advice, I will try with a real profiler to understand point 3) and the overall difference in perfs. Still open to any advice regarding the fixed vs non-fixed parameters. @MikeNakis just to be sure we're talking of the same thing, I'm only interested in the differences, notably how the fixed params have an impact when comparing the "with object reuse" and "without object reuse" for new instantiations, while it has very little impact for the object pool – Expose

@MikeNakis fill it up and you start reusing objects previously allocated Not sure I get your meaning here. The pool is filled and reuses items as far as I can see. As for freeing objects, that's something I'll want to have a look at later on. My use case is pretty simple, so a flag on the object (or cloning the few objects I want to keep) might be enough. – Expose

What I mean is that if the number of allocations exceeds POINT_POOL_SIZE then you will be reusing points which may be in use. Which is actually to be expected, since you do not even have a concept of freeing. Once you add the concept of freeing, things will become a lot more complicated. The java language runtime solved all of these problems a long time ago when implementing operator new. – Lanfri

Once you add the concept of freeing, things will become a lot more complicated: got it, In this particular case, the objects are instantiated for a short processing and can be reused afterwards, so do I need to explicitly free them? – Expose

if you don't explicitly free them, (mark them as available in the pool,) then once you reach the end of your pool you cannot warp around to the start of the pool, (as you do now,) because if you do that then you will be reusing objects that are already in use. – Lanfri

I hope this helped – Dewclaw

Answer 1

Fixed Parameters

As mentioned in the comment by Mike Nakis the difference between your tests with random parameters vs those with fixed parameters is entirely due to the expense of generating the random number, a fairer test might be to generate a 10 million entry array of random integers (1 for each parameter needed to initialise a Point) before engaging the loop and comparing that to a 10 million entry array of number picked by you (i.e 1 and 2) that way you are comparing like for like, without including the expense of the random number generation in your test results.

Performance

The reason why your pool is performing worse than initialising new objects each time(at least in terms of execution time), is because the object that you are storing in your pool is a relatively trivial object that will take next to no time to initialise. As such the conditional statement that you are evaluating:

if (pointIndex >= POINT_POOL_SIZE) {
            pointIndex = pointIndex - POINT_POOL_SIZE;
            totalPointLoops++;
        }

as well as indexing the array, is requiring more execution time than your Point object needs for init.

You might be making some memory saving in your example, however this seems unlikely as the objects that you take from the pool are never released back to the pool for reuse(from what i can see in your code). Also a simple loop probably isnt the best way to test an object pool as the purpose of an object pool is to have a cache of objects that are expensive to create/prone to failure/etc, and they tend to get used for longer than a simple loop iteration, chances are you are only really using one object at a time.

relevant info

Here are some good links to information on object pools in Java:

• http://www.oodesign.com/object-pool-pattern.html
• http://www.javaworld.com/article/2076690/java-concurrency/build-your-own-objectpool-in-java-to-boost-app-speed.html
• http://www.blackwasp.co.uk/ObjectPool.aspx

Dewclaw

Thanks Jan! I followed the concensus and did some proper profiling (I fought for some time with VisualVM but went for the trial version of YourKit in the end), and the profiling showed that the object pool was actually slightly faster, even for such simple objects. Thanks for your help! – Expose