Are objects prefetched from an array of references in Java?

Question

Imagine that we have a 1000 objects of the same type scattered across the memory (they were created at different times and other objects have been created in between).

We have an array which holds references to each of the 1000 objects.

Question

If we iterate over the array sequentially, what will be prefetched into the cache of the CPU? Only the references which the array holds or will those references be dereferenced and the objects loaded into the cache as well?

Does Java (the JVM) implement some kind of software prefetching? If not, are there libraries which provide software prefetching?

Tax

Sounds as if it would be implementation specific. I also strongly believe it depends on the software, as memory ordering may affect such things. Could be wrong on that last part though. – Excaudate

I'm pretty sure the CPU cache is 100% under control of the hardware. It's possible that as the garbage collector works, it consolidates your 1000 objects to a common area of memory. Nothing I've read however indicates that any GC does this. Hardware caches are normally full of important data anyway, any "pre-cache" is limited to one cache line, usually about 32-16 bytes. – Killarney

The Heap is divided into young and old space.The young space have recent allocated objects.When these object grow old, they are shifted to the old area.The "keep" is the place of recently allocated objects, which move swiftly and not to be garbage collected. – Fehr

I do not think that any CPU pre-fetches references. This would not make sense, because If your object's byte size modulo 64 (cache lines are typically 64 byte long) is relatively small, you loose a lot of cache (which, overall, may slow down your program). Furhtermore, the CPU would need to analyze your code to see whether you actually use the object behind the reference or only the reference (keep in mind that the same CPU may run C/C++ code). – Sartre

Nope. And this is a serious performance problem for Java. And that is why they are trying to add "value types" in java9 (or later), so that an array of things are located in the same memory region. – Fusco

Meanwhile, if you are really concerned with this issue, you can do something yourself; most primitively, use a byte[] to store all your data :) – Fusco

Answer 1

After some research, the most common JVM implementation (HotSpot) used to support prefetching. But this has been removed, since there is no practicle use for them. Thanks to @apangin for the link to the bug report.

As @markspace mentioned, objects get re-arranged for easier access during collections - this is called "compacting", and is present in the default GC used by HotSpot. You shouldn't need to worry about such underlying details, as the VM handles that for you.

A little deeper into compacting..

You've probably heard of "Stop-The-World" - this occurs when the object graph is in an inconsistent state. Objects are being moved around, so a thread might access an object thats no longer there. There are some GC implementations that are considered "pauseless", such as the Shenandoah GC, which use a forwarding pointer to allow a thread to access an object that was recently moved.

The point being, you don't need to worry about where an object may be located in memory, or how far the location is from another object. The VM was designed to take care of these decisions for you.

The final answer

So, are objects prefetched from an array of reference? You really shouldn't worry about it. You use Java to not have to care about these underlying details.

If you're REALLY interested in such details (maybe there's some strange bug you're encountering), as I've mentioned before, it's implementation specific, and you'd have to be specific about which implementation you're referring to.

Although, like I said before, it's Java; stop worrying about things you don't need to worry about. I can't emphasize this enough.

Excaudate

Agreed. The main point of Java is that you (most of the time) don't have to care about such low level details; the JIT is trained with the experience of thousands of projects and people to make Java code run efficient. Time and effort invested in one place to free brain power of millions of developers. – Usa

I've answered here about prefetch support in HotSpot JVM. However, recently these intrinsics have been completely removed from HotSpot development repository with the justification that there is no practical use for them. – Groundmass

@Groundmass Makes sense. Do you have a link to the source of this information? Or is it not documented – Excaudate

Here is a corresponding bug link. – Groundmass