onSpinWait​() method of Thread class - Java 9

Question

While learning Java 9 features I came across a new method of Thread class, called onSpinWait​. As per javadocs, this method is used for this:

Indicates that the caller is momentarily unable to progress, until the occurrence of one or more actions on the part of other activities.

Can someone help me understand this method giving a real-life example or scenario?

Prosperity

javadoc for onSpinWait contains API note that explains how to use it – Zohar

Thread.onSpinWait() is an imlementation od the JEP-285 – Shonda

Answer 1

It's the same (and probably compiles to) as the x86 opcode PAUSE and equivalent the Win32 macro YieldProcessor, GCC's __mm_pause() and the C# method Thread.SpinWait

It's a very weakened form of yielding: it tells your CPU that you are in a loop that may burn many CPU-cycles waiting for something to happen (busy-waiting).

This way, The CPU can assign more resources to other threads, without actually loading the OS scheduler and dequeuing a ready-to-run thread (which may be expensive).

A common use for that is spin-locking, when you know the contention on a shared memory is very infrequent or finishes very quickly, a spinlock may perform better than an ordinary lock.

Pseudo code for such can look like:

int state = 0; //1 - locked, 0 - unlocked

routine lock:
    while state.cas(new_value=1, wanted_value=0) == false //if state is 0 (unlocked), store 1 (locked) and return true, otherwise just return false.
       yield

routine unlock:
    atomic_store(state,0)

yield can be implemented with Thread.onSpinWait(), hinting that while trying to lock the lock, the CPU can give more resources to other threads.

This technique of yielding is extremely common and popular when implementing a lock-free algorithm, since most of them depend on busy-waiting (which is implemented almost always as an atomic compare-and-swap loop). this has every real-world use you can imagine.

Becht

oh darn yes! totally my bad, was a reading that in the wrong direction. That is still puzzling to me... this is going to be used only when C2 compiler (aka JIT) kicks in. Until than, nothing would happen... – Blackcap

@Blackcap yes, but since onSpinWait is pure optimization, anyway you don't care about your Java application performance before the JIT kicks in. So it's a win-win situation – Becht

Answer 2

Pure system hinting!

Reading this article I quote:

Goals

Define an API that would allow Java code to hint to the run-time system that it is in a spin loop. The API will be a pure hint, and will carry no semantic behaviour requirements (for example, a no-op is a valid implementation). Allow the JVM to benefit from spin loop specific behaviours that may be useful on certain hardware platforms. Provide both a no-op implementation and an intrinsic implementation in the JDK, and demonstrate an execution benefit on at least one major hardware platform.

There are many times a thread must be suspended until something outside its scope changes. A (once) common practice was the wait() notify() pattern where there's one thread waiting for the other thread to wake them up.

There is a big limitation to this, namely, the other thread must be aware that there might be waiting threads and should notify. If the work of the other thread is outside your control, there's no way to get notified.

The only way would be a spin-wait. let's say you have a program that checks for new emails and notifies the user:

while(true) {
    while(!newEmailArrived()) {
    }
    makeNotification();
}

This piece of code will execute millions of times a seconds; spinning over and over, using precious of electricity and CPU power. A common way of doing this would be to wait a few seconds on each iteration.

while(true) {
    while(!newEmailArrived()) {
        try {
            Thread.sleep(5000);
        } catch(InterruptedException e) {
        }
    }
    makeNotification();
}

This does a very good job. But in cases where you have to work instantly a sleep may be out of question.

Java 9 tries to solve this issue by introducing this new method:

while(true) {
    while(!newEmailArrived()) {
        Thread.onSpinWait();
    }
    makeNotification();
}

This will work exactly the same as without the method call, but the system is free to lower the process priority; slowing the cycle or reduce electricity on this loop when its resources are needed for other more important things.

Natterjack

While the above is correct, the example is far from ideal. Spin locks are for cases where you're waiting for the lock rarely and, if you are, for a very short time; at most a millisecond. The whole point of having spin locks is to not have the overhead of a normal lock because you're locking a lot (e.g. lock free data structures) and you can't take the hit of a normal lock overhead. This really is not the case while waiting for new mail to arrive, in which case you'll really want to use a normal lock mechanism. Long story short the chances you're going to ever need this are slim. – Flirtation

@PieterVanGinkel Yeah it's mentioned in the linked article, but I couldn't think af a better example. In fact they also say that "latency sensitive" applications, as a Disruptor might use spin-wait for long tasks too. – Natterjack

An appropriate place to use would be implementation of lock-free concurrent algorithm. Like ConcurrentHashMap of JDK, when it needs to spin wait, the author Doug Lea uses Thead.yield(). If he rewrites this now, Thread.onSpinWait() would fit better. – Pani

Answer 3

I just want to add my 2 cents after reading the document and the source code for it. This method might trigger some optimizations and might not - so care must be taken - you can't really rely on it - since this a hint to the CPU more than demand, probably there's no initial relying whatsoever either way. Meaning that it's actual source code looks like this:

@HotSpotIntrinsicCandidate
public static void onSpinWait() {}

What this actually means is that this method is basically a NO-OP until it hits the c2 compiler in the JIT, according to this.

Blackcap

Answer 4

For a real-world example, say you wanted to implement asynchronous logging, where threads that want to log something, don't want to wait for their log message to get "published" (say written to a file), just so long as it eventually does (because they've got real work to do.)

Producer(s):
concurrentQueue.push("Log my message")

And say, you decide on having a dedicated consumer thread, that is solely responsible for actually writing log messages to a file:

(Single)Consumer

while (concurrentQueue.isEmpty())
{
    //what should I do?

}
writeToFile(concurrentQueue.popHead());
//loop

The issue is what to do in inside the while block? Java has not provided ideal solutions: you could do a Thread.sleep(), but for how long and that's heavyweight; or a Thread.yield(), but that's unspecified, or you could use a lock or mutex*, but that's often too heavyweight and slows the producers down as well (and vitiates the stated purpose of asynchronous logging).

What you really want is to say to the runtime, "I anticipate that I won't be waiting too long, but I'd like to minimize any overhead in waiting/negative effects on other threads". That's where Thread.onSpinWait() comes in.

As a response above indicated, on platforms that support it (like x86), onSpinWait() gets intrinsified into a PAUSE instruction, which will give you the benefits that you want. So:

(Single)Consumer

while (concurrentQueue.isEmpty())
{
    Thread.onSpinWait();

}
writeToFile(concurrentQueue.popHead());
//loop

It's been shown empirically that this can improve latency of "busy-waiting" style loops.

I also want to clarify, that it is not just useful in implementing "spin-locks" (although it's definitely useful in such a circumstance); the code above does not require a lock (spin or otherwise) of any kind.

If you want to get into the weeds, you can't do better than Intel's specs

*For clarity, the JVM is incredibly smart in attempting to minimize the cost of mutexes, and will use lightweight locks initially, but that's another discussion.

Ostap