Do Java 21 virtual threads address the main reason to switch to reactive single-thread frameworks?

Question

To my understanding one of the reasons to switch to a reactive (e.g. Project Reactor, RxJava, Vert-X) or actor (Akka) framework is because of the cost of thread switching is high. Looking at Difference between platform thread, carrier thread and virtual thread in context of Java 21 and later and other information about virtual threads I was wondering...

Do Virtual Threads remove the reason to switch to a different paradigm because it will just swap out the blocking virtual thread to a different thread on the carrier?

Tureen

If only your "reason to switch to a different paradigm" was that threads were blocking for I/O with the existing implementation, only then possibly, yes. I doubt one can otherwise compare all these libraries/frameworks with a feature in JDK in one place. Also, tends to be off-topic according to the website. – Privatdocent

Related: Reactive Programming Advantages/Disadvantages, especially this Answer. – Cantor

Answer 1

Yes, virtual threads eliminate the need for reactive approach

Platform threads in Java are mapped directly to a thread host operating system thread. Those OS threads are “expensive” in terms of memory and CPU.

Virtual threads, in contrast, are managed within the JVM. As a result, virtual threads are extremely “cheap”, meaning they are quite efficient in both memory and CPU. With virtual threads, you can reasonable expect to run even millions of tasks simultaneously on common computer hardware.

Yes, most, if not all, of the work done as reactive code can be done instead with Java virtual threads. The coding is vastly simpler to write, comprehend, trace, and debug. Reactive approach was invented to get around the performance problems of over-using platform threads.

See video of Brian Goetz being asked what he sees as the future of reactive programming after the arrival of Project Loom and virtual threads:

I think Loom is going to kill reactive programming … reactive programming was a transitional technology …

Making reactive programming unnecessary was one of the major motivations for inventing virtual threads in Java.

Virtual threads are for blocking code

Caveat… Virtual threads are contra-indicated for tasks that are CPU-bound such as video encoding/decoding. Use virtual threads only for code that involves blocking, such logging, file I/O, accessing databases, network calls. That would cover nearly all Java business apps.

Cheap threads can perform expensive tasks

Of course, the tasks performed by these many cheap virtual threads may involve significant resources. Those resources might include consumption of large amounts of memory, tying up a limited number of network ports, overloading a database with too many connections, and so on.

In the old days, Java programmers learned that the small number of platform threads that were practical for multi-threading indirectly acted as a throttle on excessive use of resources by the executing tasks. Now, with potentially millions of virtual threads, you may need to add explicit throttling of expensive tasks to conserve precious resources. You can use throttling mechanisms such as Semaphore permits or ReentrantLock.

For more discussion, see Answer by Teddy.

Pinning

A weakness in the current implementation of virtual threads is that in some cases the virtual thread is “pinned” to its carrier platform thread, meaning it cannot be set aside while blocked for that carrier platform thread to be assigned another virtual thread.

As of Java 22, pinning happens in at least two scenarios:

• synchronized code.
• Native code, called via a mechanism such as JNI.

To be clear: You can use synchronized and native code in your virtual threads. But if the task being executed includes significantly long periods of such work, then assign that task to a platform thread rather than a virtual threads. Or in the case of long code protected by synchronized (long-running code, not all code), replace with a ReentrantLock. Excessive pinning impairs the efficiency and effectiveness of other virtual thread usages.

The Project Loom team continues their work. They are looking into ways to decrease situations resulting in pinning. So the situation may change in future versions of Java.

You can easily detect protracted pinning. Java will emit a new JDK Flight Recorder (JFR) event, jdk.VirtualThreadPinned, every time a Virtual Thread gets pinned, with a threshold of 20ms by default.

For more info

For details, see firstly the official document, JEP 444: Virtual Threads. See also the official Project Loom site.

Then see the more recent videos of presentations by Ron Pressler, Alan Bateman, or José Paumard. Publicly available on YouTube, etc.

If you really want to understand how the impressive performance gains were made, see the talk by Ron Pressler on Continuations: Continuations - Under the Covers. To be clear: This understanding is entirely optional, unnecessary to making effective use of virtual threads. But if you need to satisfy your geek curiosity, you’ll enjoy that particular talk by Pressler.

Cantor

It might eliminate the need for the reactive approach implemented within a Java code, but IMHO the Q&A misleads to directing that virtual threads is a replacement to the mentioned frameworks. In fact, the other answer still has some relevant stats to ponder upon. – Privatdocent

@Privatdocent I added two sections on expensive tasks, and on pinning, to make more clear the appropriate cases where virtual threads can replace reactive programming. – Cantor

Virtual thread is of no use if the API client performs a blocking API call like the OkHttpClient. The API client should also be non-blocking like Spring Webclient which again uses reactive libs. – Monjo

Answer 2

Virtual Threads do away with most of the heavy-OS-Thread problems that reactive programming was solving.

However, with Virtual Threads you can still overwhelm the target resources.

When we had only about 500 to 2000 threads, we used to do thread pools for web-requests and so on, so we used to get throttled right at the entry point.

Now that we don't get throttled at the entry point at all (say the web server is using the new virtual thread per request model), then all the requests will now proceed through the system and hit the DB or file I/O or 3rd party service.

So, we need to throttle manually with rate limits and queues or again use reactive frameworks and backpressure. If we don't then we may get blocked by other resource bottlenecks or downstream API rate limits / we may get banned/blocked.

The same applies to CPU as well. A handful of CPU bound tasks can still hog the CPU and put 10000 virtual threads on park.

So, per-resource throttling has to be done more deliberately.

Krilov

We have Semaphore in Java for that, and in Helidon 4(VirtualThread based), you can use fault tolerance bulkhead(fancy semaphore). – Quincunx

Answer 3

Do Virtual Threads remove reason to switch to a different paradigm [like Reactive]

Theoretically, yes, it is an intention, but the reality is too complex to give a simple answer. The research, conducted by Daniel Oh in his DZone article Demystifying Virtual Thread Performance: Unveiling the Truth Beyond the Buzz shows that nothing, including virtual threads, could beat the performance of non-blocking Reactive application, based on Event Loop.

Notice that the tests did not involve only CPU intensive operations, discussed above, which are indeed not very suitable for virtual threads.

This difference in performance is quite understandable: non-blocking Reactive paradigm minimizes thread context switching: either full-blown, OS-based or lightweight, virtual thread/continuation-based. However, this paradigm mandates rather strict behavior when it comes to blocking vs non-blocking: Reactive-compliant code should be fully non-blocking. Indeed, if only one method on the execution stack won't be non-blocking, then "single" Event Loop gets stuck and the entire application is choked. Notice that the requirement applies not only to the code a develop writes, but also to the libraries this code uses. That's why Reactive programming requires non-blocking frameworks and libraries like web servers, Netty or Helidon 3, Http Client, Reactive-compliant JDBC drivers and so on. Reactive behavior of a JDBC driver is so specific that Oracle, for example, introduced special JDBC Reactive Extensions.

Needless to say that such a development strategy it is not the easiest path to follow and might not be always affordable. Instead, virtual threads offer (relatively) quick and easy, use-and-drop, drop-in-replacement way - at the expense, of course, of performance, if compared to Reactive paradigm.

In conclusion, if you need highest performance in your multithreading-intensive application and you could afford meticulous sole-to-crown, all-non-blocking development, then virtual threads may not be a replacement for Reactive paradigm.

In general, your question is too deep and complex for a SO answer and brief follow-up discussion and, IMHO, deserves more detailed research. Also, it is easy to see that in the explanation above all the concepts, like "single Event Loop" are taken to their extremes, while the reality is much more complex. For example, Netty does not have only one, but "few" Event Loops, where "few" is, as always, very loosely defined. "Fully" non-blocking code is another good example.

Dhumma

So, according to the study case, throughput and response performance tends to be the same for reactive and VTs on around 3.5k req/s which is far more than enough for the majority of the companies out there (google is around 40k req/s i think). Apart from that you have an additional 10% cpu gain and 80-100mb of ram on 3.5k req/s approximately. So the question is "am i willing to change the whole programming paradigm from imperative to reactive in order to gain 10% cpu and 80-100mb of ram?" That's a big no from me. Project loom is still ongoing with a promising set of improvements ahead. – Frohne