Should virtual thread die fast?

Question

It' s recommended by JDK developers that virtual threads should never be pooled, because they are really cheap to create and destroy. I am a little confusing about the pooling idea, since pooling usually means two things:

1. The resources should be reused
2. The resources will have a long lifecycle until it's released

I understand that JDK developers want us to never reuse a virtual thread, and the lifecycle question confuses me, because if there are multiple virtual thread having lifecycle as long as the application itself, it might sounds like pooling without reuse.

So should a virtual thread die fast, or having a short bounded lifecycle, or is it OK that multiple virtual threads were blocking, once in a while waken up to process some tasks, and having a really long lifecycle?

Italian

Why does a long-lived thread “sound like pooling”? Pooling means that you are planning to re-use a theory, running successive tasks. Sot pooling means running only a single task. How long that single task takes is irresistible to the definition of pooling. – Briseno

@BasilBourque From the perspective of the JDK developers who invented virtual threads, I think reuse and long-lived should have the same effect, and they suggested not pooling, so I got this question, it's not from the perspective of how we use virtual thread, but from their suggestions about how we should do about it. – Italian

You mentioned "JDK developers that virtual threads should never be pooled, they are really cheap to create and destroy," which implies that the reason for thread pooling is the high cost of creating and destroying threads. If your case is to have multiple virtual threads with a lifecycle as long as the application itself, it sounds like the runtime cost of thread creation and destruction can be ignored, so it is less relevant to thread pooling. – Burmese

Fixing the typos in my Comment above: Why does a long-lived thread “sound like pooling”? Pooling means that you are planning to re-use a theory, running successive tasks. So, not-pooling means running only a single task. How long that single task takes is irrelevant to the definition of pooling. – Briseno

FYI: A pool of threads may be short-lived, used to run a limited number of tasks, and then closed. – Briseno

The JEP 425 openjdk.org/jeps/425, which was updated on 2023/06/07, says in its "Implications of virtual threads" section: "Most virtual threads will thus be short-lived and have shallow call stacks, performing as little as a single HTTP client call or a single JDBC query". Short-lived means, if my English does not leave me, "die fast", so if to follow the JEP verbatim, then the answer to your question is yes! In reality, I think, Project Loom is not mature enough to make the statements like that and, I am afraid, even its creators are not perfectly sure. – Karilynn

@Karilynn You made a massive, unwarranted leap there. The statement "most virtual threads will this be short lived" is a statistical characterization to aid intuition, not a prescriptive rule. (This should be obvious.) To leap from there to "yes, virtual threads should die fast" is misleading and unhelpful. – Nodical

@BrianGoetz, I might have made a leap, but I think, you also went overboard accusing me of a massive one. Still "short-lived", linguistically, means "dies fast". But I agree, it is not longevity of a thread that matters, but its "lightweightness". Even a short-lived thread might be heavyweight, if, for example, its stack is not "shallow". – Karilynn

@Karilynn Perhaps I was not clear what I meant. The leap was not from "short lived" to "dies fast" (that part is fine), it was from "most will" to "all should" (more precisely, that "the answer to 'should they die fast' is 'yes'.") Virtual threads should do a single task, but they can take as long as they need to get it done. While most will in fact be short-lived, it is the single-use nature that is the key characteristic, not the duration or the stack depth. – Nodical

Answer 1

There are several main reasons why objects might be pooled.

1. They are expensive to create or destroy. Pooling means that you get to amortize these costs over multiple uses.

2. They consume a finite resource that you want to manage the consumption of. Pooling means that you can bound the resource consumption by setting a limit to the pool.

Both of these considerations apply to platform threads; creating them is expensive, and they consume significant memory. If you created a platform thread for every task, you could easily run out of memory.

Note that pooling is very rarely a benefit on its own; at best, it is better than the alternatives (such as unbounded thread creation.) Used objects are usually worse than new objects, since they might have leftover state from previous use (such as orphaned ThreadLocal values.)

Neither of these considerations apply to virtual threads.

It is possible, however, that code running in a virtual thread may require the use of a finite resource, such as a database connection. In these cases, you want to manage the consumption of those resources, such as pooling the connections, or using a semaphore to limit how many of a certain kind of task can run at once. But none of these considerations argue for pooling virtual threads.

Nodical

Answer 2

tl;dr

The reason to choose a platform thread over a virtual thread is not how long-lived the task but rather (a) if the task does little to no blocking or (b) if the task has long-running code that is synchronized or native. Otherwise, virtual threads are recommended.

Details

I suspect you are overthinking things. The situation is not really that complicated.

In Java 21 and later…

Define any task you want to run as a Runnable or a Callable.

Pass that task to an Executor instance via its execute method if you don’t care whether it runs concurrently or not.

If you definitely want that task to run concurrently, pass to an ExecutorService via the submit or invoke… methods.

If that task involves blocking (is not CPU-bound), use an executor service that assigns one fresh new virtual thread to each task. “Fresh” means a new, clean, minimal stack, and no pre-existing ThreadLocal objects. “Blocking” means waiting on something so that the thread cannot do any further work; basically any I/O such as logging, file reading/writing, calls to a database, and network traffic such as sockets & Web Service calls. Virtual threads are extremely “cheap”, meaning fast to create, efficient with memory, and efficient with CPU.

In other words, virtual threads are like facial tissues: Whenever needed, grab a fresh new one, use it, and dispose.

If your task involves long-running code marked synchronized, either replace the use of synchronized with a ReentrantLock or else run it with a platform thread as described next.

If your task (a) does not involve blocking (is CPU-bound such as video encoding), or (b) calls native code (JNI or Foreign Function), then do not use a virtual thread. Submit such tasks to an executor service backed by a platform thread(s). If concerned about overburdening your computer, use an executor service backed by a pool of a limited number of threads.

Platform threads are “expensive”. So virtual threads are preferred, given your task meets the conditions described above. A task in a virtual thread may run briefly or for a long-time, even the entire duration of your app.

When using pooled threads, be careful to clear out your ThreadLocal objects to avoid inadvertent use by successive tasks in that thread.

Always shutdown an executor service before ending your app. Otherwise the backing threads may run indefinitely, like zombies 🧟‍♂️. Either use try-with-resources syntax to auto-close, or use boilerplate code given in the ExecutorService Javadoc.

FYI, virtual threads actually run your task on a platform thread in a pool automatically managed by the JVM.

To learn more, read the JEP linked above. And see talks by Ron Pressler, Alan Bateman, and José Paumard.

Briseno

Thanks for the answer! I want to know if it's recommended to use virtual thread to implement a producer-consumer mechanism using BlockingQueue, in which case the consumer thread might never die until the application exit. – Italian

@Italian You should post that as another Question, a better, more focused Question. – Briseno