Why do Java 21 virtual threads need less memory than platform threads?

Question

I do understand that platform threads are expensive as it needs more memory and it's prone to CPU context switching.

But in case of virtual threads where an unimaginably large number of virtual threads can be served by a handful of platform threads, doesn't the virtual threads still need memory space to passivate the context/stack and then attach it to the carrier thread?

How does it make a difference in terms of memory?

Why doesn't spinning 10000 virtual threads die out of memory while 10000 platform thread does?

Both of them need the same stack? And the context where the application related information need to be maintained, right?

Is there any extra overhead in the memory that's only applicable to the platform threads and this is the reason why we say virtual threads are "lighter" in memory? If so what's that that makes this difference?

Jewelfish

Why 10,000 virtual threads don't exhaust memory in the way 10,000 platform threads would is recognizing that virtual threads are designed to be lightweight, with smaller stacks and reduced per-thread overhead. – Recline

Just for reference -- Why is creating a Thread said to be expensive?, it might help to understand the differences. Additionally, an async implementation similar to coroutines signals towards a stackless threading. – Vergara

Another thing is spinning a new thread. Spinning a new platform thread takes a lot of time and resources; spinning a new virtual thread - almost none. But if each of 1 million quickly and cheaply spun virtual threads will allocate in stack 100 Mb, then the JVM will die anyway. – Tonne

Did you check with a profiler? It will show you the amount of memory allocated for various objects. – Concussion

Answer 1

Both of them need the same stack.

Not true. In fact, that's probably the main "win" of virtual threads.

The call stack of a platform thread always must be contained within a dedicated, contiguous chunk of virtual memory--often around one megabyte, but you can specify larger if needed. That's how it works on every operating system. That's how it has always worked, ever since the idea of "call stack" was invented.

If you haven't done a careful analysis to determine exactly how much memory each of your threads really needs to hold its stack in the very deepest call, then you probably are over-allocating, just to be "on the safe side." Maybe you are over-allocating by orders of magnitude. Of course, it's only virtual memory, so if your program only has a few threads, no problem. But if you want to have thousands of virtual threads,... Suddenly, problem.

I don't actually know how the call stacks of java virtual threads are implemented, but I know that it is not in a single, contiguous allocation per thread.* So, at any given moment in time, no virtual thread has any more memory reserved for its stack than it actually needs.

*If I were implementing it, I imagine I would keep it in a linked list of heap-allocated objects. Maybe each activation record would be its own object, with a next field that points to the caller's activation record. Or maybe, it would be more efficient if it were allocated in larger "chunks." I wouldn't know until I tried.

Hest

What is your knowledge of virtual threads not allocating their stacks in a "single, contiguous allocation" based upon? – Tonne

@igor.zh, Brian Goetz (Java Language Architect for Oracle) says, "Virtual threads...store their stack frames in Javas garbage-collected heap rather than in monolithic blocks of memory..." infoq.com/articles/java-virtual-threads – Hest

Curious what of this preview-related blog still holds true for 21. Looks like not "there is no new VirtualThread base type" (unless I misread Dr. Goetz). He himself might chime in, though... But other than that, makes sense, thank you. – Tonne

@Tonne in which regard do you consider the blog’s statement wrong (or outdated)? It’s about the API, not internal classes. – Spacious

@Holger, yes, you are right, VirtualThread is package-private and since github.com/openjdk/loom/commit/… it is final. So, although such class exists, it is internal and not base. – Tonne

Why are OS threads done this way? Is it just historical remnant? Couldn't the OS provide stack memory that is allocated on-demand and can grow/shrink as needed ? As well as add support for non-contiguous stack space ? – Virescent

@DidierA. Believe it is because OS/platform threads are not language-specific. Java (and more so, C/C++) offers relatively fine-grained control, but not all languages do. Grow/shrink is not the same thing as "not in a single, contiguous allocation". – Tonne

@DidierA. in practice, the stack memory of a platform thread is virtualized and hence, only allocated on demand. It’s still a linear address range though and I think, there are some resources in use, just for the maintenance of the virtual address space. In principle, it would be possible to implement a non-contiguous stack space that adapts to the needs for every thread, as the JVM is in full control of the code that uses that stack. However, it would be a drastic change to a two-decades-old code base affecting a lot of places (interpreter, code generators, all garbage collectors, debuggers…) – Spacious

Answer 2

Are there any extra overhead in the memory that's only applicable to the platform threads

Yes. A very large amount: megabytes versus kilobytes, if even that much.

This is why virtual threads are considered very cheap in comparison.

Halfsole