Threads per Processor

Question

In Java, is there a programmatic way to find out how many concurrent threads are supported by a CPU?

Update

To clarify, I'm not trying to hammer the CPU with threads and I am aware of Runtime.getRuntime().availableProcessors() function, which provides me part of the information I'm looking for.

I want to find out if there's a way to automatically tune the size of thread pool so that:

• if I'm running on a 1-year old server, I get 2 threads (1 thread per CPU x an arbitrary multiplier of 2)
• if I switch to an Intel i7 quad core two years from now (which supports 2 threads per core), I get 16 threads (2 logical threads per CPU x 4 CPUs x the arbitrary multiplier of 2).
• if, instead, I use a eight core Ultrasparc T2 server (which supports 8 threads per core), I get 128 threads (8 threads per CPU x 8 CPUs x the arbitrary multiplier of 2)
• if I deploy the same software on a cluster of 30 different machines, potentially purchased at different years, I don't need to read the CPU specs and set configuration options for every single one of them.

Milks

Answer 1

Runtime.availableProcessors returns the number of logical processors (i.e. hardware threads) not physical cores. See CR 5048379.

Fidele

Ah perfect! This just made my day :) – Milks

This answer allowed me to reject a feature request, thanks !! – Andreas

Answer 2

A single non-hyperthreading CPU core can always run one thread. You can spawn lots of threads and the CPU will switch between them.

The best number depends on the task. If it is a task that will take lots of CPU power and not require any I/O (like calculating pi, prime numbers, etc.) then 1 thread per CPU will probably be best. If the task is more I/O bound. like processing information from disk, then you will probably get better performance by having more than one thread per CPU. In this case the disk access can take place while the CPU is processing information from a previous disk read.

I suggest you do some testing of how performance in your situation scales with number of threads per CPU core and decide based on that. Then, when your application runs, it can check availableProcessors() and decide how many threads it should spawn. Hyperthreading will make the single core appear to the operating system and all applications, including availableProcessors(), as 2 CPUs, so if your application can use hyperthreading you will get the benefit. If not, then performance will suffer slightly but probably not enough to make the extra effort in catering for it worth while.

Wendy

I am already using availableProcessors to get the number of CPUs. What I'd like to do is to further tune the size of my thread pool so that if Intel's HT or some other multi-threading technology is available, I can take advantage of it. – Milks

Intel HT will show in the value returned from availableProcessors. If you have a dual core CPU with HT, then availableProcessors will return 4. – Wendy

Up-vote for available Processors will return number of threading paths and not cores. Even the OS tools see this for example check windows task manager under usage history and you will be informed that you have 4 CPUs for 2 cores with hyperthreading. Same info can be seen on Linux and since JVM uses native threads (in almost all implementations) Java will think the same as your OS. – Journey

Answer 3

There is no standard way to get the number of supported threads per CPU core within Java. Your best bet is to get a Java CPUID utility that gives you the processor information, and then match it against a table you'll have to generate that gives you the threads per core that the processor manages without a "real" context switch.

Bringhurst

From all the googling I've been doing, that does indeed seem to be the case :( Thanks for the CPUID utility idea, though. It's not exactly ideal but it's close enough to what I want. – Milks

Answer 4

Each processor, or processor core, can do exactly 1 thing at a time. With hyperthreading, things get a little different, but for the most part that still remains true, which is why my HT machine at work almost never goes above 50%, and even when it's at 100%, it's not processing twice as much at once.

You'll probably just have to do some testing on common architectures you plan to deploy on to determine how many threads you want to run on each CPU. Just using 1 thread may be too slow if you're waiting for a lot of I/O. Running a lot of threads will slow things down as the processor will have to switch threads more often, which can be quite costly. I'm not sure if there is any hard-coded limit to how many threads you can run, but I gaurantee that your app would probably come to a crawl from too much thread switching before you reached any kind of hard limit. Ultimately, you should just leave it as an option in the configuration file, so that you can easily tune your app to whatever processor you're running it on.

Jenks

I'd find a tad strange if I couldn't actually run 2 Java threads concurrently on a CPU that claims to support two threads per core. Is my understanding of CPU manufacturers' claims wrong? – Milks

Answer 5

A CPU does not normally pose a limit on the number of threads, and I don't think Java itself has a limit on the number of native (kernel) threads it will spawn.

There is a method availableProcessors() in the Runtime class. Is that what you're looking for?

Quadricycle

I read somewhere that there is a hard limit (that Java imposes) in the range of 32k. Realistically, your app will grind to a halt long before then. – Whoopee

Yes, probably. Java is not designed to uses threads in the thousands. – Quadricycle

I assume that when you say limits, you're referring to emulated concurrency (as opposed to true hardware concurrency). Is that not the case? – Milks

The usual limit to the number of threads is address space. On 32-bit systems you can get more threads by reducing the maximum stack size. Modern 64-bit operating systems should be able to cope with hundreds of thousands of threads. – Fidele

Answer 6

Basics: Application loaded into memory is a process. A process has at least 1 thread. If you want, you can create as many threads as you want in a process (theoretically). So number of threads depends upon you and the algorithms you use.

If you use thread pools, that means thread pool manages the number of threads because creating a thread consumes resources. Thread pools recycle threads. This means many logical threads can run inside one physical thread one after one.

You don't have to consider the number of threads, it's managed by the thread pool algorithms. Thread pools choose different algorithms for servers and desktop machines (OSes).

Edit1: You can use explicit threads if you think thread pool doesn't use the resources you have. You can manage the number of threads explicitly in that case.

Eu

Does Intel i7's claim of 2 logical threads per core (or 8 threads per core in Sun T2's case) not provide any further advantages for multi-threaded apps beyond what multi-core hardware provides? If that's true for all CPU manufacturers, then availableProcessors() is all I need. – Milks

I just find hard to believe that my app would be stuck with a mere 2-threads-per-core pool when running on a 8-threads-per-core machine. – Milks

You do have to consider the number of threads. Most thread pools (many application servers, built-in Java ExecutorService) require explicit configuration of the upper thread bound. – Autocade

You can set the upper thread bound. I work with .NET and you can set that value as well, but thread pool manages the number of threads in this environment. – Eu

Good answer but I was a little bit confused by "This means many logical threads can run inside one physical thread one after one." what do you consider to be logical threads? Shouldn't you name this as Runnable or Task to avoid the confusion? – Journey

Answer 7

This is a function of the VM, not the CPU. It has to do with the amount of heap consumed per thread. When you run out of space on the heap, you're done. As with other posters, I suspect your app becomes unusable before this point if you exceed the heap space because of thread count.

See this discussion.

Fulminant