How many threads can I run concurrently on Windows?

Question

A comment to another of my questions says that I can only run "so many" threads concurrently, a notion which I have seen elsewhere.

As a threading novice, how can I determine the maximum number of threads to use? Or is this a "how long is a piece of string" question? What does it depends on? Hardware config or what?

(VB in MS Visual Studio with .Net 3.5, if that matters)

---

Update: is anyone aware of any s/w tool which could suggest a number of threads (or tasks), or should I just code my own which keeps trying different numbers until throughput drops?

---

[Upperdate] Almost seven years later & we now have a software recommendations site, so I asked if there is a tool to help with this.

Homologue

+1 good question. They each make one SOAP call to transmit soem data and wait for it to return – Homologue

Except, of course, that the "return" is asynch, so they are not really waiting. Other threads can run as soon as the SOAP request (fcuntion call) is sent over HTTP – Homologue

Answer 1

It depends on hardware as you're (probably) not using a theoretical computer but a physical hardware one, so you have limited resources.

Read: Does Windows have a limit of 2000 threads per process?

Furthermore, even if you could run 5000+ threads, depending on your hardware, that could run much slower than a 10 thread equivalent program. I think you should take a look at thread pooling.

Samothrace

+1 Thanks. That gives me somethign to look at and start trying to understand. Are you aware of any s/w tool which could suggest a number of threads? – Homologue

I guess that using one thread per CPU core is a sensible choice, but it really depends on the problem you're trying to solve. – Samothrace

+1 With one per core it is going to be difficult to simulate hundreds of devices – Homologue

If you're using a network device then all data will be sent in a serial fashion anyway, so I guess there'll be no performance gain there. I think maybe there's no need to have lots of threads, you could use one to keep sending data and another to read the results and identify to which request the response belongs, instead of having lots of idle threads sending data once and waiting for a reply. – Samothrace

Answer 2

Typically, the number of threads the run truly concurrently is determined by the number of CPUs and CPU cores (including hyper threading) you have. That is to say that at any given time the number of threads running (in the operating system) is equal to the number of "cores".

How many threads you can run concurrently in your app depends on a large number of factors. The best (lay man's) number would be the number of cores on the machine but of course that's like pretending no one (no other application) else exists :).

Frankly, I'd say do a lot more study on multi-threading in .NET/Windows because one tends to do more "damage" than good when one doesn't have a really solid understanding. .NET has the concept of a thread pool and you need to know how that works in addition to Windows.

In .NET 3.5/4.0 you should be looking at Tasks (Task Parallel Library) as the library does a much better job of determining how many threads (if at all) to spawn. With the TPL the threadpool gets a major overhaul and it is a lot smarter about spawning threads and task stealing etc. But you typically work with Tasks and not threads.

This is a complex area and as a result, the .NET framework introduced Tasks so as to abstract programmers from threads and therefore allowing the runtime to be smart about this while the programmer just say what she wants and not so much about how to do it.

Lophobranch

+1 Yup, I fear that I might do more damage than good. I will also look at tasks, thanks – Homologue

It's useful, I find, to distinguish between the terms "concurrency" and "parallelism" (i.e. what you referred to as "truly concurrent"). – Bermudez

Answer 3

A pretty good rule of thumb when running intensive tasks is to run the same number as your physical core count.

Yes, you can run more tasks, but they will wait for resources (or threads in a thread pool) and your box, regardless of size can't quite allocate all of a cpu core resources 100% of the time to a thread due to background/other processes. So the more tasks you instantiate, the more threads you spawn, as they surpass actual possible concurrent threads (1 per core), the more resource management, queuing and swapping will occur.

A test we did where I work now using a viral pattern to launch additional tasks found that optimal was pretty close to the cpu count as a cap. Tasks launched at a one-to-one ratio with the physical core count ran at about 1 minute per task to complete. Set at double the cpu count, task time went from 1 minute average to about 5 minutes average time to complete. It gets geometrically slower the more tasks initiated past core count.

So for example, if you have 8 physical cores, 8 tasks (and using TPL, essentially 8 concurrent threads in active process) should be the fastest. There is your main thread or process which creates the other tasks, and other background processes, but if the box is pretty isolated for your resource exploitation pleasure, those will be fairly minimal.

The upside of programming your task cap based on core count as you chew tasks off a queue or list so when you deploy the application on different sized boxes, it adjusts itself automatically.

To determine this programmatically, we use

var CoreCount = System.Environment.ProcessorCount / 2;

Why divide by two, you ask? Because nearly all modern processors use logical cores or hyperthreading. You should find with your own testing that if you use the Logical count, your overall speed per task, and thus the whole process, will drop significantly. Physical cores is the key. We couldn't see a quick way to find physical vs logical but a quick survey of our boxes found this to be consistently true. YMMV, but this might get your pretty far pretty fast.

Bullhead

Answer 4

Each thread consumes more memory (kernel stack, thread environment block, thread-local, stack....). AFAIK there are no explicit limit in Windows, therefore the constrain will be memory (probably the stack for each thread).

In Linux threads are more like processes (with shared memory) and you're constrained by:

cat /proc/sys/kernel/threads-max

Rattlebrained

Answer 5

From my own experience, when using threads a good rule of thumb for increased performance for CPU bound processes is to use an equal number of threads as cores, except in the case of a hyper-threaded system in which case one should use twice as many cores. The other rule of thumb that can be concluded is for I/O bound processes. This rule is to quadruple the number threads per cores, except for the case of a hyper-threaded system, then one can quadruple the number of threads per core.

Overhear

Lolx - when I first posted, there was no such thing as a multi-core CPU:-) Thanks for the advice +1 – Homologue

Answer 6

It very much depends on the machine - CPU and memory are the main limiting factors (though OS limits may come into it).

In regards to .NET the thread pool configuration also comes into play.

Checkpoint

Answer 7

I was able to run 4 threads at once on my current old CPU (2005) Using EVGA's CPU burner before my CPU buzzer sounded off (Programmed inside BIOS menu) Meaning i hit over 90*c. Keep in mind we are talking about Data threads working all at once. a good example would be having multiple programs open at once. But overall it really depends how good your CPU is with multitasking. (in other words can handle many active threads) A safe way to test is is to download "ocscanner (By EVGA)" and "CPU Thermometer" use CPU burner inside OC Scanner While testing, make sure your temperature does not rise over 90*c (or any temperature you feel safe at) and look at your current number of threads you have running threw your CPU. start at 2 threads, wait 3-5 minutes while watching the CPU temperature, add another thread, repeat. (DO NOT PUSH YOUR LUCK!!!)(DO NOT ATTEMPT IF CPU THERMOMETER CAN NOT DETECT YOUR TEMPERATURE!!!)

Assai