Will multi-threading increase the speed of the calculation on a single-core processor

Question

On a single-core processor, will multi-threading increase the speed of the calculation? As we all know, multi-threading is used for increasing the user responsiveness and achieved by separating UI threads and calculation threads. But let’s talk about only console applications. Will multi-threading increase the speed of the calculation? Do we get a calculations’ result faster when we calculate through multi-threading.

What about on multi cores, will multi-threading increase the speed or not?

Edit:
I have been asked a question. At any given time, only one thread is allowed to run on a single core. If so, why people use multi-threading in a console application.

Narcolepsy

Multithreading has been around a long longer than multiple cores. – Hives

Answer 1

In general terms, no it won't speed up anything.

Presumably the same work overall is being done, but now there is the overhead of additional threads and context switches.

On a single processor with HyperThreading (two virtual processors) then the answer becomes "maybe".

Finally, even though there is only one CPU perhaps some of the threads can be pushed to the GPU or other hardware? This is kinda getting away from the "single processor" scenario but could technically be way of achieving a speed increase from multithreading on a single core PC.

Edit: your question now mentions multithreaded apps on a multicore machine. Again, in very general terms, this will provide an overall speed increase to your calculation. However, the increase (or lack thereof) will depend on how parallelizable the algorithm is, the contention for memory and cache, and the skill of the programmer when it comes to writing parallel code without locking or starvation issues.

Flocculant

Note that this holds for CPU intensive tasks, for tasks doing blocking operations(IO bound/talks to a database/etc.) multithreading can speed things up, though doing async or non-blocking operations can speed things further up, but not all apis have non-blocking variants, and those who do can be harder to program. – Beekeeping

This isn't completely true, after all doing parallel compiles is almost always faster no matter what the hardware is – Escalade

Answer 2

Few threads on 1 CPU:

• may increase performance in case you continue with another thread instead of waiting for I/O bound operation
• may decrease performance if let say there are too many threads and work is wasted on context switching

Few threads on N CPUs:

• may increase performance if you are able to cut job in independent chunks and process them in independent manner
• may decrease performance if you rely heavily on communication between threads and bus becomes a bottleneck.

So actually it's very task specific - you can parallel one things very easy while it's almost impossible for others. Perhaps it's a bit advanced reading for new person but there are 2 great resources on this topic in C# world:

• Joe Duffy's web log
• PFX team blog - they have a very good set of articles for parallel programming in .NET world including patterns and practices.

Catercornered

Answer 3

What is your calculation doing? You won't be able to speed it up by using multithreading if it a processor bound, but if for some reason your calculation writes to disk or waits for some other sort of IO you may be able to improve performance using threading. However, when you say "calculation" I assume you mean some sort of processor intensive algorithm, so adding threads is unlikely to help, and could even slow you down as the context switch between threads adds extra work.

Intervene

Answer 4

If the task is compute bound, threading will not make it faster unless the calculation can be split in multiple independent parts. Even so you will only be able to achieve any performance gains if you have multiple cores available. From the background in your question it will just add overhead.

However, you may still want to run any complex and long running calculations on a separate thread in order to keep the application responsive.

Sorrento

+1 For responsiveness, an application might appear faster if the user can interact with the application and do other stuff while a large calculation is in progress. – Africanize

Answer 5

No, no and no.

Unless you write parallelizing code to take advantage of multicores, it will always be slower if you have no other blocking functions.

Mode

As @coxymla points out there might be a CPU with HyperThreading. This might in fact increase the speed of execution, even if the CPU has only a single core. – Priam

Thanks for the reply, Lets forget Multicores for time being. I have been asked questions like, If calculation didn't speed, whey we have multithreading on single core. I have explained them that, multithreading is for responsiveness, communication related, DB related work. But many people believe that multi-threading increses the speed of calculation. like to know how. Thanks for your time. – Narcolepsy

@Harsha, @leppie: HyperThreading works for single cores. It uses CPU "idle" times for pseudo-parallel execution. From en.wikipedia.org/wiki/Hyperthreading: When execution resources would not be used by the current task in a processor without hyper-threading, and especially when the processor is stalled, a hyper-threading equipped processor can use those execution resources to execute another scheduled task. (The processor may stall due to a cache miss, branch misprediction, or data dependency.) – Priam

@leppie, it really depends on the kind of algorithm they are going to use there. Some algorithms can be split into multiple independent calculation "branches" in which case the whole process can be greatly improved in a multi-core scenario. I wouldn't just cut it down to a flat "no", unless we know the specifics. – Grassland

@Artiom Chilaru: I do understand what you are saying, but I thought I covered those aspects in my answer. – Mode

Well, I do believe that when he states "Do we get culculation result faster when we calculate through multi-threading.", it implies that the calculation algorithm will be written to take advantage of multi-threading. Which is why your direct "no" threw me off like this :) – Grassland

Answer 6

Exactly like the user input example, one thread might be waiting for a disk operation to complete, and other threads can take that CPU time.

Superabundant

Answer 7

As described in the other answers, multi-threading on a single core won't give you any extra performance (hyperthreading notwithstanding). However, if your machine sports an Nvidia GPU you should be able to use the CUDA to push calculations to the GPU. See http://www.hoopoe-cloud.com/Solutions/CUDA.NET/Default.aspx and C#: Perform Operations on GPU, not CPU (Calculate Pi).

Intrusive

Answer 8

Above mention most.

Running multiple threads on one processor can increase performance, if you can manage to get more work done at the same time, instead of let the processor wait between different operations. However, it could also be a severe loss of performance due to for example synchronization or that the processor is overloaded and cant step up to the requirements.

As for multiple cores, threading can improve the performance significantly. However, much depends on finding the hotspots and not overdo it. Using threads everywhere and the need of synchronization can even lower the performance. Optimizing using threads with multiple cores takes a lot of pre-studies and planning to get a good result. You need for example to think about how many threads to be use in different situations. You do not want the threads to sit and wait for information used by another thread.

http://www.intel.com/intelpress/samples/mcp_samplech01.pdf
https://computing.llnl.gov/tutorials/parallel_comp/
https://computing.llnl.gov/tutorials/pthreads/
http://en.wikipedia.org/wiki/Superscalar
http://en.wikipedia.org/wiki/Simultaneous_multithreading

Beautify

Hello Jakob, Thanks for the posting the answer and PDF File. I have couple of requests/questions: 1. Do you any project files or source code, through which one can learn multi threading on single processor for increasing the performance. If you have any material or link for this question. 2.Do you have any Threading Guidelines, any link please send me. 3. Do you have any Threading debug tools. Thanks for the your time and posting answer. – Narcolepsy

Hello! I'm sorry but I do not have any source code. Much of what decides how your program work depends on how the architecture of your CPU is. I don't have any source code or debug tool, sorry for that, but I've edited my post with some more links! – Beautify

Answer 9

I have been doing some intensive C++ mathematical simulation runs using 24 core servers. If I run 24 separate simulations in parallel on the 24 cores of a single server, then I get a runtime for each of my simulations of say X seconds.

The bizarre thing I have noticed is that, when running only 12 simulations, using 12 of the 24 cores, with the other 12 cores idle, then each of the simulations runs at a runtime of Y seconds, where Y is much greater than X! When viewing the task manager graph of the processor usage, it is obvious that a process does not stick to only one core, but alternates between a number of cores. That is to say, the switching between cores to use all the cores slows down the calculation process.

The way I maintained the runtime when running only 12 simulations, is to run another 12 "junk" simulations on the side, using the remaining 12 cores!

Conclusion: When using multi-cores, use them all at 100%, for lower utilisation, the runtime increases!

Ovotestis

That overhead seems to come from cold caches after the thread has been moved to another core. Ideally, the operating system would know that it doesn't make sense to move those threads. – Discontented

Answer 10

For single core CPU, Actually the performance depends on the job you are referring. In your case, for calculation done by CPU, in that case OverClocking would help if your parentBoard supports it. Otherwise there is no way for CPU to do calculations that are faster than the speed of CPU.

For the sake of Multicore CPU As the above answers say, if properly designed the performance may increase, if all cores are fully used.

In single core CPU, if the threads are implemented in User Level then multithreading wont matter if there are blocking system calls in the thread, like an I/O operation. Because kernel won't know about the userlevel threads.

So if the process does I/O then you can implement the threads in Kernel space and then you can implement different threads for different job. (The answer here is on theory based.)

Bully

Answer 11

Even a CPU bound task might run faster multi-threaded if properly designed to take advantage of cache memory and pipelineing done by the processor. Modern processors spend a lot of time twiddling their thumbs, even when nominally fully "busy".

Imagine a process that used a small chunk of memory very intensively. Processing the same chunk of memory 1000 times would be much faster than processing 1000 chunks of similar memory.

You could certainly design a multi threaded program that would be faster than a single thread.

Indulge

Answer 12

Treads don't increase performance. Threads sacrifice performance in favor of keeping parts of the code responsive.

The only exception is if you are doing a computation that is so parallelizeable that you can run different threads on different cores (which is the exception, not the rule).

Harker