Single- vs. multi-threaded programming on a single core processor

Question

Can someone please explain if there's really any advantage to writing a multi-threaded piece of code that runs on a single processor with a single core? E.g., a method that processes document pages such that the pages are mutually exclusive w/r/t the aforementioned piece of code.

At first glance, it doesn't seem like there'd be an advantage because true multi-threading is not possible. I.e., the OS would have to context switch the threads anyway. I'm wondering if just coding something in a single-threaded manner could actually be more efficient.

Clearly, there are plenty of cases where writing multi-threaded code makes sense, but again, my question gets to whether there's really an advantage of doing so when the application is running on a single-core processor.

EDIT: note, that I did not say "application" but rather "piece of code" - look at my example above. Clearly there are benefits to having a multi-threaded application.

Raviv

You can have threads that wait for files to load, for network data to come etc. Even if only one can use processor at a time. – Shermy

As @Shermy pointed out, IO-bound applications may take advantage of multi-threaded even on a single core, although you'll never get more than 2X speedup for them (if IO becomes too much of a bottleneck CPU speedup wouldn't matter by Amidhal's law). A more common reason is to design multi-threaded applications, even if you have only 1 core at this time, is for ease of porting for multiple-cores when they become available. It's a lot better to design the application right once, without hardcoding current hardware limitations, then to re-write it again when more hardware resources become available. – Lobbyism

@Lobbyism I'd agree in the case where you're pretty certain that you will be porting to a multi-core system eventually, but making your code more complicated now because of something you might want to do with it in the future often isn't worth it. – Insurer

So, this was more a philosophical question rather than one based on reality. I'm clear there are advantages to having multi-threaded applications. E.g., zch pointed out one could have [background] thread monitoring files, etc... I also agree with Michael in that premature optimization is typically not a good strategy. – Raviv

@BenBarden Seeing as it's practically impossible to build a single core PC these days, and it seems that soon all phones and tablets will have multiple cores as well, I believe you'll only be coding for single core devices in very specialized applications. I can't think of anything these days that is built to be single core and stay like that in the foreseeable future. – Elise

@Renan "I can't think of anything these days that is built to be single core", errrmm, virtual machines? Most cloud service VM plans start out with single core cpu. – Eduction

@Eduction yeah but see my last sentence. I believe that will change in a couple years time. – Elise

Answer 1

There are still advantages to be gained, but they're a bit situational.

• In many cases, giving the thing multiple threads will allow it to claim more system resources from other processes. This is finicky to balance, and each thread you introduce adds a bit of overhead, but it can be a reason.

• If you are dealing with multiple potentially blocking resources - like file IO or GUI interaction or whatnot, then multithreading can be vital.

Insurer

I don't think they are less meaningful, specially given the second item in the bulleted list. Still, nice anwer. +1. – Elise

Answer 2

Yes, multi-threading is useful in a single core. If one thread in an application gets blocked waiting for something (say data from the network card or waiting for the disk to write data), the CPU can switch to another thread to keep working.

BeOS was written with pervasive multithreading in mind, even in a time of single core processors. The result was a very responsive OS, though a rather difficult OS to program for.

Langlauf

Great post! Just curious, can a multi-threading application take advantage of other cores in the os? (JXcore's fork of NodeJS for example) – Jobie

Answer 3

On a single core processor, an application that uses asynchronous (non-blocking) I/O will be slightly more efficient than one that uses multiple blocking threads, because it avoids the overhead of context switching between threads.

Also, asynchronous I/O scales better than blocking I/O in threads because the overhead per extra I/O operation is minimal compared to the overhead of creating a new thread.

Having said that, you shouldn't generally use single-threaded asynchronous I/O in new applications because almost all new processors are multicore. Instead you should still use asynchronous I/O, but split the work amongst a set of worker threads using something like a thread pool. Your system documentation will tell you the ideal number of worker threads; usually it is equal to the number of processing cores available.

Edit: On the Windows platform at least, the async/await pattern in .NET is the modern way to perform asynchronous I/O. It makes this pattern as trivially easy to write as the old blocking I/O pattern. There is almost no excuse for writing blocking I/O now.

Rambort