Is having multiple cores in a CPU for running multiple threads/processes at once, or for instruction-level parallelism?

Question

I was just trying to get a clearer understanding of what exactly multiple cores are being used for, and what the difference is between multiple cores and multiple CPUs.

I was trying to understand if having multiple cores is just to enable instruction-level parallelism within a given CPU, or if the multiple cores within a CPU all have their own struct rq that lets them independently call __schedule() and schedule processes/threads at will, and that instruction-level parallelism is then handled by additional modules within each core.

Any thoughts appreciated.

Verwoerd

Answer 1

ILP is purely within each physical core separately.
cross-site duplicate: How does a single thread run on multiple cores?
(It doesn't - each core has multiple execution units and a wide front-end. Read my linked answer for details that I'm not going to duplicate. See also Modern Microprocessors A 90-Minute Guide!)

Also, real CPU cores can pretend to be multiple "logical cores" (i.e. each having register context and can call __schedule() independently). Generically, this is SMT; the mostly widely-known brand-name implementation of that concept is Intel's HyperThreading. Letting multiple software threads share a CPU truly simultaneously (not via software context-switching) gives the core two instruction-streams to find parallelism between as well as within, generally increasing overall throughput (at the cost of single-thread performance to some degree, depending on how busy a single thread of your workload could keep a core).

---

In some contexts, "CPU" is synonym for a single core. e.g. perf stat output saying "7.5 CPUs utilized".

But more often, a CPU refers to the whole physical package, e.g. my CPU is a quad-core, an i7-6700k. Server motherboards are often dual-socket, allowing you to plug in two separate multi-core CPUs.

Perhaps that's what created some terminology confusion?

Hygienics

Yeah, that was a source of my confusion. Another thing that helped clear up my CPU vs core understanding was reading that a CPU is the 'umbrella' that contains a cache hierarchy, and within the CPU, multiple cores can share L2/L3 cache, but have their own L1 cache. – Verwoerd

@user49404: Yup, right, there's significant infrastructure outside the cores, good point. (BTW, usually if there's an L3 cache, L2 caches are per-core private. How are the modern Intel CPU L3 caches organized? / Which cache mapping technique is used in intel core i7 processor?) – Hygienics

The 90-minute guide? Wow! Great material!!! (I'm not even halfway through) Not for the faint of heart, though. – Ehrman

Answer 2

what the difference is between multiple cores and multiple CPUs

Mulitple cores are on one piece of silicon and mulitple CPUs are on different pieces of silicon. That's pretty much the only difference between multiple cores and multiple CPUs.

Generally the term "core" is more precise these days -- it means the same thing as the original definition of CPU: the Central Processing Unit of the Von Neumann architecture model. Over time, however, the term CPU has been (ab)used by marketing to mean all the things on a single piece of silicon or in a single physical package, even when that is more that one CPU/core.

When talking about software, things become even more imprecise, as you may have mulitple "logical" or "virtual" CPUs running on a single physical CPU. The term core here generally still remains more precise, referring to the physical device, though when dealing with virtual machines you may see references to "virtual cores".

Carlos

I think you probably mean "separate packages", i.e. that plug into separate sockets. That is usually the same thing as "different piece of silicon" so it's a useful simplification, but note that some CPUs pushing the upper limits of size at the time they were new have multiple silicon chips inside a single package. (e.g. Core 2 Quad was basically two Core 2 Duo dies bolted together). – Hygienics

Also, modern "chiplet" designs have a separate die for each group of cores on a substrate (e.g. AMD Zen2 has each CCD on a separate chiplet) anandtech.com/show/13560/…. Intel also has chiplet plans, although I don't think Ice Lake actually did end up using chiplets for its Sunny Cove cores. anandtech.com/show/16021/… – Hygienics

My main point it that the difference between "cores" and "CPUs" is largely a detail of manufacturing and has essentially no impact on the software achitecture. The chiplet case illustrates this even more, as packages containing different arrangements of pieces of silicon might even be sold as the same device -- the details being completely hidden from the OS, let alone the user. – Carlos

Yeah, agreed. But I think for answering this question, it was also important to point out that "CPU" does sometimes get used to describe a core, instead of a whole CPU package. – Hygienics