What vCPUs in Fargate really mean?

Question

I was trying to get answers on my question here and here, but I understood that I need to know specifically Fargate implementation of vCPUs. So my question is:

1. If I allocate 4 vCPUs to my task does that mean that my single-threaded app running on a container in this task will be able to fully use all this vCPUs as they are essentially only a portion of time of the processor's core that I can use?
2. Let's say, I assigned 4vCPUs to my task, but on a technical level I assigned 4vCPUs to a physical core that can freely process one thread (or even more with hyperthreading). Is my logic correct for the Fargate case?

p.s. It's a node.js app that runs session with multiple players interacting with each other so I do want to provide a single node.js process with a maximum capacity.

Bawdy

Answer 1

Fargate uses ECS (Elastic Container Service) in the background to orchestrate Fargate containers. ECS in turn relies on the compute resources provided by EC2 to host containers. According to AWS Fargate FAQ's:

Amazon Elastic Container Service (ECS) is a highly scalable, high performance container management service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 instances ...

ECS uses containers provisioned by Fargate to automatically scale, load balance, and manage scheduling of your containers

This means that a vCPU is essentially the same as an EC2 instance vCPU. From the docs:

Amazon EC2 instances support Intel Hyper-Threading Technology, which enables multiple threads to run concurrently on a single Intel Xeon CPU core. Each vCPU is a hyperthread of an Intel Xeon CPU core, except for T2 instances.

So to answer your questions:

1. If you allocate 4 vCPUs to a single threaded application - it will only ever use one vCPU, since a vCPU is simply a hyperthread of a single core.

2. When you select 4 vCPUs you are essentially assigning 4 hyperthreads to a single physical core. So your single threaded application will still only use a single core.

If you want more fine grained control of CPU resources - such as allocating multiple cores (which can be used by a single threaded app) - you will probably have to use the EC2 Launch Type (and manage your own servers) rather than use Fargate.

---

Edit 2021: It has been pointed out in the comments that most EC2 instances in fact have 2 hyperthreads per CPU core. Some specialised instances such as the c6g and m6g have 1 thread per core, but the majority of EC2 instances have 2 threads/core. It is therefore likely that the instances used by ECS/Fargate also have 2 threads per core. For more details see doco

Godfearing

Thank you so much for your answer! So if I disable Hyper-threading for an EC2 instance that means that I will have 1vCPU = 1 CPU core, right? And then if I assign X number of vCPUs to my task then it will be using X CPU cores, right? – Bawdy

And one more pls, I know that single-threaded app can use (somehow) multiple cores, but do you know how well it will work specifically with node.js? – Bawdy

Yep that's right turning off hyperthreading should work. I'm not too sure about nodejs specifically sorry. – Godfearing

I'm not sure this is correct - It seems to me that each CPU core is split into 2 hyperthreads, not 4. lscpu tends to show this in EC2. That means that Fargate with 4 vCPUs should give you access to 2 full CPU cores. – Mychal

If I have a bash script which wraps 4 commands that are run in bg processes (via &), those should all be run in parallel and utilizing the 4vCPU's, correct? – Selfabsorbed

I think @Mychal is right to challenge this answer, though I don't think the maths works quite like that. I believe the actual result may be based on resource contention on the VM or even the bare metal. You should get at least 2 full CPU core equivalent. But I suspect if your task isn't contending with much else then your 4 vcpus won't be crammed onto only 2 cores but expand up to a full 4 cores. – Fertility

Answer 2

You can inspect what physical CPU your ECS runs on, by inspecting the /proc/cpuinfo for model name field. You can just cat this file in your ENTRYPOINT / CMD script or use ECS Exec to open a terminal session with your container.

I've actually done this recently, because we've been observing some weird performance drops on some of our ECS Services. Out of 84 ECS Tasks we ran, this was the distribution:

Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz      (10 tasks)
Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz  (22 tasks)
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz  (10 tasks)
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz (25 tasks)
Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz (17 tasks)

Interesting that it's 2022 and AWS is still running CPUs from 2016 (the E5-2686 v4). All these tasks are fully-paid On-Demand ECS Fargate. When running some tasks on SPOT, I even got an E5-2666 v3 which is 2015, I think.

While assigning random CPUs for our ECS Tasks was somewhat expected, the differences in these are so significant that I observed one of my services to report 25% or 45% CPU Utilization in idle, depending on which CPU it hits on the "ECS Instance Type Lottery".

Kathrynekathy

Does this mean with fargate you don't really know what the quality of the vCPU will be? Like you said it's a lottery. In that case, it seems much better to manage your own EC2 instances so you can be sure that vCPUs translate to. – Vlf

@CMCDragonkai, It all depends on your case. Managing your own EC2 Fleet for ECS may give you better stability, but comes with additional maintenance. You now need to worry about patching the EC2s, keeping them up to date, and solving (rare, but still) compatibility issues when doing so. If your system has to be HIPAA or PCI/DSS compliant, that usually means a lot of additional work. Somewhat random performance may be solved with autoscaling or oversizing images, which comes with additional cost, but may still be cheaper than the maintenance cost of extra EC2 infrastructure. – Kathrynekathy

I found the same issues with AWS ECS Fargate using NodeJS as we also spawn worker threads and found that the CPU is always 45% and RAM is stable around 30% using PM2 as a process manager. The servers (up to 6) have "blips" of slowness. 95% of requests complete in under 150ms then some hemorrhage and stall waiting 3-4 second; at worst, up to 7 seconds. Here are the [docs.aws.amazon.com/AWSEC2/latest/UserGuide/… docs). – Ptyalin

Answer 3

The accepted answer states 'a vCPU is simply a hyperthread of a single core'.
This isn't always true.

It's more accurate to state that a vCPU is a virtualized abstraction of a physical CPU core or thread, depending on the instance type and CPU type.

---

Fargate vCPUs are equal to EC2 vCPUs, as Fargate uses ECS & ECS uses EC2 instances.

Let's try to define a vCPU.

The official documentation is conflicting:

Each vCPU on Graviton-based Amazon EC2 instances is a core of AWS Graviton processor.

Each vCPU on non-Graviton-based Amazon EC2 instances is a thread of x86-based processor, except for T2 instances and m3.medium.

Each vCPU on non-Graviton-based Amazon EC2 instances is a thread of x86-based processor.

Each vCPU is a thread of either an Intel Xeon core or an AMD EPYC core, except for T2 and m3.medium.

Each vCPU is a thread of a CPU core, except for T2 instances and 64-bit ARM platforms such as instances powered by AWS Graviton2 processors and Apple Silicon Mac instances.

And not just in one one place...

Each thread is represented as a virtual CPU (vCPU) on the instance.

But multiple:

Each vCPU is a thread of either an Intel Xeon core or an AMD EPYC core, except for T2 and T4g instances.

Is it a core? A thread? What exactly is a vCPU in the context of exceptions?

Why are only T2 and T4g instances exceptions? It seems logical that T3 and T3a should also be exceptions, as they are also burstable, use a credit system, and all have a baseline performance below 100% of 1 core (with the exception of t2.2xlarge). Since the vCPU for burstable instances represents the maximum CPU performance for the instance, not the steady CPU performance, it's understandable why AWS does not equate vCPU to a 'core' or a 'thread' in this case.

However, why are Apple Silicon Mac instances an exception? Both EC2 M1 and EC2 M2 instances have 8 cores, with a vCPU count of 8. The EC2 M2 Pro has 12 cores, with a vCPU count of 12. Is this because AWS differentiates between Apple's high-performance cores (P cores) and high-efficiency cores (E cores)? Why hasn't this been clarified?

Again, why is only m3.medium mentioned? What about m4 or m5? This is something I can't figure out.

Ultimately, the documentation implies that vCPU equals thread with very vague wording:

Each thread is represented as a virtual CPU (vCPU) on the instance.

I think it should be something along the lines of:

A vCPU is a virtualized abstraction of a physical CPU core or thread, depending on the instance type and CPU type.

It is the core count multiplied by the number of threads.

While a vCPU is not always strictly a thread, the maximum CPU thread count is equal to the vCPU count of an instance, with the vCPU mapping to either a hyper-thread of a physical core or to the physical core itself.

---

If I allocate 4 vCPUs to my task does that mean that my single-threaded app running on a container in this task will be able to fully use all this vCPUs as they are essentially only a portion of time of the processor's core that I can use?

No, a single-threaded app can only use 1 thread at a time.

Let's say, I assigned 4vCPUs to my task, but on a technical level I assigned 4vCPUs to a physical core that can freely process one thread (or even more with hyperthreading). Is my logic correct for the Fargate case?

Currently, it is impossible to achieve 4 vCPUs with 1 core, as that would mean 4 threads per core. No instance type or CPU configuration provided by AWS today enables this.

Even if it was, a single-threaded application can only use 1 thread.

Kuopio