How containerd compares to runc

Question

How do these two compare?

As far as I understand, runc is a runtime environment for containers. That means that this component provides the necessary environment to run containers. What is the role of containerd then?

If it does the rest (networking, volume management, etc) then what is the role of the Docker Engine? And what about containerd-shim? Basically, I'm trying to understand what each of these components do.

Noisy

Answer 1

I will give a high level overview to get you started:

• containerd is a container runtime which can manage a complete container lifecycle - from image transfer/storage to container execution, supervision and networking.
• container-shim handle headless containers, meaning once runc initializes the containers, it exits handing the containers over to the container-shim which acts as some middleman.
• runc is lightweight universal run time container, which abides by the OCI specification. runc is used by containerd for spawning and running containers according to OCI spec. It is also the repackaging of libcontainer.
• grpc used for communication between containerd and docker-engine.
• OCI maintains the OCI specification for runtime and images. The current docker versions support OCI image and runtime specs.

More Links:

• Open Container Specification
• A nice dockercon 2016 presentation

Sashenka

What is the role of Docker Engine then? – Noisy

@DimaKnivets Docker Engine is the higher level component which manages a bunch of things such as orchestration, managing volumes, networking, scaling and so on. – Sashenka

Docker doesn't support the OCI image spec. There are some pull requests open for it, but I have my doubts they'll be merged any time soon. – Sort

I'm under the impression that runc is a command-line utility that uses libcontainer. Would it be more accurate to say that containerd uses libcontainer? – Ama

Docker Engine can now be replaced with cri-tools - github.com/kubernetes-incubator/cri-tools – Dicky

Are docker and runC interchangeable for running containers ? – Contrary

just wanted to understand in bit more depth - runc is used by containerd . From packaging perspective - does this mean that containerd included runc as part of it's package or code. – Stephanystephen

I'm not sure if containerd still uses runc. If this were the case runc list would show some running containers. My server runs a few containers, but none of them shows up with runc list. As runc is a command line utility, containerd would break every time the runc command line interface changes. – Dunt

Answer 2

Docker engine is the whole thing, it was a monolith that enabled users to run containers. Then it was broken down into individual components. It was broken down into: - docker engine - containerd - runc

runC is the lowest level component that implements the OCI interface. It interacts with the kernel and does the "runs" the container

containerd does things like take care of setting up the networking, image transfer/storage etc - It takes care of the complete container runtime (which means, it manages and makes life easy for runC, which is the actual container runtime). Unlike the Docker daemon it has a reduced feature set; not supporting image download, for example.

Docker engine just does some high level things itself like accepting user commands, downloading the images from the docker registry etc. It offloads a lot of it to containerd.

"the Docker daemon prepares the image as an Open Container Image (OCI) bundle and makes an API call to containerd to start the OCI bundle. containerd then starts the container using runC."

Note, the runtimes have to be OCI compliant, (like runC is), that is, they have to expose a fixed API to managers like containerd so that they(containerd) can make life easy for them(runC) (and ask them to stop/start containers)

rkt is another container runtime, which does not support OCI yet, but supports the appc specification. But it is a full fledged solution, it manages and makes it's own life easy, so it needs no containerd like daddy.

So, that's that. Now let's add another component (and another interface) to the mix - Kubernetes

Kubernetes can run anything that satisfies the CRI - container runtime interface.

You can run rkt with k8s, as rkt satisfies CRI - container runtime interface. Kubernetes doesn't ask for anything else, it just needs CRI, it doesn't give a FF about how you run your containers, OCI or not.

containerd does not support CRI, but cri-containerd which is a shim around containerd does. So, if you want to run containerd with Kubernetes, you have to use cri-containerd (this also is the default runtime for Kubernetes). cri-containerd recently got renamed to CRI Plugin.

If you want to get the docker engine in the mix as well, you can do it. Use dockershim, it will add the CRI shim to the docker engine.

Now, like containerd can manage and make life easy for runC (the container runtime), it can manage and make life easy for other container runtimes as well - in fact, for every container runtime that supports OCI - like Kata container runtime (known as ~kata-runtime~ - https://github.com/kata-containers/runtime.) - which runs kata containers, Clear Container runtime (by Intel).

Now we know that rkt satisfies the CRI, cri-containerd (aka CRI Plugin) does it too.

Note what containerd is doing here. It is not a runtime, it is a manager for runC which is the container runtime. It just manages the image download, storage etc. Heck, it doesn't even satisfy CRI.

That's why we have CRI-O. It is just like containerd, but it implements CRI. CRI-O needs a container runtime to run images. It will manage and make life easy for that runtime, but it needs a runtime. It will take any runtime that is OCI compliant. So, naturally, ~kata-runtime~ is CRI-O compliant, runC is CRI-O compliant.

Use with Kubernetes is simple, point Kubernetes to CRI-O as the container runtime. (yes yes, CRI-O, but CRI-O and the actual container runtime IS. And Kubernetes is referring to that happy couple when it says container runtime).

Like containerd has docker to make it REALLY usable, and to manage and make life easy for containerd, CRI-O needs someone to take care of image management - it has buildah, umochi etc.

crun is another runtime which is OCI compliant and written in C. It is by RedHat.

We already discussed, kata-runtime is another runtime which is OCI compliant. So, we can use kata-runtime with CRI-O like we discussed.

Note, here, the kubelet is talking to CRI-O via the CRI. CRI-O is talking to cc-runtime (which is another runtime for Intel's clear containers, yes, OCI compliant), but it could be kata-runtime as well.

Don't forget containerd, it can manage and make life easy for all OCI complaint runtimes too - runC sure, but also kata-runtime, cc-runtime

Here, note just the runtime is moved from runC to kata-runtime. To do this, in the containerd config, just change runtime to "kata"

Needless to say, it can run on Kubernetes either by CRI-O, or by cri-containerd (aka CRI Plugin).

This is really cool :top:

Kubernetes, represented here by it's Ambassador, Mr. Kubelet runs anything that satisfies the CRI. Now, we have several candidates that can. - Cri-containerd makes containerd do it. - CRI-O does it natively. - Dockershim makes the docker engine do it.

Now, all the 3 guys above, can manage and make life easy for all OCI compliant runtimes - runC, kata-runtime, cc-runtimes.

We also have frakti, which satisfies CRI, like rkt, but doesn't satisfy OCI, and comes bundled with it's own container runtime.

Here we have CRI-O in action managing and making life easy for OCI compliant kata-runtime and runC both

We have some more runtimes as well:

• railcar - OCI compliant, written in rust
• Pouch - Alibaba's modified runC
• nvidia runtime - nvidia's fork of runC

ref: https://github.com/darshanime/notes/blob/master/kubernetes.org#notes

Overstreet

This is really helpful content. Wanted to point something out: You say "Note what containerd is doing here. It is not a runtime,", but containerd's own readme says "containerd is an industry-standard container runtime". Source: github.com/containerd/containerd – Palmitate

The content is really helpful, but where did you got those blurry images? For example, one of the image is from this blog and it's better to use the image it provides. – Cecillececily

Answer 3

runc is one of the component of containerd and handles kernel level interaction for running containers. In earlier versions, containerd was essentially a high level abstraction around runc but now it's way more than that. From container.io:

runc is a component of containerd, the executor for containers. containerd has a wider scope than just executing containers: downloading container images, managing storage and network interfaces, calling runc with the right parameters to run containers.

This image from same source nicely describes this.

Docker Engine is the end user product that uses containerd as a main component and implements other functionalities that doesn't fall under containerd's scope.

Note that Docker extracted out containerd as a separate component, so it can be used and developed by others products too.

[Edit] I wrote more about this teminology here

Theater

Answer 4

Your assumptions regrading the seperation between responsibilities of the ContainerD and Runc were correct.

Great answers above, I'll just add an important mind map that I think is critical to remember.

Will also add the context of K8S and other tools in the map because I think it is more relevant today (2024) to many people.

Quick mind map

We can see by a quick look on the map below that ContainerD and RunC (both in bold blue text) resides on different levels.

Tools like ContainerD, CRI-O and Podman are implementing the CRI API Spec and has a larger range of domains to handle then the container management part which is the focus of OCI related tools like runc and crun.

---

(*) Please notice:
The current landscape and responsibilities of the Networking, Storage, Images and Runtime domains depends on the time of writing.

At the moment, The OCI site currently contains 3 main specs:
(1) The distribution spec.
(2) The image spec.
(3) The runtime spec.

So the Networking and Storage domains resides on different levels then the OCI at the time of writing:

The Networking domain:
Is described by the CNI spec in the case of K8S, or CNM in the case of Docker.

The Storage domain:
Is described as part of the Container Storage Interface (CSI) spec.

A storage OCI-related library is given here.

Datum