Kind kubernetes cluster failed to pull docker images

Question

I tried to use KinD as an alternative of Minikube to bootstrap a K8S cluster in my local machine.

The cluster is created successfully.

But when I tried to create some pods/deployments from images, it failed.

$ kubectl run nginx --image=nginx
$ kubectl run hello --image=hello-world

After some minutes, use get pods to get a failed status.

$ kubectl get pods
NAME    READY   STATUS             RESTARTS   AGE
hello   0/1     ImagePullBackOff   0          11m
nginx   0/1     ImagePullBackOff   0          22m

I am afraid this is another Global Firewall problem in China.

kubectl describe pods/nginx
Name:         nginx
Namespace:    default
Priority:     0
Node:         dev-control-plane/172.19.0.2
Start Time:   Sun, 30 Aug 2020 19:46:06 +0800
Labels:       run=nginx
Annotations:  <none>
Status:       Pending
IP:           10.244.0.5
IPs:
  IP:  10.244.0.5
Containers:
  nginx:
    Container ID:
    Image:          nginx
    Image ID:
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ErrImagePull
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-mgq96 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-mgq96:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-mgq96
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                From                        Message
  ----     ------     ----               ----                        -------
  Normal   Scheduled  56m                default-scheduler           Successfully assigned default/nginx to dev-control-plane
  Normal   BackOff    40m                kubelet, dev-control-plane  Back-off pulling image "nginx"
  Warning  Failed     40m                kubelet, dev-control-plane  Error: ImagePullBackOff
  Warning  Failed     13m (x3 over 40m)  kubelet, dev-control-plane  Failed to pull image "nginx": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/nginx:latest": failed to copy: unexpected EOF
  Warning  Failed     13m (x3 over 40m)  kubelet, dev-control-plane  Error: ErrImagePull
  Normal   Pulling    13m (x4 over 56m)  kubelet, dev-control-plane  Pulling image "nginx"

When I entered to the kindest/node container, but there is no docker in it. Not sure how KIND works, originally I understand it deploys a K8S cluster into a Docker container.

$ docker ps
CONTAINER ID        IMAGE                  COMMAND                  CREATED             STATUS              PORTS                       NAMES
a644f8b61314        kindest/node:v1.19.0   "/usr/local/bin/entr…"   About an hour ago   Up About an hour    127.0.0.1:52301->6443/tcp   dev-control-plane

$ docker exec -it  a644f8b61314  /bin/bash
root@dev-control-plane:/# docker -v
bash: docker: command not found

After reading the Kind docs, I can not find an option to set a repository mirror there like that in Minikube.

BTW, I am using the latest Docker Desktop beta on a Windows 10.

Chace

What is the output of kubectl describe pod nginx? – Weintrob

@DavidLosert Appended it in the post – Chace

Answer 1

First pull the image in your local system using docker pull nginx and then use below command to load that image to the kind cluster

kind load docker-image nginx --name kind-cluster-name 

Kind uses containerd instead of docker as runtime, that's why docker is not installed on the nodes.

Alternatively you can use crictl tool to pull and check images inside the kind node.

crictl pull nginx
crictl images

Sedda

This works, if possible to use the Docker registry mirror for containerd. Not sure why containerd pull images from Docker hub failed – Chace

Answer 2

I've run into same issue because I've exported http_proxy and https_proxy before creating cluster to a local proxy (127.0.0.1), which is unrechable in the cluster. After unset http(s)_proxy and recreate cluster, everything runs fine.

Vincent

Answer 3

As the other answers did not help in my case: there is the need to set the http(s)_proxy environment variables before you run kind create cluster. Only setting the environment for docker resp. containerd does not help here:

export http_proxy=http://...
export https_proxy=http://...

kind create cluster --config ...

Mathematician

Answer 4

I had to add

imagePullPolicy: never

to get it working. For some reason it was pulling from docker registry by default.

Recuperate