ImagePullBackOff local repository with Minikube

Question

I'm trying to use minikube and kitematic for testing kubernetes on my local machine. However, kubernetes fail to pull image in my local repository (ImagePullBackOff).

I tried to solve it with this : Can not pull docker image from private repo when using Minikube

But I have no /etc/init.d/docker, I think it's because of kinematic ? (I am on OS X)

EDIT :

I installed https://github.com/docker/docker-registry, and

docker tag local-image-build localhost:5000/local-image-build

docker push localhost:5000/local-image-build

My kubernetes yaml contains :

spec:
  containers:
  - name: backend-nginx
    image: localhost:5000/local-image-build:latest
    imagePullPolicy: Always

But it's still not working... Logs :

Error syncing pod, skipping: failed to "StartContainer" 
for "backend-nginx" with ErrImagePull: "Error while pulling image: 
Get http://127.0.0.1:5000/v1/repositories/local-image-build/images: 
dial tcp 127.0.0.1:5000: getsockopt: connection refused

EDIT 2 :

I don't know if I'm on the good path, but I find this :

http://kubernetes.io/docs/user-guide/images/

But I don't know what is my DOCKER_USER...

kubectl create secret docker-registry myregistrykey --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL

EDIT 3

now I got on my pod :

Failed to pull image "local-image-build:latest": Error: image library/local-image-build not found
Error syncing pod, skipping: failed to "StartContainer" for "backend-nginx" with ErrImagePull: "Error: image library/local-image-build not found"

Help me I'm going crazy.

EDIT 4

Error syncing pod, skipping: failed to "StartContainer" for "backend-nginx" with ErrImagePull: "Error response from daemon: Get https://192.168.99.101:5000/v1/_ping: tls: oversized record received with length 20527"

I added :

EXTRA_ARGS='
    --label provider=virtualbox
    --insecure-registry=192.168.99.101:5000

to my docker config, but it's still don't work, the same message....

By the way, I changed my yaml :

 spec:
      containers:
      - name: backend-nginx
        image: 192.168.99.101:5000/local-image-build:latest
        imagePullPolicy: Always

And I run my registry like that :

docker run -d -p 5000:5000 --restart=always --name myregistry registry:2

Spermatium

Did you actually run the registry? The command to do it is docker run -d -p 5000:5000 --name registry registry:2 – Tomtit

What about using the docker registry in the minikube registry? – Bently

it is still a problem with minikube 1.33.1 minikube version: v1.33.1 commit: 5883c09216182566a63dff4c326a6fc9ed2982ff – Monkshood

Answer 1

I think I solved by doing

minikube start --vm-driver="virtualbox" --insecure-registry="$REG_IP":80

instead of just

minikube start

$REG_IP is :

REG_IP=docker-machine ip registry

Source

Spermatium

what is $REG_IP? – Bently

REG_IP=docker-machine ip registry – Spermatium

docker-machine is now superseded / deprecated docs.docker.com/machine/overview – To

Any solution for 2022? Since docker-machine is deprecated. – Pestilence

Answer 2

Use the minikube docker registry instead of your local docker

https://kubernetes.io/docs/tutorials/stateless-application/hello-minikube/#create-a-docker-container-image

Set docker to point to minikube

eval $(minikube docker-env)

Push to minikube docker

docker build -t hello-node:v1 .

Set your deployment to not pull IfNotPresent

K8S default is set to "Always" Change to "IfNotPresent"

imagePullPolicy: IfNotPresent

Related Issue

Midland

thanks, quick and easy(in my view, correct solution for the stated problem). btw, first step (eval) is not required if minikube is started with "--vm-driver=none". – Stutter

eval $(minikube docker-env) this didn't work for me instead eval $(minikube -p minikube docker-env) worked for me – Lucic

Answer 3

If you're using --vm-driver=none, you'll need to set imagePullPolicy to Never.

imagePullPolicy: Never: the image is assumed to exist locally. No attempt is made to pull the image.

Cyclostome

Answer 4

If you are using minikube in Windows.

I solved it by just building my image in minikube cluster as below because I just wanted to test it.

Go to your directory where you want to build an image.

Try below command to build your images in minikube on your local

minikube image build -t your_image_name:tag .

Then you can give reference to this images with tag in your deployment.yaml or pods.yaml

Oconner

Answer 5

I tested in CentOS7 and works for me.

First load image into minikube first by using: minikube image load test/kubia

Check images list from minikube: minikube image ls

Then do not pull from remote by config yaml file

apiVersion: v1               
kind: Pod                    
metadata:
  name: kubia-manual         
spec:
  containers:
  - image: test/kubia
    name: kubia
    # This will use local image from minikube image list
    imagePullPolicy: Never
    ports:
    - containerPort: 8080
      protocol: TCP

NAME           READY   STATUS    RESTARTS   AGE
kubia-manual   1/1     Running   0          7s

Disinterested

Answer 6

Check this great article: How to Run Locally Built Docker Images in Kubernetes

My Environment

• OS: Debian
• Version: 11 (Bullseye)
• Minikube: v1.24.0
• Kubectl: v1.22.4
• Docker: 20.10.5+dfsg1

It covers all wrong image stages and workarounds.

Stages

ErrImagePull

Edit deployment.yaml and add imagePullPolicy: Never:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-local-deployment
spec:
  template:
    metadata:
      name: my-pod
    spec:
      containers:
      - name: my-container
        image: my-local-image    # The one built and tagged locally
        imagePullPolicy: Never   # So it doesn't pull the image from remote registry

ErrImageNeverPull

Run: eval $(minikube -p minikube docker-env)

that outputs environment variables needed to point the local Docker daemon to the minikube internal Docker registry

Finally

Rebuild the image, to install it in minikube registry:

docker build . -t my-image

Threadfin

one question about "eval $(minikube -p minikube docker-env)" does that means your docker command will be related always to minikube machine ? I mean I would like to keep using docker out of the minikube context for other projects etc... – Compel

Answer 7

As per the kubernetes documentation here: https://kubernetes.io/docs/concepts/containers/images/#imagepullpolicy-defaulting

If you use the latest tag for deployment then it will try to pull image. I was able to deploy service with v1 tag after following steps on WSL-2

1. eval $(minikube -p minikube docker-env)
2. minikube start
3. docker build --tag=my-service:v1 .
4. kubectl create deployment my-service --image=my-service:v1
5. kubectl expose pod "my-service" --type=NodePort --port=8080
6. minikube service my-serivce

Crenulate

Answer 8

in my case, I had to do the following steps.

$ eval $(minikube docker-env)

$ minikube config set driver virtualbox

$ minikube start

Impediment

Answer 9

Adding --insecure-registry="$REG_IP":80 does not seem to work for me.

I have to use wlredeye's answer at Can not pull docker image from private repo when using Minikube to get it working:

For an http registry this steps works for me:

1. minikube ssh
2. edit /var/lib/boot2docker/profile and add to $EXTRA_ARGS --insecure-registry 192.168.99.1(your local machine's IP):5000
3. restart the docker daemon sudo /etc/init.d/docker restart
4. Verify you can access the docker registry curl -s -S http://192.168.99.1/v2/_catalog

This is minikube version: v0.14.0

Kingcraft

Two questions about this one. By "local machine's IP" you mean "the IP of the registry" right? Also how exactly are you "adding to" that variable. – Tirpitz

I added the line: EXTRA_ARGS --insecure-registry 192.168.99.1:5000, but I get an error with $ sudo /etc/init.d/docker restart sudo: /etc/init.d/docker: command not found – Unsociable

Answer 10

First you need to start a local registry, seems you have done it.

docker run -d -p 5000:5000 --restart=always --name myregistry registry:2

You can check your minikube IP by ifconfig

There should be output (after minikube start using virtualbox as VMdriver) looks like:

vboxnet0 Link encap:Ethernet HWaddr 0a:00:27:00:00:00
inet addr:192.168.99.1 Bcast:192.168.99.255 Mask:255.255.255.0 inet6 addr: fe80::800:27ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:515 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:86256 (86.2 KB)

Using

minikube start --insecure-registry=192.168.99.1:5000

You can actually check whether you can pull your own images simply by (within minikube)

docker pull 192.168.99.1:5000/your_own_repo/your_own_images

Hope this might help.

Angio

Answer 11

in Mac I have problem with dns-resolving in minikube, as /etc/resolv.conf was configured with my mac ip. I didn't have installed dns server locally, so minikube couldn't resolve anything.

I solved this with install locally dns-server

brew install dnsmasq
sudo brew services start dnsmasq

Lenhard

Answer 12

in Mac I had docker client installed so I had to do

minikube config set driver docker
minikube start

Dogface

Answer 13

Start minikube

minikube start --insecure-registry "10.0.0.0/24,192.168.39.0/24"
minikube addons enable registry

Find minikube IP

minikube ip

Setup docker/insecure-registry

# edit /etc/docker/daemon.json
{
  ...
  "insecure-registries": [ "your-minikube-ip:5000" ]
  ...
}

Push your image

docker build --tag $(minikube ip):5000/local-image-build:latest
docker push $(minikube ip):5000/local-image-build:latest

Enjoy it.

See:
https://minikube.sigs.k8s.io/docs/handbook/pushing/#4-pushing-to-an-in-cluster-using-registry-addon

Flowerlike

Answer 14

in my case(running minikube on M1), adding imagePullPolicy: Never fixed the issue

Malaria