How can I keep a container running on Kubernetes?

Question

I'm now trying to run a simple container with shell (/bin/bash) on a Kubernetes cluster.

I thought that there was a way to keep a container running on a Docker container by using pseudo-tty and detach option (-td option on docker run command).

For example,

$ sudo docker run -td ubuntu:latest

Is there an option like this in Kubernetes?

I've tried running a container by using a kubectl run-container command like:

kubectl run-container test_container ubuntu:latest --replicas=1

But the container exits for a few seconds (just like launching with the docker run command without options I mentioned above). And ReplicationController launches it again repeatedly.

Is there a way to keep a container running on Kubernetes like the -td options in the docker run command?

Stateroom

Using this image (as Kubernetes docs suggests) is quite handy: kubectl run curl --image=radial/busyboxplus:curl -i --tty – Rathbun

This question has been mentioned at this video: Kubernetes the very hard way at Datadog with a slide-title of "Cargo culting. From wikipedia: The term cargo cult programmer may apply when an unskilled or novice computer programmer (or one inexperienced with the problem at hand) copies some program code from one place to another with little or no understanding of how it works or whether it is required in its new position. – Stalinsk

I'll admit that I'm still pretty early in my Kubernetes learning, but mentioning this as Cargo Culting seems unfair. There is a very good reason why you might want a Kubernetes pod to live forever (or, until manually killed) - if you want to test the networking within the cluster using tools, environment, etc. available within a pod. Yes, you could repeatedly spin up short-lived pods executing the commands you want, but isn't it easier to start a persistent pod and start a shell on it? – Kareem

I'm another 18 months or so into my Kubernetes learning, and I stand by past-me - in fact, the fact that I'm back at this StackOverflow question reminding myself how to do this very thing (only to find myself nodding along at the previous wise commenter with a similar name to me) just demonstrates that this technique is (still) useful to me. – Kareem

Answer 1

A container exits when its main process exits. Doing something like:

docker run -itd debian

to hold the container open is frankly a hack that should only be used for quick tests and examples. If you just want a container for testing for a few minutes, I would do:

docker run -d debian sleep 300

Which has the advantage that the container will automatically exit if you forget about it. Alternatively, you could put something like this in a while loop to keep it running forever, or just run an application such as top. All of these should be easy to do in Kubernetes.

The real question is why would you want to do this? Your container should be providing a service, whose process will keep the container running in the background.

Schaeffer

Thank you for your answer. I'm now trying to understand the behavior of containers, with dozens of containers running at the same time. Endless loop and using other heavy command, prevent me to know what is the behavior of containers. That is the reason why I need simple container like only running /bin/bash. – Stateroom

For now, I'll try running cat without arguments and top and sleep with argument of large number. – Stateroom

sleep infinity works in a lot of cases (not busybox) – Penman

There's plenty of reasons to do this. For example, you might deploy your pods with helm releases and injected configuration, which is going to make recreation of similar environments annoying and cumbersome. But having a container with that configuration in cases where the other pods crash/are deleted can be infinitely useful. – Charlatan

Because my service consists of multiple processes. – Waldo

@Adrian: I thought Container is a process inside the Node, which is a VM. I am bit confused with all this basic concept. Even opened a question for this - #66231233............ if you have got some time to look at it, and answer if it's not too stupid to answer.. – Abandoned

@КонстантинВан Hey, younger me! You want to make a Pod instead! You don't put everything in a single "container." – Waldo

"The real question is why would you want to do this?" - debugging. Maybe your service isn't immediately crashing, so you need to keep the container alive while you check everything internally (by interactively executing a shell, for example). – Reduced

Answer 2

Containers are meant to run to completion. You need to provide your container with a task that will never finish. Something like this should work:

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
spec:
  containers:
  - name: ubuntu
    image: ubuntu:latest
    # Just spin & wait forever
    command: [ "/bin/bash", "-c", "--" ]
    args: [ "while true; do sleep 30; done;" ]

Algor

But is this a best practice? – Remand

@aneeshjoshi I wouldn't say this is best practice. This is just an example providing a pod that will run without immediately exiting. Best practice is to create your containers to do the job they were designed for (a job that runs to completion, a webserver that runs perpetually, etc.). I posted this as an example because Kubernetes can feel initially frustrating when you keep creating pods only to have the disappear since the default command exits immediately. – Algor

@JoelB Thanks. I was wondering what is the "right" way of doing it. – Remand

thanks for this as I am in need of a container that can live for awhile allowing me to enter it. I was trying to do the same with a lighter image than ubuntu and tried bash image but couldn't get it to work. Any idea how to do the same as this with bash image? – Guan

I know this is an old issue; however, Kubernetes is started to support ephemeral containers. link:kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers. These containers have some limitations like resource limits, however they are designed for debugging purposes. – Culet

@Guan For a lighter image, you may use alpine, the container spec can look like: {"name": "util", "image": "alpine", "command": [ "/bin/sh", "-c", "--" ], "args": [ "while true; do sleep 30; done;" ]} (only json because the yaml won't format in comments here.) The important bit being /bin/sh instead of /bin/bash. – Equilibrant

"are meant to run to completion"? – Louque

should CMD command at Dockerfile be a better approach? see this ref – Millman

Can you please explain what "-c --" flags are? I'm curious to learn more about it. – Springclean

It's an argument to the BASH shell, telling it that everything after it should be viewed as executable and not flags to the shell. – Algor

Answer 3

You could use this CMD in your Dockerfile:

CMD exec /bin/bash -c "trap : TERM INT; sleep infinity & wait"

This will keep your container alive until it is told to stop. Using trap and wait will make your container react immediately to a stop request. Without trap/wait stopping will take a few seconds.

For busybox based images (used in alpine based images) sleep does not know about the infinity argument. This workaround gives you the same immediate response to a docker stop like in the above example:

CMD exec /bin/sh -c "trap : TERM INT; sleep 9999999999d & wait"

Acetylene

I'm using the same on kubernetes deployment yaml for debugging purposes – Brockbrocken

this gives me "sleep: invalid number 'infinity'" – Kaiserism

@Kaiserism Thanks for this. You are probably stuck with an image that uses busybox (like alpine images). See the updated answer. – Acetylene

Instead of sleep infinity, just use sleep 999999999d where d indicates days. – Massy

Thank you for the info on trapping TERM/INT! I couldn't understand why my container wasn't stopping correctly, since sleep will respond to it from the terminal – Circumlunar

If someone has working args version for Kubernetes YAML that would be superb. UPDATE: Use this: command: ["/bin/bash"] and args: ["-c", "trap : TERM INT; sleep infinity & wait"] – Knocker

It's often preferred to use the ["json array"] syntax for CMD. If that's what you need, you could write it this way: CMD ["/bin/bash", "-c", "exec /bin/bash -c 'trap : TERM INT; sleep 9999999999d & wait'"] – Maid

Answer 4

1. In your Dockerfile use this command:

   CMD ["sh", "-c", "tail -f /dev/null"]
   

2. Build your docker image.

3. Push it to your cluster or similar, just to make sure the image it's available.

4. kubectl run debug-container -it --image=<your-image>
   

Kraus

Great tips for debugging the container. – Ennui

Do you need the sh -c? I'm pretty sure thie does the trick too: CMD ["tail", "-f", "/dev/null"] – Shockproof

@PeterV.Mørch, I doubt, but still think that in case you write it without sh -c and it will work like a charm. – Yaker

Answer 5

The simplest command as it can be for k8s pod manifest to run container forever:

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
spec:
  containers:
  - name: ubuntu
    image: ubuntu:latest
    # Just sleep forever
    command: [ "sleep" ]
    args: [ "infinity" ]

Clipping

The most elegant and minimalistic solution. – Bosanquet

Getting an error with that exact code The Pod "ubuntu" is invalid: spec: Forbidden: pod updates may not change fields other than `spec.containers[*].image`, `spec.initContainers[*].image`, `spec.activeDeadlineSeconds` or `spec.tolerations` (only additions to existing tolerations) – Aesir

It means you're trying to change part of manifest which is immutable (with kubectl apply). Try with kubectl replace with optional --force flag – Clipping

Answer 6

In order to keep a POD running it should to be performing certain task, otherwise Kubernetes will find it unnecessary, therefore it stops. There are many ways to keep a POD running.

I have faced similar problems when I needed a POD just to run continuously without doing any useful operation. The following are the two ways those worked for me:

1. Running sleep command while running the container.
2. Running an infinite loop inside the container.

Although the first option is easier than the second one and may suffice the requirement, it is not the best option. As, there is a limit as far as the number of seconds you are going to assign in the sleep command. But a container with infinite loop running inside it never exits.

However, I will describe both the ways(Considering you are running busybox container):

1. Sleep Command

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  labels:
    app: busybox
spec:
  containers:
  - name: busybox
    image: busybox
    ports:
    - containerPort: 80
    command: ["/bin/sh", "-ec", "sleep 1000"]

2. Infinite Loop

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  labels:
    app: busybox
spec:
  containers:
  - name: busybox
    image: busybox
    ports:
    - containerPort: 80
    command: ["/bin/sh", "-ec", "while :; do echo '.'; sleep 5 ; done"]

Run the following command to run the pod:

kubectl apply -f <pod-yaml-file-name>.yaml

Hope it helps!

Dis

May I ask, what is sleep? Is it a command internal to Ubuntu? Or docker command? – Tigges

@Tigges It's a linux shell command, it's not specific to docker. – Dis

Answer 7

My few cents on the subject. Assuming that kubectl is working then the closest command that would be equivalent to the docker command that you mentioned in your question, would be something like this.

$ kubectl run ubuntu --image=ubuntu --restart=Never --command sleep infinity

Above command will create a single Pod in default namespace and, it will execute sleep command with infinity argument -this way you will have a process that runs in foreground keeping container alive.

Afterwords, you can interact with Pod by running kubectl exec command.

$ kubectl exec ubuntu -it -- bash

This technique is very useful for creating a Pod resource and ad-hoc debugging.

Elfreda

Works great. No need for the --restart=Never, just call kubectl run ubuntu --image=ubuntu -- sleep infinity – Richardson

or if you want to do old fashioned thicker commands you can always do something like kubectl run busybox --image=busybox --restart=Never -- /bin/sh -c 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done;' – Soppy

Answer 8

I was able to get this to work with the command sleep infinity in Kubernetes, which will keep the container open. See this answer for alternatives when that doesn't work.

Penman

This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. - From Review – Unzip

@Unzip Sure it does. sleep infinity keeps the container open, providing the same type of functionality that the question asks about (for most types of containers). It also provides a link to alternatives for cases when that specific command doesn't work – Penman

This was from review. If you add the comment text there to the answer then it's a quality answer :) My initial flag/commend was based on saying your comment wasn't successful, making me think this was supposed to be a comment. Added a quick edit and upvoted. – Unzip

"...sleep infinity in Kubernetes" is an uninformed statement. It implies that there is no unix and no docker in the picture. – Centaury

Answer 9

Use this command inside you Dockerfile to keep the container running in your K8s cluster:

• CMD tail -f /dev/null

Bolometer

Answer 10

add this : in template ->in spec-> in container ->in ports & after container port line

    command: ["/bin/sh", "-ec", "while :; do echo '.'; sleep 6 ; done"]

Laminate

sharing the whole file with command would be more helpful to others instead of command. – Torment

Answer 11

In my case, a pod with an initContainer failed to initialize. Running docker ps -a and then docker logs exited-container-id-here gave me a log message which kubectl logs podname didn't display. Mystery solved :-)

Howlett

Answer 12

There are many different ways for accomplishing this, but one of the most elegant one is:

kubectl run -i --tty --image ubuntu:latest ubuntu-test --restart=Never --rm /bin/sh

Karlakarlan

Why do you consider it the most elegant solution? – Asare

@mordowiciel, because it directly relates to docker command and it's not split into two commands like the most of the other answers. – Trix

Answer 13

I did a hack by putting it in background:

[root@localhost ~]# kubectl run hello -it --image ubuntu -- bash &
[2] 128461

Exec on pod hello

[root@localhost ~]# kubectl exec -it hello -- whoami
root
[root@localhost ~]# kubectl exec -it hello -- hostname
hello

Getting a shell

[root@localhost ~]# kubectl exec -it hello -- bash
root@hello:/# ls
bin  boot  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

Slimsy

The -it part is what I was missing! So, kubectl run test-container -it --image alpine -- sh gave me exactly what I was looking for, a pod with alpine running inside the cluster. Thanks! – Endosperm

Answer 14

This command may help

CMD exec /bin/bash -c "trap : TERM INT; sleep i infinity & wait"

Ninon

what is i for in sleep i infinity? – Hern

Answer 15

If you want to keep images pre-cached in the nodes, I think that could be the idea, you can have a daemonset that distribute the pod image to all nodes or those that you are interested, and you can use sleep infinity to keep the containers alive and also avoid that the images to be cleaned up by k8s GC. Note: Keep in mind the some docker images does not contains /bin/sh binary, for those you need a workaround. Sharing below the example where the /bin/sh binary is not present, if that is not your case, then just delete the initContainers and the volumes that shares the binary files to the containers. In the example busybox is used, you can also adjust the initContainers and use the one that contains the missing binary.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: image-cache
  namespace: default # adjust the namespace
spec:
  selector:
    matchLabels:
      app: image-cache
  template:
    metadata:
      labels:
        app: image-cache
    spec:
      # Specify node affinity if you need it
      # Specify security context global if you need it
      initContainers:
        - name: busybox-musl
          image: busybox:musl
          command:
            - sh
          args:
            - -c
            - |
              echo "Copying binary..."
              cp /bin/busybox /usr/shared/binaries
              chown 1000:1000 /usr/shared/binaries/busybox
              chmod 775 /usr/shared/binaries/busybox
              echo "... Done."
          volumeMounts:
            - name: busybox-shared
              mountPath: /usr/shared/binaries
      containers:
        - name: container-name
          image: desired-container-image:version
          command:
            - /usr/shared/binaries/busybox
          args:
            - sleep
            - infinity
          volumeMounts:
            - name: busybox-shared
              mountPath: "/usr/shared/binaries"
      volumes:
        - name: busybox-shared
          emptyDir: {}

Huygens

Answer 16

Keep a Kubernetes pod open (medium article)

Create a pod with stdin: true

Then kubectl exec -it ubuntu -- /bin/bash

Boom, you have a pod of whatever image you need running and staying open. Shell into it at will.

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: ubuntu
  name: ubuntu
spec:
  containers:
  - image: ubuntu:latest
    name: ubuntu
    stdin: true
  dnsPolicy: ClusterFirst
  restartPolicy: Always

Harberd

Answer 17

edit deployment and add:

          command:
            - /bin/bash
            - sleep 1000 

to spec.template.spec.containers section

Spoiler

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Chrysarobin