Helm install, Kubernetes - how to wait for the pods to be ready?

Question

I am creating a CI/CD pipeline.

I run helm install --wait --timeout 300 .... But that doesn't really wait, just returns when the "release" status is DEPLOYED.

So then I see a few things in kubectl get pods --namespace default -l 'release=${TAG}' -o yaml that could be used:

- kind: Pod
  status:
    conditions:
    - lastProbeTime: null
      lastTransitionTime: 2018-05-11T00:30:46Z
      status: "True"
      type: Initialized
    - lastProbeTime: null
      lastTransitionTime: 2018-05-11T00:30:48Z
      status: "True"
      type: Ready

So I guess I will look at when Ready condition becomes "True".

1. It feels a bit wrong thing to do... Everyone solves this so I assume there is some feature of kubectl for that, is there?

2. Is this the right thing to query? (See Kubernetes JSONPath reference)

   kubectl get pods --namespace default -l 'release=sc8757070' -o jsonpath='{.items[*].status.conditions[?(@.type=="Ready")].status}'

Vladikavkaz

Answer 1

Kubernetes already has something to wait on pods (and print a message every time something changes and print a summary at the end).

# kubectl rollout status RESOURCE_TYPE RESOURCE_NAME 
kubectl rollout status statefulset app1-hello

# with timeout set to 300 seconds
kubectl rollout status statefulset app1-hello --timeout=300s

Bussard

This is almost what I am looking for, except it has no timeout (at least the helm doesn't show any such option), so in my case, it waits infinitely. Maybe I should limit the number of restart attempts somewhere? – Vengeful

Should I set restartPolicy=Never? That way, would it stop waiting after first fail? Pod Lifecycle in Kubernetes docs – Vengeful

Thanks @Amit_Kumar_Gupta! This worked. I had to use kubectl rollut status deployment/<service-name-here>....it waited until rollout completed. – Topology

Answer 2

There is a good option -n | --namespace NAMESPACE that allow to wait deployments in different namespaces, for example in kube-system:

kubectl rollout status deployment tiller-deploy -n kube-system

Classics

Answer 3

I guess you should try this command. It will be rollback and deploy again until it really will be OK deployed.

helm upgrade --install --atomic --timeout 300s <<release_name>> ./<<dir_name>> --namespace <<namespace>>

Bethought

Answer 4

# Wait until gitlab UI is ready..
until kubectl get pods --namespace dev | grep gitlab-dev-webservice-default | grep Running | grep 2/2
do
    echo "Wait for healthy gitlab-dev-webservice-default Pods..."
    sleep 10
done

Cloris