How to debug kubectl apply for kube-flannel.yml?

docker run -it \ -v /etc:/rootfs/etc \ -v /opt:/rootfs/opt \ -v /usr/bin:/rootfs/usr/bin \ -e K8S_VERSION=v1.8.4 \ -e CNI_RELEASE=v0.6.0 \ xakra/kubeadm-installer:0.4.7 coreos

NAMESPACE NAME READY STATUS RESTARTS AGE kube-system etcd-coreos1 1/1 Running 0 18m kube-system kube-apiserver-coreos1 1/1 Running 0 18m kube-system kube-controller-manager-coreos1 0/1 CrashLoopBackOff 8 19m kube-system kube-scheduler-coreos1 1/1 Running 0 18m

Just tonight I used kubespray to provision a vagrant cluster, on CoreOS, using flannel (vxlan), and I was also mystified about how flannel could be a Pod inside Kubernetes

It turns out, as seen here, that they are using flannel-cni image from quay.io to write out CNI files using a flannel side-car plus hostDir volume-mounts; it outputs cni-conf.json (that configures CNI to use flannel), and then net-conf.json (that configures the subnet and backend used by flannel).

I hope the jinja2 mustache syntax doesn't obfuscate the answer, but I found it very interesting to see how the Kubernetes folks chose to do it "for real" to compare and contrast against the example DaemonSet given in the flannel-cni README. I guess that's the long way of saying: try the descriptors in the flannel-cni README, then if it doesn't work see if they differ in some way from the known-working kubespray setup

update: as a concrete example, observe that the Documentation yaml doesn't include the --iface= switch, and if your Vagrant setup is using both NAT and "private_network" then it likely means flannel is binding to eth0 (the NAT one) and not eth1 with a more static IP. I saw that caveat mentioned in the docs, but can't immediately recall where in order to cite it

update 2

Is there a way to know why this command doesn't work? Can I get some logs from anywhere?

One may almost always access the logs of a Pod (even a statically defined one such as kube-controller-manager-coreos1) in the same manner: kubectl --namespace=kube-system logs kube-controller-manager-coreos1, and in the CrashLoopBackOff circumstance, adding in the -p for "-p"revious will show the logs from the most recent crash (but only for a few seconds, not indefinitely), and occasionally kubectl --namespace=kube-system describe pod kube-controller-manager-coreos1 will show helpful information in either the Events section at the bottom, or in the "Status" block near the top if it was Terminated for cause

In the case of a very bad failure, such as the apiserver failing to come up (and thus kubectl logs won't do anything), then ssh-ing to the Node and using a mixture of journalctl -u kubelet.service --no-pager --lines=150 and docker logs ${the_sha_or_name} to try and see any error text. You will almost certainly need docker ps -a in the latter case to find the exited container's sha or name, but that same "only for a few seconds" applies, too, as dead containers will be pruned after some time.

In the case of vagrant, one can ssh into the VM in one of several ways:

vagrant ssh coreos1
vagrant ssh-config > ssh-config && ssh -F ssh-config coreos1
or if it has a "private_network" address, such as 192.168.99.101 or such, then you can usually ssh -i ~/.vagrant.d/insecure_private_key [email protected] but one of the first two are almost always more convenient

Recommended topics

Hot tags