How can I use Flannel without disabing firewalld (Kubernetes)

Question

I'm new to flannel and K8s. I'm playing around them on my 1 master and 2 nodes cluster (created from KVM).

I initialized my cluster with flannel network addon. And then I found I can't reach the Internal. It turned out that there may be something wrong with my network or DNS process.

Following https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/, I add 8285 and 8472 to firewalld

[root@k8smaster ~]# firewall-cmd --list-port
6443/tcp 80/tcp 8285/udp 8472/udp 8472/tcp 8285/tcp

And I got

[root@k8smaster ~]# kubectl exec -ti dnsutils -- nslookup kubernetes.default
;; connection timed out; no servers could be reached

command terminated with exit code 1

However, if I stop the firewalld, everything is just fine.

[root@k8smaster ~]# kubectl exec -ti dnsutils -- nslookup kubernetes.default
Server:     10.96.0.10
Address:    10.96.0.10#53

Name:   kubernetes.default.svc.cluster.local
Address: 10.96.0.1

So my question is, are there any ports I should add into firewalld? Let me know if any info. I should add here. Thank you.

Feuchtwanger

Have you configured the others kubernetes ports? Check this link – Tippler

@KoopaKiller Oh man, you're right! – Feuchtwanger

Since it helped you to solve your issue, I've just post an answer to resume all ports you need to allow in firewall rule. Will be nice if you could accept the answer to help the community. – Tippler

Answer 1

The rules you are applying referred only for Flannel communication, you need to allow all Kubernetes ports to make it work.

In resume, you need to apply rules for these ports:

Control-plane node(s)
Protocol        Direction       Port Range      Purpose Used By
TCP     Inbound 6443*   Kubernetes API server   All
TCP     Inbound 2379-2380       etcd server client API  kube-apiserver, etcd
TCP     Inbound 10250   Kubelet API     Self, Control plane
TCP     Inbound 10251   kube-scheduler  Self
TCP     Inbound 10252   kube-controller-manager Self

Worker node(s)
Protocol        Direction       Port Range      Purpose Used By
TCP     Inbound 10250   Kubelet API     Self, Control plane
TCP     Inbound 30000-32767     NodePort Services†      All
† Default port range for NodePort Services.
~                                          

Here you could see all necessary ports for Control-plane and Worker nodes.

And then after this rules applied you need to apply more other 2 rules to allow flannel network as mentioned here.

Make sure that your firewall rules allow UDP ports 8285 and 8472 traffic for all hosts participating in the overlay network. The Firewall section of Flannel’s troubleshooting guide explains about this in more detail.

Tippler

Answer 2

Suggested by @KoopaKiller, I indeed missed configuring the ports for k8s itself. I paste my script below for your reference.

# Master
firewall-cmd --permanent --add-port=6443/tcp # Kubernetes API server
firewall-cmd --permanent --add-port=2379-2380/tcp # etcd server client API
firewall-cmd --permanent --add-port=10250/tcp # Kubelet API
firewall-cmd --permanent --add-port=10251/tcp # kube-scheduler
firewall-cmd --permanent --add-port=10252/tcp # kube-controller-manager
firewall-cmd --permanent --add-port=8285/udp # Flannel
firewall-cmd --permanent --add-port=8472/udp # Flannel
firewall-cmd --add-masquerade --permanent
# only if you want NodePorts exposed on control plane IP as well
firewall-cmd --permanent --add-port=30000-32767/tcp
firewall-cmd --reload
systemctl restart firewalld


# Node
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=8285/udp # Flannel
firewall-cmd --permanent --add-port=8472/udp # Flannel
firewall-cmd --permanent --add-port=30000-32767/tcp
firewall-cmd --add-masquerade --permanent
firewall-cmd --reload
systemctl restart firewalld

Feuchtwanger

The firewall-cmd --add-masquerade --permanent was the key for me – Oconnell