kubernetes networkpolicy allow external traffic to internet only

Question

Im trying to implement network policy in my kubernetes cluster to isolate my pods in a namespace but still allow them to access the internet since im using Azure MFA for authentication.

This is what i tried but cant seem to get it working. Ingress is working as expected but these policies blocks all egress.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes:
  - Ingress 

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: grafana-policy
  namespace: default
spec:
  podSelector:
    matchLabels: 
      app: grafana
  ingress:
  - from:
    - podSelector:
       matchLabels: 
        app: nginx-ingress

Anybody who can tell me how i make above configuration work so i will also allow internet traffic but blocking traffic to other POD's?

Fumed

Answer 1

Try adding a default deny all network policy on the namespace:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

Then adding an allow Internet policy after:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-internet-only
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 10.0.0.0/8
        - 192.168.0.0/16
        - 172.16.0.0/20

This will block all traffic except for internet outbound. In the allow-internet-only policy, there is an exception for all private IPs which will prevent pod to pod communication.

You will also have to allow Egress to Core DNS from kube-system if you require DNS lookups, as the default-deny-all policy will block DNS queries.

Chloromycetin

Does not work for me :( when i add this my PODs are still able to communicate but they are not able to access the internet. – Fumed

I believe 172.16.0.0/20 should be 172.16.0.0/12 as per en.wikipedia.org/wiki/Private_network#Private_IPv4_addresses – Saltzman

Seems like cidr should be indented one more level thanipBlock. – Toolis

Allow DNS egress traffic example from Calico (K8s-native). Make sure you have the correct namespace label (kubectl get ns --show-labels kube-system), for K3s it was actually kubernetes.io/metadata.name=kube-system – Farming

This policy blocks all egress when i apply it to my namespace :( I have even tried removing the "except" section, and still all egress (fro curl) is blocked. – Indecipherable

@EsbenEickhardt you need to whitelist the DNS server too. I added an answer to complement this – Lithium

Answer 2

Something like what @user100.. should do but you ALSO need to allow DNS lookup on top of that like below.

  egress:

  # Allow communication to Kubernetes DNS service
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
    - podSelector:
        matchLabels:
          k8s-app: kube-dns
    ports:
    - protocol: UDP
      port: 53

  # Allow internet access
  - to:
    - ipBlock: 
        cidr: 0.0.0.0/0

        # Exclude traffic to Kubernetes service IPs and pods
        except:
          - 10.0.0.0/8
          - 172.16.0.0/12
          - 192.168.0.0/16

Lithium

Answer 3

Kubernetes will allow all traffic unless there is a network policy. If a Network Policy is set, it will only allow traffic set by the network policy and deny everything else.

By default, pods are non-isolated; they accept traffic from any source.

Pods become isolated by having a NetworkPolicy that selects them. Once there is any NetworkPolicy in a namespace selecting a particular pod, that pod will reject any connections that are not allowed by any NetworkPolicy. (Other pods in the namespace that are not selected by any NetworkPolicy will continue to accept all traffic.)

https://kubernetes.io/docs/concepts/services-networking/network-policies/#isolated-and-non-isolated-pods

So you will need to specify the Egress rules as well in order for it to work the way you want :)

Nozicka

Answer 4

Four steps are required to solve this ask.

1. Deny all ingress/egress.
2. Grant internet access to the specific pod(s).
3. Grant ingress/egress from some specific pods to the allowed pod(s).
4. Grant kube-dns access to the "from" pod(s).

Denies everything:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

Allows internet

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-internet
spec:
  podSelector:
    matchLabels:
app: some-http
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
cidr: 0.0.0.0/0

Grants pod to pod access.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-some-http
spec:
  podSelector:
    matchLabels:
      app: some-other-http
    policyTypes:
    - Ingress
    ingress:
    - from:
      - podSelector:
          matchLabels:
            app: some-http
        ports:
          - port: 80
            protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-http
spec:
  podSelector:
    matchLabels:
      app: some-http
      policyTypes:
        - Egress
      egress:
      - to:
        - podSelector:
            matchLabels:
              app: some-other-http
          ports:
            - port: 80
              protocol: TCP

Grants kube-dns access

---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
spec:
  podSelector:
    matchLabels:
      app: some-http
  policyTypes:
  - Egress
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          name: kube-system
    - podSelector:
        matchLabels:
          k8s-app: kube-dns

Goering

Answer 5

Can you try like this?

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes:
  - Ingress,Egress
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0

It should allow egress to all destinations. But if the destination is a pod, it should be blocked by the lacking ingress rules of the same NetworkPolicy.

Cere

I already tried something like this,anyway i just added your NetworkPolicy and my pod are able to talk with Azure but pod to pod traffic is still getting allowed. i'm testing by taking a console to the pod and doing curl to another pod in same namespace. – Fumed