Monitor or log dropped network traffic for Kubernetes NetworkPolicy

Question

I'm interested in using Kubernetes NetworkPolicy to control network policy. I want to know if the NetworkPolicy is blocking traffic so I can either fix the policies or fix/stop whatever is in violation.

We use Calico and they view this as a paid feature. https://github.com/projectcalico/calico/issues/1035

Cilium has cilium monitor which sounds like it would work if we started using Cilium. http://docs.cilium.io/en/latest/troubleshooting/

Is there a general, vendor-neutral way to monitor network traffic that violates Kuberenetes NetworkPolicy?

Keli

Answer 1

AFAIU, there is no way to create such vendor-neutral tool because NetworkPolicy is just an abstraction. Each networking plugin enforces them differently, (Cilium does that mostly in BPF for L3 and L4 and Envoy for L7), so each plugin needs to provide its own means of accessing this information.

AFAIK, there is no initiative in Kubernetes community to store this information and provide an interface for CNI plugins to provide this information, but it seems like it would be a fun project.

Disclaimer: I am on Cilium dev team.

Cleveite

This is the conclusion I came to as I understood things better. BTW, I really like Cilium. It seems a little raw to trust in production but I anticipate switching to it in 6 months or a year. – Keli

Answer 2

Calico's native NetworkPolicy supports a "log" action that allows you to log packets. Then, you can monitor these logs with a monitoring software. Logging is not a default option using calico! (see calico's doc)

So, for example, you have a pod called "db", and you want to create a network policy that blocks and LOGS all TCP traffic destined for the "db" pod, here is a sample manifest (calico-npc-db.yaml):

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: calico-npc-db-netpol
  namespace: npc
spec:
  selector: app == 'db'
  ingress:
  - action: Log
    protocol: TCP

Then you apply this manifest:

k apply -f calico-npc-db.yaml

In your cluster node, let's assume the node name is ks8-worker-02, you will see the following type of blocking message in the standard log files (/var/log):

Apr 24 11:18:42 k8s-worker-02 kernel: [586144.409226] calico-packet:
IN=cali945ac7714c6 OUT=calie2d9c08122c 
MAC=ee:ee:ee:ee:ee:ee:0a:c8:dc:75:88:f5:08:00 SRC=192.168.118.67 
DST=192.168.118.68 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=2617 DF 
PROTO=TCP SPT=53712 DPT=3306 WINDOW=64860 RES=0x00 SYN URGP=0 

Pastose