What does userspace mode means in kube-proxy's proxy mode?

Question

kube-proxy has an option called --proxy-mode，and according to the help message, this option can be userspace or iptables.(See below)

# kube-proxy -h
Usage of kube-proxy:
...
      --proxy-mode="": Which proxy mode to use: 'userspace' (older, stable) or 'iptables' (experimental). If blank, look at the Node object on the Kubernetes API and respect the 'net.experimental.kubernetes.io/proxy-mode' annotation if provided.  Otherwise use the best-available proxy (currently userspace, but may change in future versions).  If the iptables proxy is selected, regardless of how, but the system's kernel or iptables versions are insufficient, this always falls back to the userspace proxy.
...

I can't figure out what does userspace mode means here.

Anyone can tell me what the working principle is when kube-proxy runs under userspace mode?

Profiterole

Answer 1

Userspace and iptables refer to what actually handles the connection forwarding. In both cases, local iptables rules are installed to intercept outbound TCP connections that have a destination IP address associated with a service.

In the userspace mode, the iptables rule forwards to a local port where a go binary (kube-proxy) is listening for connections. The binary (running in userspace) terminates the connection, establishes a new connection to a backend for the service, and then forwards requests to the backend and responses back to the local process. An advantage of the userspace mode is that because the connections are created from an application, if the connection is refused, the application can retry to a different backend.

In iptables mode, the iptables rules are installed to directly forward packets that are destined for a service to a backend for the service. This is more efficient than moving the packets from the kernel to kube-proxy and then back to the kernel so it results in higher throughput and better tail latency. The main downside is that it is more difficult to debug, because instead of a local binary that writes a log to /var/log/kube-proxy you have to inspect logs from the kernel processing iptables rules.

In both cases there will be a kube-proxy binary running on your machine. In userspace mode it inserts itself as the proxy; in iptables mode it will configure iptables rather than to proxy connections itself. The same binary works in both modes, and the behavior is switched via a flag or by setting an annotation in the apiserver for the node.

Degroot

Answer 2

I tried to make it simple

working principle is when kube-proxy runs under userspace mode

In the userspace proxy mode, the kube-proxy would perform the following steps:

1. Configure iptables rules to redirect connections destined to the service IP addresses to the proxy server process running on the node.

2. The proxy server process would accept these redirected connections.

3. The proxy server would then proxy (forward) the connections to one of the backend pods for the service.

4. Responses from the backend pods would be sent back through the proxy server process to the client.

Essentially, the userspace proxy mode involved an extra hop, where the connections had to go through an actual proxy server process running on each node before reaching the backend pods.

Hope this help to visualise better.

Overcrop