How can I rate limit network traffic on a docker container

Question

I want to setup a docker container for a peer 2 peer app. This app doesn't have app level rate limiting so I'm attempting to set a limit at the container level. I would like to rate limit outgoing and incoming connections on all ports but the one used by the app's web UI.

Foible

Answer 1

I'm surprised at how difficult it was to find the answer to this question. Most answers on the various forums are incorrect (I tested them with two iperf3 nodes and found that the solutions didn't work or only limited one direction of traffic (only incoming or only outgoing). A P2P application that has much more symmetric data usage than traditional client/server applications so traffic must be limited in both directions.

The best way I've found is to limit network bandwidth (both incoming and outgoing) for a Docker container is to use Linux's own traffic control settings within the running container. Execute the tc commands inside the container before you start your P2P application.

For example, you could create a start-up script like the following, copy it into your docker image and invoke it as the ENTRYPOINT.

Dockerfile (snippet):

COPY start-my-p2p.sh /
RUN chmod +x /start-my-p2p.sh    
ENTRYPOINT /start-my-p2p.sh   

Put something like this in your start-my-p2p.sh (the tc cmdlines are probably what you've been searching the Internet for):

#/bin/sh

# Limit all incoming and outgoing network to 1mbit/s
tc qdisc add dev eth0 handle 1: ingress
tc filter add dev eth0 parent 1: protocol ip prio 50 u32 match ip src 0.0.0.0/0 police rate 1mbit burst 10k drop flowid :1
tc qdisc add dev eth0 root tbf rate 1mbit latency 25ms burst 10k`

# Now start your p2p application
myp2pservice -d 

IMPORTANT: When starting the container you'll need to use --cap-add=NET_ADMIN:

docker run --rm -it --cap-add=NET_ADMIN -p6969:p6969 myimage

Impetus

First line fails with: "Exclusivity flag on, cannot modify" – Colleencollege

Answer 2

To apply tc policy on a docker host machine.

# find container pid
container_id=$(docker inspect some_container -f '{{.State.Pid}}')
mkdir -p /var/run/netns
# link network namespace for `some_container`
ln -sfT /proc/$container_id/ns/net /var/run/netns/some_container
# view the interface of the container
ip netns exec some_container ip -br -c link
# add traffic control policy to the interface in network namespace `some_container`
# tc -n some_container qdisc add dev eth0 tbf rate 1024kbps 1024b limit 1024b
# update tc 2023/05/19
tc -n some_container qdisc add dev eth0 handle 10: root tbf limit 1024 burst 2048 rate 1024

Ladner

The last line returns tbf: unknown parameter "1024b" so I tried tc -n some_container qdisc add dev eth0 tbf rate "10.0Mbit" burst "55000" latency "50ms" but this returns Error: Handle cannot be zero.. I really dislike tc, as its syntax is overcomplicated and on errors it returns only useless messages. :| – Cartercarteret

tc is powerful but not easy to use. but tc is tool for traffic control. another method is use iptables to do traffic control for network namespace. – Ladner

Last tc -n command gives me RTNETLINK answers: No such file or directory. Is eth0 the interface of the host or the container? – Animalist

all on host. root is required for operating network namespace. tc -n (run tc on network namespace). can check network namespace by ip netns ls – Ladner

Answer 3

You could use the iptables limits module. For example, you could add a rule to the PREROUTING table using the options "-m limit --limit 10/s" to limit a particular port to receive only 10 connections per second.

Pickard