My company runs a kubernetes setup with 6 nodes and traefik 1.6 deployed as a DaemonSet to each node in production. Each traefik instance takes incoming (ssl terminated) requests on port 80 from an AWS ALB and forwards them to the individual pods.
After a pretty heavy DDoS attack on our backend we enabled traefiks rate limiting feature to prevent the requests getting through to the actual pods.
Since we run multiple instances of traefik that are unaware of each other, the rate limits perceived by the user are between 1x - 6x higher then the ones that are set in the traefik config since requests hit one out of 6 traefik instances randomly. This makes it impossible to set precise limits and we cant rely on the AWS ALB sticky session setting to ensure that clients hit the same box every time, as attackers could just omit the cookie.
Using a single traefik instance with a deployment instead of a daemonset is not a great solution. It would require an additional jump between the nodes for each request to get from the node accepting the request to the traefik instance. In 5 out of 6 requests that instance would run on a different node.
Is there a way to share rate limiting state between the traefik daemons on each node?