Kubernetes overlay network comparison

Question

I was wondering if someone could give me a brief overview of the differences/ advantages between all of the different Kubernetes network overlays. The getting started guide (http://kubernetes.io/docs/getting-started-guides/scratch/#network) mentions the following:

• Flannel
• Calico
• Weave
• Romana
• Open vSwitch (OVS)

But doesn't really explain what the differences between them are or what the advantages and disadvantages each one has. I was wondering if someone could give me an idea of which one of these solutions I should be using for a bare metal CentOS 7 cluster.

Thanks!

Hardman

The purpose of each is the same (route pod traffic) but the implementation is different. The answer for this would be complex because each overlay has many different settings e.g. flannel can be run with a backend supported by AWS, UDP tunnel, VXLAN, host gateway. Each has it's own pros/cons depending on your needs. A simpler approach for getting started on bare metal I suggest you manually create routes on your hosts and don't worry about overlay until you need it. Here's how I do it with static routes on bare metal medium.com/@rothgar/no-sdn-kubernetes-5a0cb32070dd#.r49reax8k – Illegitimacy

Some implementation will also depend on what base OS you are using and how you are configuring/deploying the OS + Kubernetes. I would suggest starting with kubeadm kubernetes.io/docs/getting-started-guides/kubeadm which supports weave (default) and flannel (config flag). If that works for you then stay with it. If you have problems on your network or need security/performance/etc. then look at the other options. – Illegitimacy

Answer 1

This comparison matrix was shared several times on Kubernetes' Slack and may be useful.

However, beware potentially out-of-date information, keep in mind the "devil is in the details" so the reality may not be as simple as it would seem according to this document. All available solutions will have pros and cons, but will also be more suitable for some use-cases than others, so as always, it is a question of trade-offs and YMMV.

Fowler