Automatic self-configuration of an etcd cluster as a Docker swarm service

Question

I want to find a way to deploy an etcd cluster as a Docker Swarm service that would automatically configure itself without any interaction. Basically, I think of something in spirit of this command:

docker service create --name etcd --replicas 3 my-custom-image/etcd

I'm assuming that overlay network is configured to be secure and provide both encryption and authentication, so I believe I don't need TLS, not even --auto-tls. Don't want an extra headache finding a way to provision the certificates, when this can be solved on the another layer.

I need an unique --name for each instance, but I can get that from an entrypoint script that would use export ETCD_NAME=$(hostname --short).

The problem is, I'm stuck on initial configuration. Based on the clustering guide there are three options, but none seems to fit:

• The DNS discovery scenario is closest to what I'm looking for, but Docker doesn't support DNS SRV records discovery at the moment. I can lookup etcd and I will get all the IPs of my nodes' containers, but there are no _etcd-server._tcp records.
• I cannot automatically build ETCD_INITIAL_CLUSTER because while I know the IPs, I don't know the names of the other nodes and I'm not aware about any way to figure those out. (I'm not going to expose Docker API socket to etcd container for this.)
• There is no preexisting etcd cluster, and while supplying the initial configuration URI from discovery.etcd.io is a possible workaround I'm interested in not doing this. I'm aiming for "just deploy a stack from this docker-compose.yml and it'll automatically do the right thing, no questions asked" no-brainer scenario.

Is there any trick I can pull?

Shockley

Answer 1

As you have correctly said you know the IPs of your nodes’ containers, so the suggested trick is to simply build the required etcd names as derivatives of each node’s IP address.

• inside each container etcd is named using this particular container's IP i.e. etcd-$ip
• ETCD_INITIAL_CLUSTER is populated using other containers' IPs in a similar way

The names could be as simple as etcd-$ip or even better i.e. we could use the netmask to calculate the node’s IP on this network to make the names prettier.

In this case in a simple 3-nodes configuration one could end up having names like etcd-02 etcd-03 etc

No specific requirements exist for the name attribute, it just needs to be unique and human-readable. Although it indeed looks like a trick it might work

Revisionism

Don't the IP addresses change on service container updates? It would be odd if container is removed, started on a same node (so it still has the same data volume) yet believes it's something else than what it was before. I thought the point of names is that even though network identifiers may change, names still persist so nodes know their peers are still the same. – Shockley

Answer 2

I was also looking for a simple solution to a docker swarm etcd cluster and found this repo.

The repository contains a run.sh bash script that does the "magic" of building the advertise-client-urls parameter and the initial cluster by playing "ping pong" with the nodes.

Expostulation