how do I get etcd values into my systemd service on coreOS?

Question

I have two services A and B.

A sets a value in etcd as it's being started, say the public IP address which it gets from an environment file:

ExecStartPost=/usr/bin/etcdctl set /A_ADDR $COREOS_PUBLIC_IPV4

B needs that value as it starts up, as well as its own IP address. So something like this would be nice:

ExecStart=/usr/bin/docker run -e MY_ADDR=$COREOS_PUBLIC_IPV4 -e A_ADDR=$ETCD_A_ADDR mikedewar/B

but that's obviously not possible as etcd variables don't present as systemd environment variables like that. Instead I can do some sort of /usr/bin/bash -c 'run stuff' in my ExecStart but it's awkward especially as I need systemd to expand $COREOS_PUBLIC_IPV4 and my new bash shell to expand $(etcdctl get /A_ADDR). It also reeks of code smell and makes me think I'm missing something important.

Can someone tell me the "right" way of getting values from etcd into my ExecStart declaration?

-- update

So I'm up and running with

ExecStart=/usr/bin/bash -c 'source /etc/environment && /usr/bin/docker run -e A_ADDR=$(/usr/bin/etcdctl get /A_ADDR) -e MY_ADDR=$COREOS_PUBLIC_IPV4 mikedewar/B'

but it's pretty ugly. Still can't believe I'm not missing something..

Intervenient

This works somewhat - but when your docker containers fall over the systemd service sometimes doesn't realize it. – Roaring

Answer 1

I've was struggling with the same thing until recently. After reading much of the documentation of CoreOS and systemd, here is a slightly 'cleaner' version of what you're doing:

[Service]
EnvironmentFile=/etc/environment
ExecStart=/bin/sh -c '/usr/bin/docker run -e A_ADDR=$(/usr/bin/etcdctl get /A_ADDR) -e MY_ADDR=$COREOS_PUBLIC_IPV4 mikedewar/B'

Additionally, I have adopted a pattern where my services depend on a systemd 'oneshot' service that will compute some value and write it in to /etc/environment. This allows you to keep more complex shell scripting out of the main service unit and place it into it's own oneshot service unit.

Here are the docs for EnvironmentFile: http://www.freedesktop.org/software/systemd/man/systemd.exec.html#EnvironmentFile=

Finally, a quick gotchya: you must use a shell invocation if you use any variable in your ExecStart/Stop commands. systemd does no shell invocation when executing the command you provide, so variables will not be expanded.

Gynophore

Check PassEnvironment= - that helped me to pass env variables in docker systemd setup. – Jablonski

Answer 2

I am currently using such a workaround:

I've created scripts which extracts data from particular etcd directory

#! /bin/sh
for entry in `etcdctl ls /my_dir --recursive`  ; do
  echo ' -e '`grep -o '[^/]*$'  <<< ${entry}`=`etcdctl get ${entry}`
done

its output looks following:

 -e DATABASE_URL=postgres://m:[email protected]:5432/m
 -e WEB_CONCURRENCY=4

So then eventually I can in my init file place that in such way

/bin/sh -c '/usr/bin/docker run -p 9000:9000 $(/home/core/envs.sh) me/myapp -D FOREGROUND'

It's not the most elegant way, and I'd love to know how to improve it, but placing that for loop as a one-liner requires lots of escaping.

Fantast

Answer 3

Can you container read directly from etcd as it starts, over the docker0 bridge IP, instead of passing in the values? This will also allow you to do more complex logic on the response, parse JSON if you are storing it as the etcd value, etc.

Urdu

i had no idea one could read etcd from within the container. That said, I'm running a 3rd party binary as the container's entrypoint so short of reading etcd from within the Dockerfile (which seems a little scary) I'm not sure this solves the problem... – Intervenient