Why do I need to be in Swarm mode to use Docker secrets?

Question

I am playing around with a single container docker image. I would like to store my db password as a secret without using compose (having probs with that and Gradle for now). I thought I could still use secrets even without compose but when I try I get...

$ echo "helloSecret" | docker secret create helloS -

Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.

Why do I need to use swarm mode just to use secrets? Why can't I use them without a cluster?

Iggie

They're stored in the raft log (which is Swarm specific). You don't need to use compose though (can create everything through the cli). No real downside just to using a single node Swarm. – Country

@Country It just seems counter intuitive, but I am pretty new to docker. I realize that it probably doesn't "hurt" to have a one node swarm I just figured it would add at least a little overhead and I wasn't sure why that was ness for a secret. I will take a look at the raft log though and maybe that will shed some light. Thanks! – Iggie

but why do you need to use secrets? is it because you what the same code to work in production and development? – Gath

Eventually yes and I want to make it so I don't have to store EnvVars in the docker file. – Iggie

Answer 1

You need to run swarm mode for secrets because that's how docker implemented secrets. The value of secrets is that workers never write the secret to disk, the secret is on a need-to-know basis (other workers do not receive the secret until a task is scheduled there), and on managers encrypt that secret on disk. The storage of the secret on the manager uses the raft database.

You can easily deploy a single node swarm cluster with the command docker swarm init. From there, docker-compose up gets changed to docker stack deploy -c docker-compose.yml $stack_name.

---

Secrets and configs in swarm mode provide a replacement for mounting single file volumes into containers for configuration. So without swarm mode on a single node, you can always make the following definition:

version: '2'
services:
  app:
    image: myapp:latest
    volumes:
    - ./secrets:/run/secrets:ro

---

Or you can separate the secrets from your app slightly by loading those secrets into a named volume. For that, you could do something like:

tar -cC ./secrets . | docker run -i -v secrets:/secrets busybox tar -xC /secrets

And then mount that named volume:

version: '2'
volumes:
  secrets:
    external: true
services:
  app:
    image: myapp:latest
    volumes:
    - secrets:/run/secrets:ro

Myrmecophagous

Little late to the convo. In case I would like to store the secretes into an independent volume as you suggests, how do I distribute that volume over standalone containers over different machines? I mean, it is not that anyone who can mount the volume into no matter what container could read the data? Or I missing something here? – Huerta

@JoséPulido How would you distribute the compose file to different machines? – Myrmecophagous

I don't understand your question. I've an image with my production code, and I need to run a standalone container per machine (little shields like Raspberry), but I would like to find a way to inject the secrets on runtime. Your approach suggests to pack up that secrets into a volume so it can be mounted on runtime through Compose or just Docker, so my question is how do I share the volume over differentes machines. Thanks in advance for your quick response, I appreciate it. – Huerta

@JoséPulido If you aren't using Swarm Mode, then you would need to connect to each machine, push a compose file to that machine, and run docker compose up, to start the container. The same way you are able to connect to those machines and run your deploys would be my advice to distribute your secrets. It's outside of docker at that point. – Myrmecophagous

The underlying question is probably better answered over here: https://mcmap.net/q/108800/-what-is-the-best-way-to-pass-aws-credentials-to-a-docker-container – Myrmecophagous

Answer 2

Check out this answer: https://serverfault.com/a/936262 as provided by user sel-en-ium :-

You can use secrets if you use a compose file. (You don't need to run a swarm).

You use a compose file with docker-compose: there is documentation for "secrets" in a docker-compose.yml file.

I switched to docker-compose because I wanted to use secrets. I am happy I did, it seems much more clean. Each service maps to a container. And if you ever want to switch to running a swarm instead, you are basically already there.

Unfortunately the secrets are not loaded into the container's environment, they are mounted to /run/secrets/

Asdic

Can you show a working example of docker-compose using a secret? The documentation and even the OP's error message all indicate that the secret will not be configured in the container. – Myrmecophagous

For a working example, the steps in the following answer build on above answer: https://mcmap.net/q/181571/-how-do-you-manage-secret-values-with-docker-compose-v3-1 – Asdic

That example does not use docker-compose, it deploys the secret with swarm mode (see the docker swarm init and docker stack deploy commands). – Myrmecophagous

Perhaps the confusion is that a compose file is not the same as the docker-compose command. Swarm mode uses the same yaml compose file (with the version 3 syntax), but it deploys the containers very differently. – Myrmecophagous

+1 upvote for distinguishing between compose and docker-compose.yml. On a separate note, I would guess use of dockers secret is an advantage of using swarm, on the basis that it is part of it, along with perhaps it being OK for smaller scale setups. bretfisher.com/is-swarm-dead-answered-by-a-docker-captain – Asdic

Swarm gives you orchestration, fault tolerance, multi node, overlay networking, a routing mesh, and secrets (off the top of my head). Swarm will scale nicely, the big difference with k8s is configurability and extensibility. Swarm mode is a one size fits many solution, really nice if it is your size. – Myrmecophagous

+1 upvote thank you BMitch that is good to know. Valueable point amongst what seems to be a bit of hype elsewhere about the lifespan of Swarm. As The Eagles once sung, "Everybody's talking about the new kid in town", that new kid being Kubernetes. Nevertheless, from what I've read Kubernetes seems to have earnt its hype so to speak, but to the detriment of swarm - as from what you say - is still a player. – Asdic

@Myrmecophagous see forums.docker.com/t/… for a working example of docker-compose using a secret. would be very curious to know what you think. – Lanellelanette

@Lanellelanette and BMitch - check out this question as well from me: devops.stackexchange.com/questions/10570/… - could lando.dev be a good starter production platform before moving onto swarm and then kubernetes ? – Asdic

thanks but I don't about about lando.dev and am not qualified to answer. – Lanellelanette

I believe this answer should be removed. Although you can use secrets with compose, that's not answering the OP's question. They are trying not to use compose. – Stack