Docker image push over SSH (distributed)

Question

TL;DR Basically, I am looking for this:

docker push myimage ssh://myvps01.vpsprovider.net/

I am failing to grasp the rationale behind whole Docker Hub / Registry thing. I know I can run a private registry, but for that I have to set up the infrastructure of actually running a server.

I took a sneak peek inside the inner workings of Docker (well, the filesystem at least), and it looks like Docker image layers are just a bunch of tarballs, more or less, with some elaborate file naming. I naïvely think it would not be impossible to whip up a simple Python script to do distributed push/pull, but of course I did not try, so that is why I am asking this question.

Are there any technical reasons why Docker could not just do distributed (server-less) push/pull, like Git or Mercurial?

I think this would be a tremendous help, since I could just push the images that I built on my laptop right onto the app servers, instead of first pushing to a repo server somewhere and then pulling from the app servers. Or maybe I have just misunderstood the concept and the Registry is a really essential feature that I absolutely need?

EDIT Some context that hopefully explains why I want this, consider the following scenario:

• Development, testing done on my laptop (OSX, running Docker machine, using docker-compose for defining services and dependencies)
• Deploy to a live environment by means of a script (self-written, bash, few dependencies on dev machine, basically just Docker machine)
• Deploy to a new VPS with very few dependencies except SSH access and Docker daemon.
• No "permanent" services running anywhere, i.e. I specifically don't want to host a permanently running registry (especially not accessible to all the VPS instances, though that could probably be solved with some clever SSH tunneling)

The current best solution is to use Docker machine to point to the VPS server and rebuild it, but it slows down deployment as I have to build the container from source each time.

Oestrone

has an answer here: #23935641 – Collegiate

@Collegiate It does not answer the original question, as was discussed in the answers here. This question is 8 years old. These days, a multitude of open source solution would solve this. I'll see if I find some time to write up a more up-to-date answer. – Oestrone

Answer 1

If you want to push docker images to a given host, there is already everything in Docker to allow this. The following example shows how to push a docker image through ssh:

docker save <my_image> | ssh -C [email protected] docker load

• docker save will produce a tar archive of one of your docker images (including its layers)
• -C is for ssh to compress the data stream
• docker load creates a docker image from a tar archive

---

Note that the combination of a docker registry + docker pull command has the advantage of only downloading missing layers. So if you frequently update a docker image (adding new layers, or modifying a few last layers) then the docker pull command would generate less network traffic than pushing complete docker images through ssh.

Hooten

I know about load and save, but the whole point would be to be smart and pushing/pulling only the layers that differ. – Oestrone

docker save by itself outputs "Cowardly refusing to save to a terminal" xD – Spermatogonium

/root/.bashrc: line 13: bind: warning: line editing not enabled I got this error when using the command. – Picard

Thanks, that worked for me. If I can add, on my system I had to call docker as root so I added "sudo" in front. Otherwise I would get errors "permission denied while trying to connect to the Docker". – Photoreceptor

Answer 2

I made a command line utility just for this scenario.

It sets up a temporary private docker registry on the server, establishes an SSH Tunnel from your localhost, pushes your image, then cleans up after itself.

The benefit of this approach over docker save is that only the new layers are pushed to the server, resulting in a quicker upload.

Oftentimes using an intermediate registry like dockerhub is undesirable, and cumbersome.

https://github.com/brthor/docker-push-ssh

Install:

pip install docker-push-ssh

Example:

docker-push-ssh -i ~/my_ssh_key [email protected] my-docker-image

Biggest caveat is that you have to manually add your local ip to docker's insecure_registries config.

https://mcmap.net/q/41060/-where-should-i-set-the-39-insecure-registry-39-flag-on-mac-os

Fever

Answer 3

Expanding on the idea of @brthornbury.

I did not want to dabble with running python, so I came up with bash script for the same.

#!/usr/bin/env bash
SOCKET_NAME=my-tunnel-socket
REMOTE_USER=user
REMOTE_HOST=my.remote.host.com

# open ssh tunnel to remote-host, with a socket name so that we can close it later
ssh -M -S $SOCKET_NAME -fnNT -L 5000:$REMOTE_HOST:5000 $REMOTE_USER@$REMOTE_HOST

if [ $? -eq 0 ]; then
  echo "SSH tunnel established, we can push image"
  # push the image to remote host via tunnel
  docker push localhost:5000/image:latest
fi
# close the ssh tunnel using the socket name
ssh -S $SOCKET_NAME -O exit $REMOTE_USER@$REMOTE_HOST

Roshan

This still requires a registry to be running on the remote though, correct? – Nicias

Yes, thats right @Nicias – Roshan

Answer 4

Saving/loading an image on to a Docker host and pushing to a registry (private or Hub) are two different things.

The former @Thomasleveil has already addressed.

The latter actually does have the "smarts" to only push required layers.

You can easily test this yourself with a private registry and a couple of derived images.

If we have two images and one is derived from the other, then doing:

docker tag baseimage myregistry:5000/baseimage
docker push myregistry:5000/baseimage

will push all layers that aren't already found in the registry. However, when you then push the derived image next:

docker tag derivedimage myregistry:5000/derivedimage
docker push myregistry:5000/derivedimage

you may noticed that only a single layer gets pushed - provided your Dockerfile was built such that it only required one layer (e.g. chaining of RUN parameters, as per Dockerfile Best Practises).

On your Docker host, you can also run a Dockerised private registry.

See Containerized Docker registry

To the best of my knowledge and as of the time of writing this, the registry push/pull/query mechanism does not support SSH, but only HTTP/HTTPS. That's unlike Git and friends.

See Insecure Registry on how to run a private registry through HTTP, especially be aware that you need to change the Docker engine options and restart it:

Open the /etc/default/docker file or /etc/sysconfig/docker for editing.

Depending on your operating system, your Engine daemon start options.

Edit (or add) the DOCKER_OPTS line and add the --insecure-registry flag.

This flag takes the URL of your registry, for example.

DOCKER_OPTS="--insecure-registry myregistrydomain.com:5000"

Close and save the configuration file.

Restart your Docker daemon

You will also find instruction to use self-signed certificates, allowing you to use HTTPS.

Using self-signed certificates

[...]

This is more secure than the insecure registry solution. You must configure every docker daemon that wants to access your registry

Generate your own certificate:

mkdir -p certs && openssl req \ -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \ -x509 -days 365 -out certs/domain.crt

Be sure to use the name myregistrydomain.com as a CN.

Use the result to start your registry with TLS enabled

Instruct every docker daemon to trust that certificate.

This is done by copying the domain.crt file to /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt.

Don’t forget to restart the Engine daemon.

Verdha

Thank your for the detailed explanation. I updated my question with some context as to what I am trying to accomplish. Basically, I am fine with running the Docker engine itself, and I want to host a registry pointing to my dev machine's Docker installation, served over the SSH connection I already use. I realize this is probably not possible currently, which means your answer is likely the acceptable one. – Oestrone

Ah, I understand. I will add information to my answer, stating that it is my current understanding that SSH is not an option for (private) registries. Barring some bizarre SSH tunneling scenario. – Verdha

One additional point re: your Edits: some of what you are aiming towards leans strongly towards Orchestration and CI/CD. Platforms like Bamboo or Jenkins are able to do this with Docker, but as I'm still learning and evaluating those options myself, I won't put that in my answer as I feel it wouldn't be definite. – Verdha

While there are certainly good orchestration solutions out there for the larger projects, my question should be read in the context of small side projects managed from a single laptop without access to / resources for infrastructure servers like CI servers. – Oestrone

Understood and for much of my own work I also don't use some massive CI/CD tool (yet ;). My suggestions for a solution I use myself, nearly every day, hopefully you find them useful. – Verdha