Dockerfile COPY from image to host

Question

I have a Dockerfile in which I first compile the Mosquitto server in one layer, then use COPY to copy the source files of an authentication plug-in into the image and finally RUN the compilation of that plug-in.

All in all, the resulting image is good to be used in a container which then has the Mosquitto server running with that plug-in loaded.

I want to modify this plug-in and recompile it by re-running the build of the Dockerfile. Since the first layer is unmodified, it just copies the modified files and runs the compilation again.

What I want to do now is to extract the plug-in (.so file) from that new image and move it to a mounted directory on the host, so that the currently running Mosquitto server would only need to be restarted.

Is it possible to use the COPY command in reverse, so that it copies the compiled plug-in to a specified host directory so that I can then delete the newly created image?

Or is this a bad approach altogether? Should I better exec into the running container and have it rebuild the plug-in (which would limit me to building the plug-in on the machine on which the server is running)?

Dahlberg

Why wouldn't you use a volume? – Smitten

@Smitten Could you elaborate a bit more on what kind of volume you are referring to and where to use it? The server is running on a development machine so no reason to use any other volumes than mounts from the host, this eases the development process of the scripts the plug-in uses. This development machine is a server (not a workstation) and the Mosquitto server is supposed to run for hours/days unless the plug-in needs to be reloaded (restart of Mosquitto is around a second of downtime) – Dahlberg

Possible duplicate of Copying files from Docker container to host – Aflutter

Answer 1

I do not know the details of the sepcific compiler tools you are using, but I think I get what you are trying to achieve:

I would not include the COPY command in the Dockerfile. The Dockerfile must only contain the necessary instructions to have an image with the necessary tools and dependencies to carry out the compilation process, and maybe a shell script with the specific compiling orders.

Now you run docker build and you have your image, let's call it mosq. Let's assume that:

• You have your source code in your local machine in /home/me/my-source-code
• Once complied, you have the result inside a subfolder dist of that folder: /home/me/my-source-code/dist/result.so
• Your image has a script /compile.sh that compiles the source code present in /compilation (that folder should be empty in the image)

Then, you run the image mounting volume param: /home/me/my-source-code onto /compilation inside the container

Assuming all the previous points, the docker run command should look something similar to:

docker run -d --name my-compiler -v /home/me/my-source-code:/source mosq /compile.sh

Et voila, the container will run silently and die, and after that you'll have your compilation in /home/me/my-source-code/dist/result.so

The specifics might vary a lot depending on the details, but I hope you get the idea: prepare everything in your image so that executing a single sh script, the compiler takes the code from somewhere and runs. Mount a volume with the code in that folder. If the compiler outputs the result somewhere else, mount another volume from your host machine to get the result there.

Smitten

Thanks for your effort. I think I've accepted the fact that will need to create a container to get the .so file, yet using docker cp seems then to be a better idea, specially in conjunction with docker create instead of docker run since there is no need to execute anything. – Dahlberg

Answer 2

COPY is probably not the right tool for what you are trying to achieve.

Either use a runtime volume, like @gmc suggests or copy it on the host using docker cp.

Usage

docker cp CONTAINER:SRC_PATH DEST_PATH

However, I'm not sure that is the right approach in general. It doesn't sound like Docker is the tool you need for what you are trying to achieve. If you want a mutable server instance, there are better options.

Skindive

hey @Skindive i am working on something where is need to change parts of server during runtime, you said If you want a mutable server instance, there are better options. can you please tell me what options would those be ? – Monometallic

Ansible, Chef, Puppet? Although using Docker just to build/compile artifacts is not a bad idea. But it would be part of a larger orchestration that might be managed by something like Ansible. – Skindive

Answer 3

Copying files from a docker image vs. a running container are different things. The solutions posted by others copy files from a container. To copy a directory from an image, you can use tar:

docker run --rm  --entrypoint tar  MY_IMAGE_NAME czf - /path/to/mydir/inside/the/image > mydir.tar.gz
tar xzf mydir.tar.gz -C /host/path/to/mydir

Kennel