In Dockerfile, COPY all contents of current directory except one directory

Question

In my Dockerfile, I have the following:

COPY . /var/task

...which copies my app code into the image.

I need to exclude the vendor/ directory when performing this copy.

• I cannot add vendor/ to .dockerignore, because that directory needs to be part of the image when it gets built within the image with a RUN composer install.
• I cannot specify every file and directory that should be copied, because they may change and I can't rely on other developers to keep the list updated.

I've tried the following, with the following errors:

COPY [^vendor$]* /var/task

When using COPY with more than one source file, the destination must be a directory and end with a /

COPY [^vendor$]*/ /var/task

COPY failed: no source files were specified

Section

I believe that the first error you had was related with /var/task since that is your destination. Did you try COPY [^vendor$]* /var/task/? – Tamra

I am confused by "I cannot add vendor/ to .dockerignore...". If you're excluding the directory from COPY, as you're asking here, it's not going to be available within the image so it doesn't matter whether or not it's listed in the .dockerignore file. – Jampan

.dockerignore only affects whether COPY can see a given directory; it won't prevent that directory from being in the final image. – Cutinize

@NPinheiro That command worked without error (I misread the initial warning, adding / to the src, not dest). But the copy resulted in a strange result, with contents of subdirectories ending up in the top-level directory, and overwrites where subdirectories and files has matching names. – Section

@Jampan vendor is created within the image using RUN before the app code is copied in from the host. This is to make sure that, if the app code already has a vendor directory, it won't overwrite the one created in the image (which is happening when I use an existing workspace on Jenkins) – Section

@DavidMaze Thank you! You're right. This was the solution. – Section

Answer 1

It is actually enough to add the vendor directory to the .dockerignore file.

You can broadly follow the flow of files through docker build in three phases:

1. docker build reads files from the directory you name, ignoring things in the .dockerignore file, and sends them to the Docker daemon as the build context.
2. The COPY instruction copies files from the build context into the container filesystem.
3. RUN instructions do further transformation or processing.

If you put vendor in the .dockerignore file, it prevents the directory from being included in the build context. The build will go somewhat faster, and COPY won't have the files to copy into the image. It won't prevent a RUN composer install step later on from creating its own vendor directory in the image.

Cutinize

(How) can I add a new .dockerfile halfway through the build, with or without a multistage dockerfile? – Parthenopaeus

Answer 2

I don't think there is an easy solution to this problem.

If you need vendor for RUN composer install and you're not using a multistage build then it doesn't matter if you remove the vendor folder in the copy command. If you've copied it into the build earlier then it's going to be present in your final image, even if you don't copy it over in your COPY step.

One way to get around this is with multi-stage builds, like so:

FROM debian as base
COPY . /var/task/
RUN rm -rf /var/task/vendor
FROM debian
COPY --from=base /var/task /var/task

If you can use this pattern in your larger build file then the final image will contain all the files in your working directory except vendor.

There's still a performance hit though. You're still going to have to copy the entire vendor directory into the build, and depending on what docker features you're using that will still take a long time. But if you need it for composer install then there's really no way around this.

Pustule

Answer 3

buildkit now has an experimental COPY --excludes option. That looks like:

# syntax=docker.io/docker/dockerfile:1.7-labs
# Enable the experimental features with the above syntax line

FROM your_base
COPY --excludes=vendor . /var/task
# ...

Documentation for the feature can be found at https://github.com/moby/buildkit/blob/v0.15.2/frontend/dockerfile/docs/reference.md#copy---exclude

Another option is to leverage RUN --mount=type=bind to mount the context and then copy the files with a normal copy command:

FROM debian
RUN --mount=type=bind,target=/context \
 cp -r /context /var/task && rm -r /var/task/vendor

There are probably better ways to write the above cp command, but they may depend on your shell, and the container typically defaults to /bin/sh.

Simonton