Dockerfile: how to redirect the output of a RUN command to a variable?

Question

I am writing a dockerfile and want to put the output of the "ls" command into a variable as shown below:

$file = ls /tmp/dir

Here, "dir" only has one file inside it.

The following RUN instruction within a dockerfile is not working

RUN $file = ls /tmp/dir

Policeman

Answer 1

You cannot save a variable for later use in other Dockerfile commands (if that is your intention). This is because each RUN happens in a new shell.

However, if you just want to capture the output of ls you should be able to do it in one RUN compound command. For example:

RUN file="$(ls -1 /tmp/dir)" && echo $file

Or just using the subshell inline:

RUN echo $(ls -1 /tmp/dir)

If you have an actual error or problem to solve I could expand on this instead of a hypothetical answer.

A full example Dockerfile demonstrating this would be:

FROM alpine:3.7
RUN mkdir -p /tmp/dir && touch /tmp/dir/file1 /tmp//dir/file2
RUN file="$(ls -1 /tmp/dir)" && echo $file
RUN echo $(ls -1 /tmp/dir)

When building you should see steps 3 and 4 output the variable (which contains the list of file1 and file2 creating in step 2). The option for --progress plain forces the output to show the steps in later version of Docker:

$ docker build --no-cache --progress plain -t test .
Sending build context to Docker daemon  2.048kB
Step 1/4 : FROM alpine:3.7
 ---> 3fd9065eaf02
Step 2/4 : RUN mkdir -p /tmp/dir && touch /tmp/dir/file1 /tmp//dir/file2
 ---> Running in abb2fe683e82
Removing intermediate container abb2fe683e82
 ---> 2f6dfca9385c
Step 3/4 : RUN file="$(ls -1 /tmp/dir)" && echo $file
 ---> Running in 060a285e3d8a
file1 file2
Removing intermediate container 060a285e3d8a
 ---> 2e4cc2873b8c
Step 4/4 : RUN echo $(ls -1 /tmp/dir)
 ---> Running in 528fc5d6c721
file1 file2
Removing intermediate container 528fc5d6c721
 ---> 1be7c54e1f29
Successfully built 1be7c54e1f29
Successfully tagged test:latest

Ajaajaccio

Me neither: Step 10/20 : RUN file="$(conda list --explicit)" && echo $file ---> Using cache ---> 66f20d5489c0 Step 11/20 : RUN echo $(conda list --explicit) ---> Using cache ---> f923669d8444 ... – Human

Cached steps are not running the command. You'd need to break the cache in a previous step or run the build with --no-cache. – Ajaajaccio

For me I had to add --progress=plain to see ls output. I have Docker v20.10.5. – Chauchaucer

Note, --progress=plain is the option to the docker build command. That part wasn't clear to me. – Sax

Answer 2

Just highlight the answer given in the comments, which is probably the correct one if you are using a modern version of Docker (in my case v20.10.5) and the logs do not show the expected output, when, for example, you run RUN ls.

You should use the option --progress <string> in the docker build command:

 --progress string         Set type of progress output (auto, plain, tty). Use plain to show container output
                            (default "auto")

For example:

docker build --progress=plain .

In the latest versions of docker, the classic build engine that docker ships with has been upgraded to Buildkit, which displays different information.

You should see output like:

#12 [8/8] RUN ls -alh
#12 sha256:a8cf7b9a7b1f3dc25e3a97700d4cc3d3794862437a5fe2e39683ab229474746c
#12 0.174 total 184K
#12 0.174 drwxr-xr-x    1 root     root        4.0K Mar 28 19:37 .
#12 0.174 drwxr-xr-x    1 root     root        4.0K Mar 28 19:35 ..
#12 0.174 drwxr-xr-x  374 root     root       12.0K Mar 28 19:37 node_modules
#12 0.174 -rw-r--r--    1 root     root        1.1K Mar 28 19:36 package.json
#12 0.174 -rw-r--r--    1 root     root         614 Mar 28 15:48 server.js
#12 0.174 -rw-r--r--    1 root     root      149.5K Mar 28 16:54 yarn.lock
#12 DONE 0.2s

As noted by @venimus in the comments, you may also need --no-cache because cached containers do not show any output.

Related question with more details.

Hayfork

I might be misunderstanding but string is not a valid parameter or is this just a generic placeholder for the other options (e.g. auto, plain, tty)? – Copt

You are rigth, string is the placeholder, it should be replaced by a valid option – Hayfork

you may also need --no-cache because cached containers do not show any output – Businesslike

Answer 3

docker compose alternative to:

docker build --progress=plain .

is

BUILDKIT_PROGRESS=plain docker compose build

or in compose.yml file

services:
    build:
        progress: plain

Reiko

I have error with this option Additional property progress is not allowed – Flying

@Flying check compose file version #43566927 – Reiko

Answer 4

I couldn't get Andy's (or any other) approach to work in the Dockerfile itself, so I set my Dockerfile entrypoint to a bash file containing:

#!/bin/bash
file="$(conda list --explicit)" && echo $file
echo $(conda list --explicit)

Note the second method doesn't render line breaks, so I found the first method - echo via the $file variable - superior.

Human

Upvote for the sh file idea! Note, I get line breaks with either one on my ubunutu....but that's secondary issue. – Monnet

Answer 5

Another option is to save result in a temporary file and use it in a consequent RUN statement or even copy it to another step, with COPY --from command:

Here is an example:

FROM alpine:3.17.0
WORKDIR /output

# here we can store result of any command
# or even store multiple variables in a same file
RUN echo "2 + 3" >> /output/temp_file

# will print 5, almost anything can be retrieved from a
# text file in bash and be used in following commands
RUN echo $(( $(cat /output/temp_file) ))

CMD tail -f /dev/null

Erminiaerminie