Terminate docker compose when test container finishes

Question

I am currently running a docker-compose stack for basic integration tests with a protractor test runner, a nodejs server serving a web page and a wildfly server serving a java backend.

The stack is run from a dind(docker in docker) container in my build server(concourse ci).

But it appears that the containers does not terminate on finishing the protractor tests.

So since the containers for wildfly, and nodejs are still running the build task never finishes...

How can I make the compose end in success or failure when the tests are finished?

# Test runner
test-runner:
  image: "${RUNNER_IMG}"
  privileged: true
  links:
    - client
    - server
  volumes:
  - /Users/me/frontend_test/client-devops:/protractor/project
  - /dev/shm:/dev/shm
  entrypoint:
    - /entrypoint.sh
    - --baseUrl=http://client:9000/dist/
    - /protractor/conf-dev.js
    - --suite=remember
# Client deployment
client:
  image: "${CLIENT_IMG}"
  links:
    - server
# Server deployment
server:
  image: "${SERVER_IMG}"

Whitehead

Answer 1

Compose has added the --exit-code-from {container} flag to docker-compose up which makes this easier.

docker-compose up --exit-code-from test-runner

See Michael Spector's answer for more detail.

---

Original Answer

Similar to this rspec q/a, you need to run the tests as a standalone task that report an exit status back to your CI.

You could separate the test-runner into it's own yaml or modify the test-runner to default to a no op command/entrypoint.

Separate the test runner

Specify the test-runner config separately (You might need to upgrade to version 2 networks instead of using links to work across multiple compose files).

docker-compose up -d
docker-compose -f test-runner.yml run test-runner
rc=$?
docker-compose down
exit $rc

No op test runner

Default the test-runner to a no op entrypoint/command and then manually run the test command

services:
  test-runner:
    image: "${RUNNER_IMG}"
    command: 'true'

Then

docker-compose up -d
docker-compose run test-runner /launch-tests.sh
rc=$?
docker-compose down
exit $rc

Return codes

If your CI has the concept of "post tasks" you might be able to skip the rc capturing and just run the docker-compose down after the test-runner CI task has completed. It's also possible your CI cleans up the containers for you.

Defect

Is it possible to write something like --exit-code-from test-runner into the docker-compose file? – Marzipan

@SerhiiKushchenko There's nothing like exit-code-from in the spec – Defect

Answer 2

You can use these docker-compose parameters to achieve that:

--abort-on-container-exit Stops all containers if any container was stopped.

--exit-code-from Return the exit code of the selected service container.

For example, having this docker-compose.yml:

version: '2.3'

services:
  elasticsearch:
    ...
  service-api:
    ...
  service-test:
    ...
    depends_on:
      - elasticsearch
      - service-api

The following command ensures that elasticsearch and service-api go down after service-test is finished, and returns the exit code from the service-test container:

docker-compose -f docker-compose.yml up \
    --abort-on-container-exit \
    --exit-code-from service-test

Anywhere

Thanks you so much I read half of the internet to find your solution :-) – Carotid

agreed, that was surprisingly hard to search for – Beefwitted

Indeed. Starred and upvoted to make sure it's not lost again. – Alumnus

very ... trivial answer. I hope this answer grows to the top so everyone doesn't lose it while searching. – Haustellum

best answer i ever got for any question on stackoverflow !! – Shelby

Nice answer! btw, you can omit --abort-on-container-exit docs.docker.com/compose/reference/up – Manille

Answer 3

The solution I found to be most elegant is to use depends_on in your docker-compose.yml file.

services:
  dynamodb:
  ...
  test_runner:
    ...
    depends_on:
      - dynamodb

Now you can use docker-compose run --rm test_runner which will set up your dependencies, run your tests, tear down everything, and propagate the return code.

docker-compose run test_runner false
Starting test_dynamodb_1 ... done
echo $?
1
docker-compose run test_runner true
Starting test_dynamodb_1 ... done
echo $?

Upandcoming

Not sure why this isn't upvoted more - it seems to me that it solves the question much better than the accepted answer. – Allow

Note that this does not tear down dependencies, as opposed to https://mcmap.net/q/271961/-terminate-docker-compose-when-test-container-finishes – Alumnus

Answer 4

I have tried the solution offered in this discussion but problem still in the case

Case-1: docker-compose up -d

You can use docker-compose up -d, test it by a test-runner then terminate by docker-compose down but the problem when you use docker-compose up -d is that you won't see the logs of the standard output anymore.

Case-2: docker-compose up --exit-code-from a service

You can view the docker logs if you use --exit-code-from <service> that implies --abort-on-container-exit but the service won't send exit command when it is succeeded. Then you need to catch your container is finish tested before sending docker-compose down to stop them.

One of the solution to see the logs before terminating them is using --tail=1000 -f

docker-compose up -d
docker-compose logs --tail=1000 -f test-runner
docker-compose down
exit $rc

There is also a solution using nohup but you need to wait until process is completely finished then open the output logs file, so the above should be easier

Silvan

Answer 5

My docker compose up had some combination of flags (--build) that prevented all the service containers from being stopped when --abort-on-container-exit triggered.

e.g. the following:

docker compose up \
  --build service1 \
  --attach service1 --attach service2 \
  --abort-on-container-exit

will start both service1 and service2, but will only stop service1 when it aborts because as far as Docker is concerned it was only responsible for building service1 and it refuses to assume that service2 isn't pre-existing — even though it's, y'know, defined in the same docker-compose.yml and linked with depends_on and it literally just started it. It just leaves service2 running.¹

To fix this all I needed to do was make sure to run docker compose down after docker compose up.

This can be done using a trap to ensure that it always runs even if docker compose up fails and would exit the script early:

#!/usr/bin/env sh

cleanup() {
  docker compose down
}
trap cleanup EXIT  # will always run when the script exits

docker compose up \
  --build service1 \
  --attach service1 --attach service2 \
  --abort-on-container-exit

---

Footnotes

1: This was a problem for me because my service1 expected its service2 to be in a fresh state.

Boland

Answer 6

To avoid having separate config files you can update the docker-compose configuration to introduce the dependencies between services with the depends_on option, available from version 2 format and up. As result start of the test-runner will initiate the run of the clients.

Please note that if you need to wait some time when the actual web server will be started from inside services you are testing, you can use wait-for-it.sh script to wait until the server became available.

# Test runner
test-runner:
  image: "${RUNNER_IMG}"
  privileged: true
  links:
    - client
    - server
  volumes:
  - /Users/me/frontend_test/client-devops:/protractor/project
  - /dev/shm:/dev/shm
  depends_on:
    - client
  entrypoint:
    - wait-for-it.sh
    - client
    - -t
    - '180'
    - --
    - /entrypoint.sh
    - --baseUrl=http://client:9000/dist/
    - /protractor/conf-dev.js
    - --suite=remember
# Client deployment
client:
  image: "${CLIENT_IMG}"
  depends_on:
    - server
  links:
    - server
# Server deployment
server:
  image: "${SERVER_IMG}"

After updating config, simple docker-compose up test-runner will trigger start of the related services.

Conquian

Answer 7

You can do clean-up tasks using ensure on a task step in Concourse. https://concourse-ci.org/ensure-step.html

In your case, you could add an ensure block after your protractor tests, and run a task to tear-down what was docker-compose'd earlier. You could also use an on-success step https://concourse-ci.org/on-success-step.html to do the teardown, and the docker-compose'd containers would be kept if your tests fail.

Korella