How to dynamically change the docker-compose image field

Question

I have a docker-compose.yml something like bellow:

networks:
  smstake: 
    ipam:
      config:
        - subnet: 10.0.10.0/24
services:
    app:
        
        image: smstake:latest
        ports:
          - 8000:80
        networks:
          - smstake

        depends_on:
          - db
        deploy:
          mode: replicated
          replicas: 1
          placement:
            constraints:
              - node.role == manager

I am using it for deploying the service in nodes running in swarm mode.

Every time an image is build, the image name may differ based on user passed branch name or tagname which works as tag for the image. I am running it from the jenkins. For eg: smstake:

How can I dynamically add the image name to the image parameter of the service. As docker stack does not support build. I cannot even use it. I am not able to figure out the right way to do it.

I am trying to deploy with docker stack deploy -c docker-compose.yml stackname

My exact Requirement being:

1. Have a build job in jenkins which builds the image for us.
2. The image name differs or changes if the tag or branch name changes
3. We have a build job to deploy the jobs again with the newly created image.

The reason behind creating new image for new TAG is so that I can rollback to previously build image.

Some edit: Added the image-name to add in configuration.env file which will be passed using echo command in deploy job before deploy command runs. than the docker-compose will look like following

version: '3.4'
networks:
  smstake: 

services:

    db:
        image: mysql:5.7
        networks:
          - smstake
        ports:
          - "3306"
        env_file:
          - configuration.env
        environment:
          MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD}
          MYSQL_DATABASE: ${DB_NAME}
          MYSQL_USER: ${DB_USER}
          MYSQL_PASSWORD: ${DB_PASSWORD}
        volumes:
          - mysql_data:/var/lib/mysql
        deploy:
          mode: replicated
          replicas: 1
          
    app:
        env_file:
          - configuration.env
        image: ${SMSTAKE_VERSION}
        ports:
          - 8000:80
        networks:
          - smstake
        depends_on:
          - db
        deploy:
          mode: replicated
          replicas: 1
          placement:
            constraints:
              - node.role == manager
volumes:
    mysql_data:

Why is it not reading from the configuration.env file ,the right value with that key is set there I have confirmed .

Error message:

Creating service smstake_app
failed to create service smstake_app: Error response from daemon: rpc error: code = InvalidArgument desc = ContainerSpec: image reference must be provided
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Armil

But you can rollback by using the version, i.e.image: smstake:12345678901234567890 – Skeen

@ConstantinGalbenu from where shall i get that version and what will be flow – Armil

from the docker repository, there you can see the history – Skeen

Answer 1

In the docker-compose file, you can have variables substitution based on environment variables. This is documented under Variable Substitution.

You can use the following to specify a different version for the image:

image: smstake:${SMSTAKE_VERSION}

And inside the jenkins job that deploys, you can just set this environment variable and run the docker stack command:

SMSTAKE_VERSION=v1.2.0 docker stack deploy -c docker-compose.yml stackname

Furuncle

Yes I am doing it exactly as What you recommended. How do i confirm or any way to check it is successfully getting the value. than after I will upvote your answers. Thanks – Armil

This is really not working. The value is not passed to the image field in docker-compose.yml and thus I am getting the issue. failed to create service smstake_app: Error response from daemon: rpc error: code = InvalidArgument desc = ContainerSpec: image reference must be provided. As per the error jenkins error logs its not getting the image name set in docker-compose.yml – Armil

@TaraPrasadGurung you can run the docker-compose config command to verify that you are setting the right values. See docs.docker.com/compose/environment-variables/#the-env-file – Jarboe

A $ is missing: image: smstake:${SMSTAKE_VERSION} – Semiskilled

This works if a variable is declared in a dockerfile via ARG. ARG SMSTAKE_VERSION \n FROM smstake:${SMSTAKE_VERSION} – Clarke

It is best to export the variable if running in Linux export SMSTAKE_VERSION=v1.2.0 – Swaim

Answer 2

You need to do it in two stages

docker-compose config | docker stack deploy -c - stackname

You need to use docker-compose and not the docker compose V2 as docker stack does not support version-less manifests (at least not until the 21.xx version which as of this writing is not out yet)

You also get the added bonus of using a .env file to read your environments which makes it easy for devs as well.

Archaeornis