GitLab Ci/Cd to Amazon LightSail

Question

I have a problem with deploying docker image to the AWS LightSail. I'm using private containers on GitLab and my images are pushing there after build. I create second stage for ci/cd for deploying image to lightsail.

image: docker:19.03.12

services:
  - docker:19.03.12-dind

build:
  stage: build
  before_script:
    - docker login registry.gitlab.com --username $UserName -p $CiCdToken
  script:
    - docker build -t registry.gitlab.com/nickname/testprojectname .
    - docker push registry.gitlab.com/nickname/testprojectname
    
deploy:
  stage: deploy
  image: python:latest
  script: 
    - pip install awscli
    - pip install lightsailctl
    - aws lightsail push-container-image --service-name testprojectname --label testprojectname --image registry.gitlab.com/nickname/testprojectname      

Unfortunately python does not have lightsailctl and awscli doesn't support lightsail.

1. I dont know how to push builded container from private containers on gitlab to the lightsail
2. I dont know how to pass credentials to aws ctl via runner.

Best, Marcin Włoch

Dopester

Answer 1

There are 2 versions of the AWS CLI, and you are after version 2 which is the only one that contains the lightsail command push-container-image. You can ditch the python:latest image, as this is only for buidling the AWSCLI v1.

Note that in order to upload the docker image you will need BOTH docker-in-docker AND the AWSCLI (v2), so that you can get an image locally that you can upload. To do this the best approach is to use a docker image and build the AWSCLI (v2) locally using a script. Alternatively, you could also try adding docker to the default AWSCLIv2 image, but I didn't like that approach as much as i'm more familiar with alpine (the base linux distro for the docker images) and I like how lightweight and fast it is.

Here is my approach:

image: docker:19.03.12

services:
  - docker:19.03.12-dind

build:
  stage: build
  before_script:
    - docker login registry.gitlab.com -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD}
  script:
    - docker build -t registry.gitlab.com/nickname/testprojectname:${CI_PIPELINE_ID} .
    - docker push registry.gitlab.com/nickname/testprojectname:${CI_PIPELINE_ID}
    
deploy:
  stage: deploy
  image: docker # NOTE: we need docker cli to make this work!
  variables:
    AWS_ACCESS_KEY_ID: MYSUPERSECRETACCESSKEYID
    AWS_SECRET_ACCESS_KEY: MYSUPERSECRETACCESSKEYSECRET
    AWS_DEFAULT_REGION: eu-west-1
  before_script: 
    # 1. Install AWSCLIv2 (https://mcmap.net/q/258232/-awscli-version-2-on-alpine-linux#answer-61268529)
    - ./alpine.awscliv2.install.sh
    - aws --version
    # 2. Install LightsailCTL Plugin (https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-install-software)
    - apk --no-cache add curl jq
    - curl https://s3.us-west-2.amazonaws.com/lightsailctl/latest/linux-amd64/lightsailctl -o /usr/local/bin/lightsailctl
    - chmod +x /usr/local/bin/lightsailctl
  script: 
    # 3. Download the docker image for this pipeline
    - docker info
    - docker login registry.gitlab.com -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD}
    - docker pull registry.gitlab.com/nickname/testprojectname:${CI_PIPELINE_ID}
    # 4. Upload the docker image for this pipeline
    - aws lightsail push-container-image 
        --service-name testprojectname 
        --label pipeline-${CI_PIPELINE_ID} 
        --image registry.gitlab.com/nickname/testprojectname:${CI_PIPELINE_ID}
    # 5. Get the uploaded image (its different every time)
    - PIPELINE_IMAGE_TAG=$(aws lightsail get-container-images --service testprojectname | jq -r .containerImages[0].image)
    # 6. Create a deployment with the uploaded docker image
    - aws lightsail create-container-service-deployment 
        --service-name testprojectname 
        --containers "{\"testprojectname\":{\"image\":\"$PIPELINE_IMAGE_TAG\",\"ports\":{\"8000\":\"HTTP\"}}}"
        --public-endpoint "{\"containerName\":\"testprojectname\",\"containerPort\":8000,\"healthCheck\":{\"path\":\"/\"}}"

Agnesagnese

Spent half a day figuring this out. I used this docker image bentolor/docker-dind-awscli and some parts of your code and it worked! BTW, if anyone uses env vars in Lightsail for secrets or credentials, you might wanna add >/dev/null in the end of the aws lightsail create-container-service-deployment command to prevent output of those secrets in the CI logs. – Shoddy

Hi. Is it possible to setup aws credentials in order to execute these commands? I've tried to setup aws configure but I'm still having the same issue: UnrecognizedClientException: The security token included in the request is invalid. I test the credentials locally and it works perfectly. – Floorwalker

AWS credentials added to script. I had these as CI/CD variables so werent appearing. – Agnesagnese

Ok. But where are u configure those credentials in order to avoid my error? I tried several things and I cannot find why I received that error. Maybe, you can check it out this code gist.github.com/igiagante/8cbc8b87b0dd91ef34e67103e812f2e0 and give me a hand. – Floorwalker

Answer 2

"aws lightsail push-container-image" probably requires docker.
I created an image that contains awscli, lightsailctl and docker.

-  image: python:latest
+  image: ytoune/aws-lightsail-cli
   script: 
-    - pip install awscli
-    - pip install lightsailctl

Einberger