Push Docker image to ECR using CloudFormation

Question

I am new to Devops.

As part of gitci, I have a Docker file in GitLab. I am planning to create a Docker image and push it to ECR and then use that image for batch processing.

I have already completed batch processing part using existing image in ECR. But not able to create Docker image and push using CloudFormation.

Should I use command in init?

Thanks in advance gurus

Metalware

Answer 1

I had the same problem at work and sumbled across this already asked question that does not have a proper answer, so I'll give instructions of how I did.

Baseline: you cannot do this with cloudformation, cloudformation is used to create infrastructure and automations. Although, you can accomplish this with codebuild and you can use cloudformation to create a codebuild project. This repository does that for a practical example.

What you would do is: create a cloudformation template that creates a codebuild project and in your codebuild create a buildspec.yml (file that specifies the build) that will push your image to ECR.

The codebuild project would look like this:

CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
  Artifacts:
    Type: CODEPIPELINE
  Description: "Codebuild project to push flask api image to ecr"
  Environment:
    ComputeType:
      !FindInMap [CodeBuildComputeTypeMap, !Ref GithubBranch, type]
    EnvironmentVariables:
      - Name: AWS_DEFAULT_REGION
        Value: !Ref AWS::Region
      - Name: AWS_ACCOUNT_ID
        Value: !Ref "AWS::AccountId"
      - Name: AWS_ECR_REPOSITORY_URI
        Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${EcrRepository}
      - Name: IMAGE_REPO_NAME
        Value: !Ref GithubRepository
      - Name: IMAGE_TAG
        Value: "latest"
    Image: "aws/codebuild/standard:5.0"
    PrivilegedMode: true
    Type: "LINUX_CONTAINER"
  ServiceRole: !GetAtt CodeBuildRole.Arn
  Source:
    Type: "CODEPIPELINE"
    BuildSpec: buildspec.yml

EcrRepository:
     Type: AWS::ECR::Repository
     Properties:
       RepositoryName: !Ref GithubRepository

CodeBuildRole:
Type: AWS::IAM::Role
Properties:
  AssumeRolePolicyDocument:
    Version: "2012-10-17"
    Statement:
      - Effect: Allow
        Principal:
          Service:
            - codebuild.amazonaws.com
        Action:
          - "sts:AssumeRole"
  Policies:
    - PolicyName: "PushImageToEcr"
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action:
              - ecr:BatchGetImage
              - ecr:BatchCheckLayerAvailability
              - ecr:CompleteLayerUpload
              - ecr:GetDownloadUrlForLayer
              - ecr:InitiateLayerUpload
              - ecr:PutImage
              - ecr:UploadLayerPart
              - ecr:GetAuthorizationToken
            Resource: "*"
    - PolicyName: "CodeBuildLogsRole"
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action:
              - logs:CreateLogGroup
              - logs:CreateLogStream
              - logs:PutLogEvents
            Resource:
              - !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/codebuild/*"
    - PolicyName: "GetAndPutArtifacts"
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: "Allow"
            Action:
              - s3:GetObject
              - s3:PutObject
              - s3:ListBucket
            Resource:
              - !GetAtt ArtifactBucket.Arn
              - !Sub ${ArtifactBucket.Arn}/*

ArtifactBucket:
    Type: AWS::S3::Bucket

This should go in the Resources section of the cloudformation and will create the codebuild project, the ecr repository, the codebuild service role and the s3 bucket for the artifacts.

Then you need a buildspec.yml template to push your image, this would look like this:

version: 0.2

phases:
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
  build:
    commands:
      - echo Build started on `date`
      - echo Building the Docker image...
      - cd app/
      - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
      - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Pushing the Docker image...
      - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG

Tired

Answer 2

You won't be able to do it using CloudFormation as it's not intended to do this style of operation.

However, you mentioned that you're using gitlab-ci. This means you could easily create a job that build your docker image and uploads it to ECR.

In my opinion, it's even easier to create a CodeBuild project (using CloudFormation, IaC FTW!) that handle the building and uploading of your Docker image. The advantage of CodeBuild over GitLab-ci is that you'll be able to give the proper ECR access to the CodeBuild worker for it to upload the image to the repository.

Katharinakatharine

Do you think it's better to put your buildspec.yml directly into your cloudformation template or to keep a buildspec file in the repository for this use case? – Shoveler

That's a good question JIMPRO, I think having it the repository gives you more flexibility you can have different buildspecs for different branches. However, if maintainer of the repo is not a direct stakeholder of the automation (maybe for security concerns) you can "hide" the buildspec inside the CodeBuild project definition. – Katharinakatharine