How do you handle config files for AWS CodePipelines?

Question

I am on a team of developers using Git as our version control.

We want to have a minimum of 3 stages of our development process: staging, dev, and production.

The only thing that should change between these stages is a single config file, to tell the Serverless framework what to name the lambda functions, S3 buckets, and any other resource that needs to be created for the CloudFormation stack.

However, this makes source control a bit harder. If we put the config files directly in the source code, then we have to make sure that those files don't get overridden when we commit/push to origin. But the CodeBuild has to have access to it somehow, and it has to be sure to grab the right config file for the specified stage.

I would prefer a solution to this issue that is a part of the AWS ecosystem.

Horribly

Have you read aws.amazon.com/blogs/devops/… ? It looks very similar to what you want to do. – Overlay

Answer 1

What I'd suggest is to have your environment variables stored in EC2 Parameter Store which you can reference in your CodeBuild buildspec.yml.

To use CodePipeline in your case, you also need different pipelines and different CodeBuild projects for each environment.

For example, say you store the following variables in EC2 Parameter Store (or AWS SSM),

DEVELOPMENT_DB_PASSWORD='helloworld'
STAGING_DB_PASSWORD='helloworld'
PRODUCTION_DB_PASSWORD='helloworld'

In your CodeBuild project, you have to specify the environment as a variable (e.g. $ENVIRONMENT=DEVELOPMENT). Don't use buildspec for this. You can use AWS Console or CloudFormation.

Then, your buildspec.yml can look like this:

env:
  parameter-store:
    DEVELOPMENT_DB_PASS: "DEVELOPMENT_DB_PASSWORD"
    STAGING_DB_PASS: "DEVELOPMENT_DB_PASSWORD"
    PRODUCTION_DB_PASS: "DEVELOPMENT_DB_PASSWORD"

These variables are then accessible in your serverless.yml using ${env:ENVIRONMENT}_DB_PASS like so:

provider:
  environment:
    DB_PASS: ${env:${env:ENVIRONMENT}_DB_PASS}

All you have to do now is to create those three CodePipelines each having their own CodeBuild project (with each project using a different $ENVIRONMENT).

Testate

I'm stuck on the last step, I think my serverless.yml file is interpreting the value as a literal string, not realizing it is an environment variable – Horribly

Answer 2

Why don't you use three config file ? one for each stage.

Immortal