Get value from AWS Systems Manager Parameter Store during Elastic Beanstalk deploy

Question

I have a database connection string stored in AWS Systems Manager Parameter Store that I want to pass to the environment variables of an Elastic Beanstalk instance. I don't want to commit it to source code nor have to rely on the deployer setting the values on deploy or doing it manually via the web console.

I have tried Dynamic References like {{resolve:ssm:DATABASE_CONNECTION_STRING:1}} (with and without back ticks) which work in CloudFormation stacks but not Elastic Beanstalk config.

I have tried using container_commands like

export DATABASE_CONNECTION_STRING=`aws ssm get-parameter --name DATABASE_CONNECTION_STRING --region eu-west-1 --query Parameter.Value --output text

but the Elastic Beanstalk instance does not have the right permissions and I'm unsure how to set them.

I have tried creating a file from the contents of an S3 file using files: and source: but get errors.

Ideally Dynamic References would work e.g. .ebextensions/env.config =>

OptionSettings:
  aws:elasticbeanstalk:application:environment:
    DATABASE_CONNECTION_STRING: {{resolve:ssm:ANNOTATOR_DATABASE_CONNECTION_STRING:1}}

Background

Answer 1

I was able to get this working by creating the file .ebextensions/options.config with the contents:

option_settings:
  aws:elasticbeanstalk:application:environment:
    ENCRYPT_CERT: '{{resolve:ssm:SOA_ENCRYPT_CERT:1}}'
    ENCRYPT_KEY: '{{resolve:ssm:SOA_ENCRYPT_KEY:1}}'

Vinegarish

what kind of permissions steps did you have to go through to make this work? – Spelunker

@Spelunker I use Bitbucket Pipelines for deployment; the user for this has the AmazonSSMReadOnlyAccess and AWSElasticBeanstalkFullAccess permissions – Vinegarish

For anyone else working through this - The value from Parameter Store will not appear in your elastic beanstalk console as an Environment Property. Instead, it will appear as {{resolve:ssm:WHATEVER:1}}. Then when your environment tries to grab this property, it will get it from Parameter Store. Best to cache these values on deploy, so that it's not constantly grabbing them from Parameter Store at runtime. – Amadaamadas

Note that this solution doesn't work if using Docker platform ECS running on 64bit Amazon Linux 2 as the value of the env var is copied into the ECS Task without being resolved. Probably as a security measure so that the value doesn't show in the ECS console. Use platform Docker running on 64bit Amazon Linux 2023 unless you really need to use ECS – Frenetic

Answer 2

I've gone with a files configuration that downloads a file from S3 into /tmp/app.env and then the Python app uses load_dotenv('/tmp/app.env'). Not ideal but it works for now.

Here is the .ebextensions/env.config:

Resources:
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Auth:
          type: "s3"
          buckets: ["S3_BUCKET_NAME"]
          roleName: 
            "Fn::GetOptionSetting": 
              Namespace: "aws:autoscaling:launchconfiguration"
              OptionName: "IamInstanceProfile"
              DefaultValue: "aws-elasticbeanstalk-ec2-role"
files:
  "/tmp/app.env":  
    mode: "444"
    owner: wsgi
    group: wsgi
    authentication: "S3Auth"
    source: https://URL_TO_S3_BUCKET/app.env

Make sure to change the S3_BUCKET_NAME and URL_TO_S3_BUCKET to your settings.

Then in Python I use;

if os.path.exists('/tmp/app.env'):
    load_dotenv('/tmp/app.env')
else:
    load_dotenv('.env')

Background

Answer 3

You need to use singles quotes in your config file, on the browser you can paste it without quotes.

DATABASE_CONNECTION_STRING: '{{resolve:ssm:ANNOTATOR_DATABASE_CONNECTION_STRING:1}}'

If you update the parameter change the 1 at the end to the version it is. So if you updated it once after create the parameter key it will be 2.

Logan

Answer 4

I resolved a similar issue following these steps:

Followed these two articles:

• https://repost.aws/knowledge-center/elastic-beanstalk-env-variables-shell
• https://repost.aws/knowledge-center/elastic-beanstalk-use-env-variables

Created a .ebextensions\env-variables.config file in project root with following contents:

container_commands:
    setvars:
        command: /opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' > /etc/profile.d/local.sh
packages:
    yum:
        jq: []

Go to Elastic Beanstalk > Environments > [your environment] > Configuration > Updates, monitoring, and logging > Environment properties.

• Give .env variables and values as required

Multivalent