AWS Elastic Beanstalk: How to use environment variables in ebextensions?

Question

We are trying to store environment specific application configuration files in s3. The files are stored in different subdirectories which are named after the environment and also have the environment as part of the file name.

Examples are

dev/application-dev.properties
stg/application-stg.properties
prd/application-prd.properties

The Elastic Beanstalk environments are named dev, stg, prd and alternatively I also have an environment variable defined in Elastic Beanstalk named ENVIRONMENT which can be dev, stg or prd.

My question now is, how do I reference the environment name or ENVIRONMENT variable when downloading the configuration file from a config file in .ebextensions?

I tried using a {"Ref": "AWSEBEnvironmentName" } reference in .ebextensions/myapp.config but get a syntax error when deploying.

The content of .ebextensions/myapp.config is:

files:
  /config/application-`{"Ref": "AWSEBEnvironmentName" }`.properties:
    mode: "000666"
    owner: webapp
    group: webapp
    source: https://s3.amazonaws.com/com.mycompany.mybucket/`{"Ref": "AWSEBEnvironmentName" }`/application-`{"Ref": "AWSEBEnvironmentName" }`.properties 
    authentication: S3Access

Resources:
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Access:
          type: S3
          roleName: aws-elasticbeanstalk-ec2-role
          buckets: com.mycompany.api.config

The error I get is:

The configuration file .ebextensions/myapp.config in application version
manualtest-18 contains invalid YAML or JSON. YAML exception: Invalid Yaml: 
mapping values are not allowed here in "<reader>", line 6, column 85: 
... .config/stg/application-`{"Ref": "AWSEBEnvironmentName" }`.prop ... ^ , 
JSON exception: Invalid JSON: Unexpected character (f) at position 0.. 
Update the configuration file.

What is the correct way of referencing an environment variable in a .ebextensions config file in AWS Elastic Beanstalk?

Grosbeak

Answer 1

I struggled to get this working, until I discovered that the Sub function doesn't appear to be available in ebextensions: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions-functions.html

This means that you need to fall back to Fn::Join and Ref, at least until support for Sub is introduced to ebextensions. It also seems that the files attribute requires a fixed path (and I couldn't use Fn::Join in this context).

My overall solution to this was as follows:

Resources:
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Auth:
          type: S3
          buckets: arn:aws:s3:::elasticbeanstalk-xxx
          roleName: aws-elasticbeanstalk-ec2-role

files:
  "/tmp/application.properties" :
    mode: "000644"
    owner: root
    group: root
    source: { "Fn::Join" : ["", ["https://s3-xxx.amazonaws.com/elasticbeanstalk-xxx/path/to/application-", { "Ref" : "AWSEBEnvironmentName" }, ".properties" ]]}
    authentication: S3Auth

container_commands:
  01-apply-configuration:
    command: mkdir -p config && mv /tmp/application.properties config

This will result in an application.properties file (without the environment name qualifier) in a config directory next to the deployed application instance.

If you want to keep the name of the environment as part of the file name using this approach, you will need to adjust the command that moves the file to use another Fn::Join expression to control the filename.

Sudden

Answer 2

Your .ebextensions config file was almost correct. Substituting the file name with environment variable or AWS resource name won't work, for that do as in Mark's answer to rename the file created in container_commands section.

The source option value trying to access AWS resource name using Ref was correct, it just had to be surrounded by single quote ', like below:

files:
  /config/application.properties:
    mode: "000666"
    owner: webapp
    group: webapp
    source: 'https://s3.amazonaws.com/com.mycompany.mybucket/`{"Ref": "AWSEBEnvironmentName" }`/application-`{"Ref": "AWSEBEnvironmentName" }`.properties'
   authentication: S3Access

And to access environment variables use Fn::GetOptionSetting. Environment variables are in aws:elasticbeanstalk:application:environment namespace.

Below example access an environment variable ENVIRONMENT in source option of files:

files:
  "/tmp/application.properties" :
    mode: "000666"
    owner: webapp
    group: webapp
    source: 'https://s3.amazonaws.com/com.mycompany.mybucket/`{"Ref": "AWSEBEnvironmentName" }`/application-`{"Fn::GetOptionSetting": {"Namespace": "aws:elasticbeanstalk:application:environment", "OptionName": "ENVIRONMENT ", "DefaultValue": "dev"}}`.properties'
    authentication: S3Auth

Colonize

this is not correct this gives Service:AmazonCloudFormation, Message:Template format error: Unresolved resource dependencies [AWS_ENVIRONMENT] in the Resources block of the template – Exuberate

Answer 3

You are almost there .ebextensions are using YAML format, while your trying to use JSON. Use Ref: AWSEBEnvironmentName. In addition, you can take advantage of Sub function to avoid pesky Join:

!Sub "/config/application-${AWSEBEnvironmentName}.properties"

Muddy