Environment specific ebextensions Beanstalk commands

Question

I have a spring-boot application for which I need to specify graphite server and port (to send metrics). For that to work, I have to install and configure statsd. I do that using the ebextensions file.

commands:
  01_nodejs_install:
    command: sudo yum -y install nodejs npm --enablerepo=epel
    ignoreErrors: true

  02_mkdir_statsd:
    command: mkdir /home/ec2-user/statsd
    ignoreErrors: true

  03_fetch_statsd:
    command: git clone https://github.com/etsy/statsd.git /home/ec2-user/statsd
    ignoreErrors: true

  04_change_example_config:
    command: "cat exampleConfig.js | sed 's/2003/<graphite-port>/g' | sed 's/graphite.example.com/<my-graphite-server>/g' > config.js"
    cwd: /home/ec2-user/statsd

  05_run_statsd:
    command: setsid node stats.js config.js >/dev/null 2>&1 < /dev/null &
    cwd: /home/ec2-user/statsd

The problem with this configuration is that I can specify only 1 graphite server here for all environments.

So I decided to move commands 04 and 05 into container_commands. I am thinking of defining an environment variable called ENV_NAME using the beanstalk console/UI, and set it to dev, qa, or prod according to the environment. Then I can use test option of container_commands to run 04 and 05 commands only for specific environment based on this ENV_NAME.

So my problem is - how can I use AWS console to define environment variable? I tried using Beanstalk console to define my variable as explained in the documentation here but it did not work. I also found (see the answer with 5 upvotes) that this method sets only JVM properties and not ENV variables.

I cannot define environment variable using ebextensions because then I'll have the same problem - can't define different env vars for different envs :)

So I need help with either:

• Set the ENV_NAME environment variable using beanstalk UI.

Or

• Suggest a way to use ENV_NAME system property in container_commands to condition whether or not to run the command based on the value of ENV_NAME.

And in case you know a simpler/better way to specify different Graphite servers for different environments, please feel free to pitch in.

Patrolman

Answer 1

The way I resolved this was to define ENV_NAME as dev and prod in dev and prod environments respectively and use the following ebextensions configuration.

commands:
  01_nodejs_install:
    command: sudo yum -y install nodejs npm --enablerepo=epel
    ignoreErrors: true

  02_mkdir_statsd:
    command: mkdir /home/ec2-user/statsd
    ignoreErrors: true

  03_fetch_statsd:
    command: git clone https://github.com/etsy/statsd.git /home/ec2-user/statsd
    ignoreErrors: true

container_commands:
  04a_container_change_example_config:
    command: "cat exampleConfig.js | sed 's/2003/<graphite-dev-port>/g' | sed 's/graphite.example.com/<graphite-dev-host>/g' > config.js"
    cwd: /home/ec2-user/statsd
    test: '[ "${ENV_NAME}" == "dev" ]'

  04b_container_change_example_config:
    command: "cat exampleConfig.js | sed 's/2003/<graphite-prod-port>/g' | sed 's/graphite.example.com/<graphite-prod-host>/g' > config.js"
    cwd: /home/ec2-user/statsd
    test: '[ "${ENV_NAME}" == "prod" ]'

  05_run_statsd:
    command: setsid node stats.js config.js >/dev/null 2>&1 < /dev/null &
    cwd: /home/ec2-user/statsd

Using test I can condition the execution of the container command on the ENV_NAME property which I have already defined in the beanstalk environment.

Patrolman

This looks good. Just wondering how you found out what syntax to use for the test. Do you know of an official aws doc for that? Thanks. – Andvari

@Dennis I don't remember exactly how I figured out but an example on some AWS documentation page helped. Perhaps this. There must have been some trial and error on my part to get it to work. – Patrolman

Thanks for the link. I guess it is actually just bash scripting? – Andvari

Being unfamiliar with bash scripting, this reference helped me make sense of some peculiarities of the syntax. – Andvari

Both tests are returning positive for me. – Keslie

this link mentions the use of test to execute a command or not – Miceli

@Patrolman Where did you define ENV_NAME? – Cid

Answer 2

In addition to the answer by @Nik:

Instead of manually adding an environment variable ENV_NAME, you could also obtain the actual environment name and store that in ENV_NAME automatically. This is achieved using option_settings in your ebextensions config file.

For example:

option_settings:
  aws:elasticbeanstalk:application:environment:
    # assign the actual env name to ENV_NAME
    ENV_NAME: '`{ "Ref" : "AWSEBEnvironmentName" }`'
    
container_commands:
  0100_execute_only_in_dev:
    # this will turn up in ebactivity.log
    command: echo "this is the development environment"
    test: '[[ $ENV_NAME = "dev" ]]'

A side note, for those who are not so familiar with shell scripting, like me: the spaces in the test expression are important (example).

Update: Amazon Linux 2

As of 2021, Amazon Linux 2 is the new standard. Amazon Linux 2 uses platform hooks instead of the container_commands in .ebextensions.

Here's a bash script equivalent to the 0100_execute_only_in_dev container command defined above, that can be used as a platform hook on Amazon Linux 2:

0100_execute_only_in_dev.sh

#!/bin/bash
if [ $ENV_NAME = "dev" ]
# the following output will end up in "/var/log/eb-hooks.log"
then echo "this is the development environment" 
fi

The shell script requires execution permission, as described here, and can be placed in any of the subdirectories of .platform/hooks and/or .platform/confighooks, in your source bundle, depending on the applicable deployment phase (prebuild, predeploy, or postdeploy). See the deployment workflow for more information. It also helps to inspect /var/log/eb-engine.log, to see what exactly happens during the deployment.

Note that platform hooks also have access to the EB environment properties.

Andvari

This is a list of other properties that can be used: docs.aws.amazon.com/elasticbeanstalk/latest/dg/… – Sunderland

Answer 3

The answer is in this Spring documentation, but I'll put it a little in my words: Since you are running a spring-boot application, you can create different 'application.properties' files, like this:

Inside each file you can place your graphite (or whatever) configuration:

In my application-dev.yml:

And in my application-prod.yml:

As you can see there is a configuration for each environment.

You can run your application with different maven profiles, in this case, let's say: dev and prod... In my case my 'dev' profile is set by default, so when the application starts it will load the dev profile and therefore, the application-dev.yml configuration.

A snippet of my pom.xml

<profiles>
    <profile>
        <id>dev</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <build>
            ...

Then, when you run your application with each profile, it will load the desired .yml file

Let's see, if I run:

java -jar mywar.war

My console loads the dev profile (because remember it is my default profile)

But if I specify the prod profile, like:

java -jar mywar.war --spring.profiles.active=prod

My console will show:

To set the environment variable in Elastic Beanstalk, go to Configuration -> Software configuration:

And set spring.profile.active, like this:

One last comment: Do not confuse Environment Properties with Environment Tags!

• Environment Properties: The ones I just showed you: The environment variables.
• Environment Tags: Elastic Beanstalk tags for resources, as explained here

Liability

Thanks for your reply. However, this won't help in my case. Note that my goal here is to configure statsd and not to configure the spring-boot application. I am already using the method you described to specify different Graphite settings for different environments in my application. However, I want to configure statsd which runs as a daemon on each host alongside the spring-boot application. – Patrolman