aws-sam-local environment variables

Question

I am following the readme here: https://github.com/awslabs/aws-sam-local

I have a lambda written in python 3.6 and its similar to the helloworld example here : https://github.com/awslabs/aws-sam-local/tree/develop/samples/hello-world/python

template.yml looks as such:

AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: MyFunction1 API
Resources:
  MyFunction1:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: MyFunction1
      Handler: lambda_module.lambda_handler
      Runtime: python3.6
      CodeUri: lambda.zip
      MemorySize: 128
      Timeout: 10
      Policies:
        -AWSLamdbaBasicExecutionRole
      Events:
        BingLambdaEndpoint:
          Type: Api
          Properties:
            Path: MyFunction1/search
            Method: get

I have environment variables within the lambda but not able to wire them up on start up. Documentation says I can create a environments.json file and append the following on the invoke command : Use --env-vars argument of invoke

My environment file looks like the example and I get an error: Unable to find environment variable: api_key

environment.json looks as such:

{
  "MyFunction1": {
    "api_key": "123456789",
    "BUCKET_NAME": "testBucket"
  }
}

command I run is as such:

sam local invoke MyFunction1 --env-vars environment_variables.json -e event.json

Can anyone provide additional insight?

Shulamite

Answer 1

Any environment variables you want to use with SAM Local in this manner need to exist in your SAM template. From this GitHub issue:

... SAM Local only parses environment variables that are defined in the SAM template.

In the template, you can provide no value, an empty string, or choose a sensible default.

A portion of a template including environment variables:

Resources:
  MyFunction1:
    Type: 'AWS::Serverless::Function'
    Properties:
      .....
      Environment:
        Variables:
          api_key:
          BUCKET_NAME:

---

You can think of the environment variables file as a mechanism to override environment variables that the template "knows" about. It does not work as a mechanism to inject arbitrary environment variables into the local runtime.

Bibliopole

I understand that, but when running the lambda it seems to have no knowledge of the environment variables. However, I can configure them in the template file and it works. I would like to use a separate config file to house my variables as outlined in the Sam local docs. – Shulamite

I found this from google when looking for a reference to using environment variables. According to the docs about sam local environment variables: The Environment key defines the variables and the default values. One can use the --env-vars cli argument to define a json file to override the values. I will update the answer with this info. – Mister

You can also apply this at a Global level: Globals: Function: Environment: Variables: api_key: – Inveracity

Answer 2

template file

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
  
Globals:

  Function:
    Timeout: 3

Parameters:

  SomeVar:
    Type: String
    Description: My SomeVar
    Default: default value

Resources:

  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello-world/
      Handler: app.lambdaHandler
      Runtime: nodejs12.x
      Environment:
        Variables:
          SOME_VAR: !Ref SomeVar
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get

Outputs:

  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"

  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn

  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

from https://github.com/awslabs/aws-sam-cli/issues/1163#issuecomment-557874976

then in code

console.log(process.env.SOME_VAR);

when you run sam local start-api and it will print default value

when you run sam local start-api --parameter-overrides SomeVar=other_value and it will print other_value

then if you create file env.json with this content

{ "PreviewsFunction": { "SOME_VAR": "123" } }

when you run sam local start-api --env-vars env.json and it will print 123

p.s. it will work with start-api/start-lambda/invoke all in the same way, but it looks like sam deploy only works with --parameter-overrides SomeVar=other_value and no --env-vars

Glottis

Thanks! This is great... finally some sanity dealing with envvars and SAM. Good thing about this is that when you run sam deploy --guided you are prompted to add any envvars you've specified like this. Exactly what I needed. – Seay

It's not working with env.json file, it still prints the default value with env.json – Sharpedged

still prints the default value! – Mraz

Also add "NoEcho: true" to not print secrets in the console. – Pub

Answer 3

Ensure that the variables are declared in template.yml. A config file overwrites the variables but does not create variables when they don't exist in the original template.

Durstin

Thank you! I kept trying to pull in env variables with sam build --use container --container-env-var-file env.json, but I didn't have those variables pre-declared in template.yaml file so it kept failing – Sloop

Answer 4

I had the same issue. When I ran

sam local start-api --env-vars  env.json 

The values in env.json where not being read. What fixed the issue for me was to use the following format in env.json

"Parameters": {
    "PARAM_NAME": "VALUE"
}

The other format did not work for me:

{ "function": { "PARAM_NAME": "VALUE" } }

Rattail

Answer 5

You can use the option -n, --env-vars PATH. This this is JSON file and you need to have an object that maps with your resource name in the template.

template.yaml:

...
Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      ...
      Environment:
        Variables:
          SOME_VAR: !Ref SomeVar
...

Let's assume your file name for the variables is .env.local.json. The content should match something like this:

.env.local.json:

{
  "HelloWorldFunction": {
     "SOME_VAR": "NEW_VALUE"
  }
}

Then you need to run the command:

sam local start-api --env-vars .env.local.json

Calfee

Where does the file .env.local.json actually live? – Phelan

Anywhere you want, since you're just specifying its location with the -n flag. – Belsky

Answer 6

Make sure that your template's parameters have NoEcho equals true, like in this example:

Parameters:
  # Build variables
  Stage:
    Type: String
  # Environment variables
  MssqlServer:
    Type: String
    NoEcho: true
  MssqlDatabase:
    Type: String
    NoEcho: true
  MssqlUser:
    Type: String
    NoEcho: true
  MssqlPassword:
    Type: String
    NoEcho: true

Then you can simply pass the environment variables with a command like this:

sam deploy --parameter-overrides "MssqlServer='$MSSQL_SERVER' MssqlDatabase='$MSSQL_DATABASE' MssqlUser='$MSSQL_USER' MssqlPassword='$MSSQL_PASSWORD' Stage=dev" --config-env dev

Cheesecake

I'm not 100% sure it applies here, but AWS recommends against putting secrets into "dynamically referencable values". See here: docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/… "To determine which resource properties comprise a resource type's primary identifier, refer to the resource reference documentation for that resource. In the Return values section, the Ref function return value represents the resource properties that comprise the resource type's primary identifier." Unless that's what NoEcho is for? – Shel

More info here: docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/… "Rather than embedding sensitive information directly in your CloudFormation templates, we recommend you use dynamic parameters in the stack template to reference sensitive information that is stored and managed outside of CloudFormation, such as in the AWS Systems Manager Parameter Store or AWS Secrets Manager. For more information, see the Do not embed credentials in your templates best practice." – Shel

Answer 7

Here’s a comprehensive solution to use environment-specific variables with AWS SAM. Do not use this approach for secrets!

In the Cloudformation template (template.yaml by default), define the variables and refer to them as needed, for example:

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'

Parameters:
  TargetEnv:
    Type: String
    Description: Target environment name
  DynamoDB:
    Type: String
    Description: ARN of the DynamoDB table
  SecurityGroups:
    Type: List<String>
    Description: List of VPC security groups
  Subnet:
    Type: String
    Description: VPC subnet identifier

Globals:
  Function:
    Environment:
      Variables:
        TargetEnv: !Ref TargetEnv
        DynamoDB: !Ref DynamoDB
  …

Resources:
  ServiceRole:
    Type: 'AWS::IAM::Role'
    Properties:
      Policies:
        - PolicyName: dynamodb
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - dynamodb:Query
                Resource: !Ref DynamoDB
      …

  MyFunction1:
    Type: AWS::Serverless::Function
    Properties:
      Role: !GetAtt ServiceRole.Arn
      VpcConfig: &vpc_props
        SecurityGroupIds: !Ref SecurityGroups
        SubnetIds:
          - !Ref Subnet
    …

The variables that are listed under Globals.Function.Environment.Variables are also available at runtime – in Node.js in this example as process.env.TargetEnv and process.env.DynamoDB

Then, set the parameters’ values in SAM’s configuration (samconfig.yaml by default, or use the TOML format if you prefer).

version: 0.1

default:
  build:
    parameters:
      cached: true
      parallel: true
  deploy:
    parameters: &default_deploy_parameters
      stack_name: my-service
      s3_prefix: my-service
      resolve_s3: true
      region: eu-central-1
      capabilities: CAPABILITY_IAM
      image_repositories: []

staging:
  deploy: &staging_deploy
    parameters:
      <<: *default_deploy_parameters
      parameter_overrides:
        - TargetEnv=staging
        - HandshakeTable=arn:aws:dynamodb:eu-central-1:07867760868:table/MyTable
        - SecurityGroups=sg-61f4470b87daef18c,sg-72a87ea4bf7f4b5d2
        - Subnet=subnet-16b4ca88aca7ab4b8

  local_start_api: *staging_deploy
  local_invoke: *staging_deploy

production:
  deploy:
    parameters:
      <<: *default_deploy_parameters
      parameter_overrides:
        - TargetEnv=production
        - HandshakeTable=arn:aws:dynamodb:eu-central-1:870517811788:table/MyTable
        - SecurityGroups=sg-9fa4b495c33d84a18,sg-fcb69e4ca84aa7e81
        - Subnet=subnet-daad36a5d2a700a8e

Finally, you can use the environments defined in the template (staging, production) with the respective commands.

sam deploy --config-env staging
sam local start-api --config-env staging
sam local invoke MyFunction1 --config-env staging
sam deploy --config-env production

For sam build, the environment must not be specified.

Catcall