How to set a stage name in a SAM template

Question

I want to set a stage name for the API Gateway in a SAM template.yaml. But whatever I try I'm not succeeding. Without trying to name my stage, everything works as expected but with the default stage names Prod and Stage.

My sam-cli version is 0.47.0

I did find three comparable questions here on Stackoverflow but none of the answers work for me.

• How can I change the name of the API stage in a SAM template?
• How can I use api gateway stages via cloudformation or sam?
• Using SAM file to remove default “Stages” in AWS ApiGateway?

I always get an error something like this:

Unresolved resource dependencies [ServerlessRestApi] in the Outputs block of the template

So how do I get a stage name I choose myself. I don't care much if Prod and Stage coexist with my chosen name.

Just to be complete, my template.yaml file is below:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sam-app

  Sample SAM Template for sam-app
  
Globals:
  Function:
    Timeout: 3
  Api:
    Cors:
      AllowMethods: "'OPTIONS,PUT'"
      AllowHeaders: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
      AllowOrigin: "'*'"
  
Resources:

  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello-world/
      Handler: app.lambdaHandler
      Runtime: nodejs12.x
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello-world
            Method: put

Outputs:
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/dev/hello-world/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

I probably don't understand the intended workflow behind this. Why have 2 stage names when the lambda function the API Gateway is pointing to, is the same?

I will have 'dev' and 'prod' environments but they will use different stack names so I can never mix up the different environments.

I always use deploy-dev.sh and deploy-pod.sh scripts that check if I'm on the development or master (production) branch before actually deploying something. So those scripts would point to a different template.yaml files because they are called from different git branches. I'm using this way for deployment already for a long time and it works well for me.

On a side note: Why the existing stage names start with a capital? It looks so ugly and unusual.

Schedule

Answer 1

So I found my own answer which is sort of a combination of two answers to the questions I found on StackOverflow that are mentioned in my question.

I still don't understand why this is so needlessly complicated.

I added a parameter to the top level of the template.yaml file. The use of a parameter is not strictly needed. I added this so I can have a single template file that is called from both my deploy-dev.sh and deploy-prod.sh scripts. Below is the parameter declaration:

Parameters:
  Stage:
    Type: String
    Default: dev

Then, under the Resources group, I added a new ApiDeployment resource. The name you use is totally up to you as long as you use the exact same name elsewhere as a !Ref. The only reason to add this resource is that you are not allowed to simply use the StageName in the properties of the Api section of the function event. You are also not allowed to put StageName in the Globals Api section.

ApiDeployment:
  Type: AWS::Serverless::Api
    Properties:
      StageName: !Ref Stage <- this is referencing the parameter but it could be a fixed value

Then, under the Events section of the Lambda function I added a property RestApiId that is referencing the ApiDeployment resource. The last line in the block below.

HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello-world/
      Handler: app.lambdaHandler
      Runtime: nodejs12.x
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello-world
            Method: put
            RestApiId: !Ref ApiDeployment

As I mentioned in my question, I got errors complaining about the output section of the yaml file. It turns out that the output section is optional anyway. So when I commented it out, everything worked.

But I used the output section in my deploy script to show me the URL of the API Gateway so with some trying I got that working too. The error was caused in the 4th line. It originally had ${ServerlessRestApi}. Just replace it with the new resource name I added to the yaml file: ${ApiDeployment} and everything is fine.

Outputs:
  ApiDeployment:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ApiDeployment}.execute-api.${AWS::Region}.amazonaws.com/${Stage}/hello-world/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

Because I use a parameter in the yaml file, you need to call sam deploy with the parameter name and value. The exact syntax for this is, like a lot of AWS's documentation, very well hidden. So below is the way you start your deployment:

sam deploy --parameter-overrides "ParameterKey=Stage,ParameterValue=dev"

You probably still have the Stage stage in the API Gateway console under Stages but you can delete that without any repercussions.

For completeness here is my full template.yaml file which is, by the way, the file you get when you do sam init

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
      sam-app: Sample SAM Template for sam-app

Parameters:
  Stage:
    Type: String
    Default: dev

Globals:
  Function:
    Timeout: 3
  Api:
    Cors:
      AllowMethods: "'OPTIONS,PUT'"
      AllowHeaders: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
      AllowOrigin: "'*'"
  
Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello-world/
      Handler: app.lambdaHandler
      Runtime: nodejs12.x
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello-world
            Method: put
            RestApiId: !Ref ApiDeployment
  ApiDeployment:
    Type: AWS::Serverless::Api
    Properties:
      StageName: !Ref Stage

Outputs:
  ApiDeployment:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ApiDeployment}.execute-api.${AWS::Region}.amazonaws.com/${Stage}/hello-world/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

Schedule

why doesnt this have more upvotes?! You saved my day. That must have been a few days of research. – Adalai

@Eric Dela Cruz Thank you. The stage parameter is now being used as a stage and inside the url. I do however still have 2 api gateways in the end, 1 with a dev stage and another one with prd stage. Is there a way for them to be merged? – Hydrophilous

@Hydrophilous I don't think you can merge them. Delete the one you don't like. – Schedule

If stage already exists, it will give an error :( – Horner

Are we need to edit the samconfig.toml file too as stated here? Or just edit the template.yaml only? @DV82XL – Excruciation

@curiouscheese I don't know the answer to that. As you can read below I dropped SAM a long time ago and switched to CDK. Give it a try, it's a lot easier. – Schedule

Answer 2

Here is an update for anyone landing on this question.

I'm not using SAM anymore. I switched to AWS CDK. The CDK lets you define your entire AWS infrastructure in Code! You can use Javascript, Typescript, Python, C# and Java although most examples seem to be in Typescript.

It's not a very easy switch but it is worthwhile. SAM seems to be a dead-end street.

SAM only covers a small subset of all AWS has to offer but the CDK covers everything.

It's quite new and it's a moving target, plus the devs don't give a rat's ass about breaking updates as they are still moving things around between modules.

But after a few days, you'll start to get a grasp and it has infinite flexibility because you are using a normal programming language to set up things like API Gateways, Lambdas, Custom Domains, IAM rules, etc. It's also very compact (compared to SAM templates).

I used it to have different stacks for staging and production based upon the got branch I'm in. So when I deploy while my repo is on the dev branch I'll have a different environment (including different domain names and so) then when I would deploy from the master or prod branch. The names of the different services also differ, depending on the git branch.

To deploy, you just run "cdk deploy"

To get started look at this excellent workshop: https://cdkworkshop.com/

Below is an example of this branch switching. I only show parts of the method for the dev branch. For the prod branch, I just have a copy of the same method in the same file but the method has a different name and the variables for the service names also differ. Just look at the piece of (incomplete) code and you should get an idea of how it works.

import * as branchName from 'current-git-branch'

const branch = branchName()

/*-------- This is the development stack --------*/
export class StripePaymentsDev extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props)

    if (branch !== 'dev') {
      throw new Error('Not on dev branch')
    }

    // Env vars from Parameter Store
    const STRIPE_PUBLIC = StringParameter.valueForStringParameter(this, `/${branch}/STRIPE_PUBLIC`)
    const STRIPE_SECRET = StringParameter.valueForStringParameter(this, `/${branch}/STRIPE_SECRET`)
    const STRIPE_API_VERSION = StringParameter.valueForStringParameter(this, `/${branch}/STRIPE_API_VERSION_PAYMENTS`)

    // Names for the dev environment
    const domainMapName = 'PaymentsDev'
    const eventBusName = 'WebhooksBusDev'
    const ruleName = 'WebhooksRuleDev'

    const eventBus = new EventBus(stackScope, eventBusName, { eventBusName })
    const cert = Certificate.fromCertificateArn(stackScope, certName, certArn)
    const stackScope = this

    // IAM rules
    const lambdaPolicy = new iam.PolicyStatement({
      actions: ['events:*'],
      resources: ['*']
    })

    const sqsPolicy = new iam.PolicyStatement({
      actions: ['sqs:*'],
      resources: ['*']
    })
    const webhooks = new lambda.Function(stackScope, lambdaWebhooksName, {
      runtime: lambda.Runtime.NODEJS_12_X,
      code: lambda.Code.fromAsset('webhook-handler'),
      handler: 'webhooks.handler',
      timeout: Duration.seconds(600),
      description: 'Processes Stripe Webhooks',
      retryAttempts: 0,
      environment: {
        STRIPE_PUBLIC,
        STRIPE_SECRET,
        STRIPE_API_VERSION,
        MONGO_URL,
        MONGO_DB,
        MONGO_PORT,
        DEBUG
      }
    })

    webhooks.addToRolePolicy(sqsPolicy)

    const rule = new Rule(stackScope, ruleName, {
      description: 'Triggers lambda to process stipe webhooks',
      enabled: true,
      eventBus: eventBus,
      eventPattern: {
        detailType: ['transaction'],
        source: ['custom.payments']
      },
      ruleName: ruleName
    })

    rule.addTarget(new eventTargets.LambdaFunction(webhooks))

    new HttpApi(stackScope, apiName, {
      defaultIntegration: new LambdaProxyIntegration({ handler: payments }),
      defaultDomainMapping: {
        domainName: new DomainName(stackScope, domainMapName, {
          domainName: PAYMENT_DOMAIN,
          certificate: cert
        })
      }
    })
  }
}

Schedule

I just started with SAM a couple days ago, but you convinced me to change to CDK. Thanks for bringing light to the post. – Hydro

Answer 3

This way creates only your specified stage instead of creating one more stage named Stage.

This setup did trick.

Globals:
Api: OpenApiVersion: 3.0.1

I've also created the new AWS::Serverless::Api named RestApi to overwrite implicit ServerlessRestApi. Remember to set RestApi into RestApiId of every API event.

template.yaml

    AWSTemplateFormatVersion: 2010-09-09
    Description: >-
      app-sam
    
    
    Transform:
    - AWS::Serverless-2016-10-31
    
    # ====================================
    # PARAMETERS SETUP
    # ====================================
    Parameters:
      StageName:
        Type: String
        Default: dev
        Description: (Required) Enter dev, prod. Default is dev.
        AllowedValues:
          - dev
          - prod
      ProjectName:
        Type: String
        Default: sam-api
        Description: (Required) The name of the project
        MinLength: 3
        MaxLength: 50
        AllowedPattern: ^[A-Za-z_-]+$
        ConstraintDescription: "Required. Can be characters, hyphen, and underscore only. No numbers or special characters allowed."
      ExistingTable:
        Type: String
        Default: example-table
        Description: (Required) The name of existing DynamoDB
        MinLength: 3
        MaxLength: 50
        AllowedPattern: ^[A-Za-z_-]+$
        ConstraintDescription: "Required. Can be characters, hyphen, and underscore only. No numbers or special characters allowed."
    
    
    # ====================================
    # GLOBAL SETUP
    # ====================================
    Globals:
      Api:
        OpenApiVersion: 3.0.1
      Function:
        Runtime: nodejs14.x
        Timeout: 180
        MemorySize: 256
        Environment:
          Variables:
            TABLE_NAME: !Ref ExistingTable
    
    Resources:
      # Reference this one to overwrite implicit stage
      # https://github.com/aws/serverless-application-model/issues/191#issuecomment-580412747 
      RestApi:
        Type: AWS::Serverless::Api
        Properties:
          Name: !Ref ProjectName
          StageName: !Ref StageName
          
      # This is a Lambda function config associated with the source code: get-all-items.js
      getAllItemsFunction:
        Type: AWS::Serverless::Function
        Properties:
          Handler: src/handlers/get-all-items.getAllItemsHandler
          Description: A simple example includes a HTTP get method to get all items from a DynamoDB table.
          Policies:
            # Give Create/Read/Update/Delete Permissions to the ExistingTable
            - DynamoDBCrudPolicy:
                TableName: !Ref ExistingTable
          Events:
            Api:
              Type: Api
              Properties:
                Path: /
                Method: GET
                RestApiId: !Ref RestApi
      
    
    Outputs:
      WebEndpoint:
        Description: "API Gateway endpoint URL for Prod stage"
        Value: !Sub "https://${RestApi}.execute-api.${AWS::Region}.amazonaws.com/${StageName}/"

Magulac

Deploying the "dev" stage works, but when I deploy the "prod" stage, the "dev" stage gets deleted! Then when I deploy the "dev" stage again, the "prod" stage gets deleted! How do I get one to stay when I deploy the other? – Fortnightly

Also, if deploying to the same stage name again, it says that the stage already exists and gives an error. – Horner

@Fortnightly you need to maintain 2 seperate stacks for "dev" and "prod". Example, for "dev" stage, you can use "sam-api-dev" and for "prod" you can use "sam-api-prod" as as your stack names. sam deploy --guided will prompt you to provide new stack name and will allow you to override default "StageName" value. – Luke

I was getting Error: [InvalidResourceException('helloWorldFunctionApi', 'property RESTApiId not defined for resource of type Api')] ('helloWorldFunctionApi', 'property RESTApiId not defined for resource of type Api') to fix it : Move RESTApiId: !Ref RestApi under Type: Api. It shouldn't be under Properties. SAM CLI, version 1.107.0 – Digastric

Answer 4

I too moved to CDK, here is what I did.

Step 1: Define Environment-Specific Configurations

Define environment-specific configurations in the cdk.json file:

{
  "app": "npx ts-node bin/my-lambda-project.ts",
  "context": {
    "dev": {
      "dbHost": "dev-db.example.com",
      "dbPort": 5432,
      "dbName": "dev_db",
      "dbUser": "dev_user",
      "dbPassword": "dev_password"
    },
    "prod": {
      "dbHost": "prod-db.example.com",
      "dbPort": 5432,
      "dbName": "prod_db",
      "dbUser": "prod_user",
      "dbPassword": "prod_password"
    }
  }
}

Step 2: Update CDK Stacks to Use Configurations In lib/my-lambda-project-stack.ts, update the stack definitions to read and use these configurations:

lib/my-lambda-project-stack.ts

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';

interface EnvironmentConfig {
  dbHost: string;
  dbPort: number;
  dbName: string;
  dbUser: string;
  dbPassword: string;
}

class LambdaStack extends cdk.Stack {
  constructor(scope: Construct, id: string, config: EnvironmentConfig, props?: cdk.StackProps) {
    super(scope, id, props);

    const helloLambda = new lambda.Function(this, 'HelloLambda', {
      runtime: lambda.Runtime.NODEJS_14_X,
      code: lambda.Code.fromAsset('lambda'),
      handler: 'hello.handler',
      environment: {
        DB_HOST: config.dbHost,
        DB_PORT: config.dbPort.toString(),
        DB_NAME: config.dbName,
        DB_USER: config.dbUser,
        DB_PASSWORD: config.dbPassword,
      },
    });

    const functionUrl = helloLambda.addFunctionUrl({
      authType: lambda.FunctionUrlAuthType.NONE,
    });

    new cdk.CfnOutput(this, 'FunctionUrl', {
      value: functionUrl.url,
    });
  }
}

export class DevStack extends LambdaStack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    const config = scope.node.tryGetContext('dev') as EnvironmentConfig;
    super(scope, id, config, props);
  }
}

export class ProdStack extends LambdaStack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    const config = scope.node.tryGetContext('prod') as EnvironmentConfig;
    super(scope, id, config, props);
  }
}

Step 3: Update bin/my-lambda-project.ts to Instantiate Stacks bin/my-lambda-project.ts

import * as cdk from 'aws-cdk-lib';
import { DevStack } from '../lib/my-lambda-project-stack';
import { ProdStack } from '../lib/my-lambda-project-stack';

const app = new cdk.App();

new DevStack(app, 'DevStack', {
  env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },
});

new ProdStack(app, 'ProdStack', {
  env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },
});

Step 4: Update Lambda Handler to Use Environment Variables Update the Lambda handler to use the environment variables:

lambda/hello.ts

export const handler = async (event: any = {}): Promise<any> => {
  const dbHost = process.env.DB_HOST;
  const dbPort = process.env.DB_PORT;
  const dbName = process.env.DB_NAME;
  const dbUser = process.env.DB_USER;
  const dbPassword = process.env.DB_PASSWORD;

  // Use these variables to connect to your database or other operations

  return {
    statusCode: 200,
    body: JSON.stringify({
      message: 'Hello, World!',
      dbHost,
      dbPort,
      dbName,
      dbUser,
      dbPassword,
    }),
  };
};

Step 5: Bootstrap and Deploy Bootstrap the Environment (if not already done):

cdk bootstrap

Step 6: Deploy desired Stack:

cdk deploy DevStack

and/or

cdk deploy ProdStack

Shewmaker