How to obtain AWS IOT endpoint URL from within a CloudFormation template?

Question

I want some of my Lambda resources to push to an AWS IOT endpoint using aws-sdk's AWS.IotData({ endpoint: url }) function - where endpoint is a required parameter.

Right now, I am passing the endpoint URL via an environment variable to my Lambda. However, when put into a SAM/CF template, I can't find a way to retrieve my IOT endpoint URL, so that I could simply !Ref it.

Browsing through the AWS resource type reference I did not find any resource that corresponds to an IOT endpoint.

It seems like IOT endpoint can only be provisioned manually, via AWS Console (enabled / disabled), as on the screenshot below:

Any advice on how to have control over provisioning an IOT endpoint or at least reading the IOT URL from within a SAM/CF template, without scripting this with aws-cli?

Damato

Answer 1

I'm afraid you cannot provision an IoT endpoint, as the only API call that is related to an IoT endpoint is DescribeEndpoint.

What you can do is create a Lambda-backed CloudFormation Custom Resource. The Lambda function will execute the DescribeEndpoint call (using the AWS SDK of your choice depending on the Lambda's runtime) and return the endpoint's URL so your other CloudFormation resources can consume it.

Here's a good example on Lambda-backed Custom Resources: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-lambda.html.

Folkways

Answer 2

For anyone interested in the solution with CloudFormation Custom Resource, I wrote a simple Lambda and a CF template that provides an IOT endpoint address to other CF stacks.

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
  IotEndpointProvider:
    Type: 'AWS::Serverless::Function'
    Properties:
      FunctionName: IotEndpointProvider
      Handler: iotEndpointProvider.handler
      Runtime: nodejs6.10
      CodeUri: .
      MemorySize: 128
      Timeout: 3
      Policies:
        - Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action: 
              - iot:DescribeEndpoint
            Resource:
              - '*'
  IotEndpoint:
    Type: 'Custom::IotEndpoint'
    Properties:
      ServiceToken: !GetAtt IotEndpointProvider.Arn
Outputs:
  IotEndpointAddress:
    Value: !GetAtt IotEndpoint.IotEndpointAddress
    Export:
      Name: IotEndpointAddress

iotEndpointProvider.js

var aws = require("aws-sdk");

exports.handler = function(event, context) {
    console.log("REQUEST RECEIVED:\n" + JSON.stringify(event));

    // For Delete requests, immediately send a SUCCESS response.
    if (event.RequestType == "Delete") {
        sendResponse(event, context, "SUCCESS");
        return;
    }

    const iot = new aws.Iot();
    iot.describeEndpoint({}, (err, data) => {
    let responseData, responseStatus;
        if (err) {
            responseStatus = "FAILED";
            responseData = { Error: "describeEndpoint call failed" };
            console.log(responseData.Error + ":\n", err);
        } else  {
            responseStatus = "SUCCESS";
            responseData = { IotEndpointAddress: data.endpointAddress };
            console.log('response data: ' + JSON.stringify(responseData));
        }

        sendResponse(event, context, responseStatus, responseData);
    });
};

// Send response to the pre-signed S3 URL 
function sendResponse(event, context, responseStatus, responseData) {

    var responseBody = JSON.stringify({
        Status: responseStatus,
        Reason: "See the details in CloudWatch Log Stream: " + context.logStreamName,
        PhysicalResourceId: context.logStreamName,
        StackId: event.StackId,
        RequestId: event.RequestId,
        LogicalResourceId: event.LogicalResourceId,
        Data: responseData
    });

    console.log("RESPONSE BODY:\n", responseBody);

    var https = require("https");
    var url = require("url");

    var parsedUrl = url.parse(event.ResponseURL);
    var options = {
        hostname: parsedUrl.hostname,
        port: 443,
        path: parsedUrl.path,
        method: "PUT",
        headers: {
            "content-type": "",
            "content-length": responseBody.length
        }
    };

    console.log("SENDING RESPONSE...\n");

    var request = https.request(options, function(response) {
        console.log("STATUS: " + response.statusCode);
        console.log("HEADERS: " + JSON.stringify(response.headers));
        // Tell AWS Lambda that the function execution is done  
        context.done();
    });

    request.on("error", function(error) {
        console.log("sendResponse Error:" + error);
        // Tell AWS Lambda that the function execution is done  
        context.done();
    });

    // write data to request body
    request.write(responseBody);
    request.end();
}

Damato

Does it mean every time I need to get endPoint url I have to call this lambda function? – Negotiation

That depends on your needs but in this case lambda will be fired once as a custom resource, during CloudFormation template deployment to provide the URL to all resources which need to know that URL. – Damato

thanks for the answer, can you tell me how exactly I can refer to that url within same repo plus other repos? can I do something like this - environment: IOT_END_POINT: ${self: Outputs.IotEndPointAddress} ? – Negotiation

I'm just really new to CloudFormation and still figuring out lot of things – Negotiation