Getting json body in aws Lambda via API gateway

Question

I'm currently using NodeJS to build a bot on AWS lambda via AWS Api Gateway and I'm running into an issue with POST requests and JSON data. My api uses 'Use Lambda Proxy integration' and even when I test the proxy sending a content-type of Application/json and some json in the body e.g {"foo":"bar"} I can't access the object without parsing it first

e.g

  var json = JSON.parse(event.body);
  console.log(json.foo);

Now I know this doesn't seem a big deal just running it through JSON.parse, but I've seen a number of other examples where this isn't the case at all. see here https://github.com/pinzler/fb-messenger-bot-aws-lambda/blob/master/index.js

Do I need to add anything to my API gateway to handle this correctly? my 'request body' step in the 'post method request' section has a content-type of application/json set-up for the request body.

The readme for the example above doesn't seem to use proxy integration as far as I can tell so I'm not sure what I should be doing here

Unhook

Answer 1

There are two different Lambda integrations you can configure in API Gateway:

1. Lambda non-proxy integration (docs), also called Lambda custom integration
2. Lambda proxy integration (docs)

For Lambda non-proxy integration, you can customise what you are going to pass to Lambda in the payload that you don't need to parse the body, but when you are using Lambda Proxy integration in API Gateway, API Gateway will proxy everything to Lambda in payload like this:

{
    "message": "Hello me!",
    "input": {
        "path": "/test/hello",
        "headers": {
            "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
            "Accept-Encoding": "gzip, deflate, lzma, sdch, br",
            "Accept-Language": "en-US,en;q=0.8",
            "CloudFront-Forwarded-Proto": "https",
            "CloudFront-Is-Desktop-Viewer": "true",
            "CloudFront-Is-Mobile-Viewer": "false",
            "CloudFront-Is-SmartTV-Viewer": "false",
            "CloudFront-Is-Tablet-Viewer": "false",
            "CloudFront-Viewer-Country": "US",
            "Host": "wt6mne2s9k.execute-api.us-west-2.amazonaws.com",
            "Upgrade-Insecure-Requests": "1",
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36 OPR/39.0.2256.48",
            "Via": "1.1 fb7cca60f0ecd82ce07790c9c5eef16c.cloudfront.net (CloudFront)",
            "X-Amz-Cf-Id": "nBsWBOrSHMgnaROZJK1wGCZ9PcRcSpq_oSXZNQwQ10OTZL4cimZo3g==",
            "X-Forwarded-For": "192.168.100.1, 192.168.1.1",
            "X-Forwarded-Port": "443",
            "X-Forwarded-Proto": "https"
        },
        "pathParameters": {"proxy": "hello"},
        "requestContext": {
            "accountId": "123456789012",
            "resourceId": "us4z18",
            "stage": "test",
            "requestId": "41b45ea3-70b5-11e6-b7bd-69b5aaebc7d9",
            "identity": {
                "cognitoIdentityPoolId": "",
                "accountId": "",
                "cognitoIdentityId": "",
                "caller": "",
                "apiKey": "",
                "sourceIp": "192.168.100.1",
                "cognitoAuthenticationType": "",
                "cognitoAuthenticationProvider": "",
                "userArn": "",
                "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36 OPR/39.0.2256.48",
                "user": ""
            },
            "resourcePath": "/{proxy+}",
            "httpMethod": "GET",
            "apiId": "wt6mne2s9k"
        },
        "resource": "/{proxy+}",
        "httpMethod": "GET",
        "queryStringParameters": {"name": "me"},
        "stageVariables": {"stageVarName": "stageVarValue"},
        "body": "{\"foo\":\"bar\"}",
        "isBase64Encoded": false
    }
}

For the example you are referencing, it is not getting the body from the original request. It is constructing the response body back to API Gateway. It should be in this format:

{
    "statusCode": httpStatusCode,
    "headers": { "headerName": "headerValue", ... },
    "body": "...",
    "isBase64Encoded": false
}

Pestilential

and then JSON.parse(event.body) (Lambda Proxy integration) – Fetor

@Fetor this is the main point, the one single line should be the accepted answer – Begga

@Fetor can you explain a little bit more what you mean by your comment? – Hebephrenia

@Hebephrenia its been a while since I've used it, but I think I was just highlighting that you also need to parse the JSON when using lambda proxy integration. Looks like it helped at least 39 people out, I was quite surprised. – Fetor

Answer 2

I think there are a few things to understand when working with API Gateway integration with Lambda.

Lambda Integration vs Lambda Proxy Integration

There used to be only Lambda Integration which requires mapping templates. I suppose this is why still seeing many examples using it.

• How to pass a querystring or route parameter to AWS Lambda from Amazon API Gateway

As of September 2017, you no longer have to configure mappings to access the request body.

• Serverless Architecture on AWS

Lambda Proxy Integration, If you enable it, API Gateway will map every request to JSON and pass it to Lambda as the event object. In the Lambda function you’ll be able to retrieve query string parameters, headers, stage variables, path parameters, request context, and the body from it.

Without enabling Lambda Proxy Integration, you’ll have to create a mapping template in the Integration Request section of API Gateway and decide how to map the HTTP request to JSON yourself. And you’d likely have to create an Integration Response mapping if you were to pass information back to the client.

Before Lambda Proxy Integration was added, users were forced to map requests and responses manually, which was a source of consternation, especially with more complex mappings.

Words need to navigate the thinking. To get the terminologies straight.

• Lambda Proxy Integration = Pass through
  Simply pass the HTTP request through to lambda.

• Lambda Integration = Template transformation
  Go through a transformation process using the Apache Velocity template and you need to write the template by yourself.

body is escaped string, not JSON

Using Lambda Proxy Integration, the body in the event of lambda is a string escaped with backslash, not a JSON.

"body": "{\"foo\":\"bar\"}" 

If tested in a JSON formatter.

Parse error on line 1:
{\"foo\":\"bar\"}
-^
Expecting 'STRING', '}', got 'undefined'

The document below is about response but it should apply to request.

• I received a "malformed Lambda proxy response" error or a 502 status code with Amazon API Gateway. How do I resolve this?

The body field, if you are returning JSON, must be converted to a string or it will cause further problems with the response. You can use JSON.stringify to handle this in Node.js functions; other runtimes will require different solutions, but the concept is the same.

For JavaScript to access it as a JSON object, need to convert it back into JSON object with json.parse in JapaScript, json.dumps in Python.

• How To Work with JSON in JavaScript

Strings are useful for transporting but you’ll want to be able to convert them back to a JSON object on the client and/or the server side.

The AWS documentation shows what to do.

if (event.body !== null && event.body !== undefined) {
    let body = JSON.parse(event.body)
    if (body.time) 
        time = body.time;
}
...
var response = {
    statusCode: responseCode,
    headers: {
        "x-custom-header" : "my custom header value"
    },
    body: JSON.stringify(responseBody)
};
console.log("response: " + JSON.stringify(response))
callback(null, response);

Craddock

Thanks for pointing out that the body is just escaped string NOT JSON. – Nuclide

With serverless, when invoking locally, using serverless invoke local --function <functionName> --path <path/to/data.json>, body is JSON object... – Bootee

Answer 3

If you are using a Lambda Proxy Integration, you need to use JSON.parse(event.body) for JavaScript or json.loads(event["body"]) for Python.

The other examples you are looking at are for an AWS Service Lambda Integration (which is different from a Lambda Proxy Integration), where they have constructed the event template themselves.

See https://mcmap.net/q/159086/-getting-json-body-in-aws-lambda-via-api-gateway for more detail on the structure of the Lambda Proxy Integration

Radioelement

Answer 4

I am using lambda with Zappa; I am sending data with POST in json format:

My code for basic_lambda_pure.py is:

import time
import requests
import json
def my_handler(event, context):
    print("Received event: " + json.dumps(event, indent=2))
    print("Log stream name:", context.log_stream_name)
    print("Log group name:",  context.log_group_name)
    print("Request ID:", context.aws_request_id)
    print("Mem. limits(MB):", context.memory_limit_in_mb)
    # Code will execute quickly, so we add a 1 second intentional delay so you can see that in time remaining value.
    print("Time remaining (MS):", context.get_remaining_time_in_millis())

    if event["httpMethod"] == "GET":
        hub_mode = event["queryStringParameters"]["hub.mode"]
        hub_challenge = event["queryStringParameters"]["hub.challenge"]
        hub_verify_token = event["queryStringParameters"]["hub.verify_token"]
        return {'statusCode': '200', 'body': hub_challenge, 'headers': 'Content-Type': 'application/json'}}

    if event["httpMethod"] == "post":
        token = "xxxx"
    params = {
        "access_token": token
    }
    headers = {
        "Content-Type": "application/json"
    }
        _data = {"recipient": {"id": 1459299024159359}}
        _data.update({"message": {"text": "text"}})
        data = json.dumps(_data)
        r = requests.post("https://graph.facebook.com/v2.9/me/messages",params=params, headers=headers, data=data, timeout=2)
        return {'statusCode': '200', 'body': "ok", 'headers': {'Content-Type': 'application/json'}}

I got the next json response:

{
"resource": "/",
"path": "/",
"httpMethod": "POST",
"headers": {
"Accept": "*/*",
"Accept-Encoding": "deflate, gzip",
"CloudFront-Forwarded-Proto": "https",
"CloudFront-Is-Desktop-Viewer": "true",
"CloudFront-Is-Mobile-Viewer": "false",
"CloudFront-Is-SmartTV-Viewer": "false",
"CloudFront-Is-Tablet-Viewer": "false",
"CloudFront-Viewer-Country": "US",
"Content-Type": "application/json",
"Host": "ox53v9d8ug.execute-api.us-east-1.amazonaws.com",
"Via": "1.1 f1836a6a7245cc3f6e190d259a0d9273.cloudfront.net (CloudFront)",
"X-Amz-Cf-Id": "LVcBZU-YqklHty7Ii3NRFOqVXJJEr7xXQdxAtFP46tMewFpJsQlD2Q==",
"X-Amzn-Trace-Id": "Root=1-59ec25c6-1018575e4483a16666d6f5c5",
"X-Forwarded-For": "69.171.225.87, 52.46.17.84",
"X-Forwarded-Port": "443",
"X-Forwarded-Proto": "https",
"X-Hub-Signature": "sha1=10504e2878e56ea6776dfbeae807de263772e9f2"
},
"queryStringParameters": null,
"pathParameters": null,
"stageVariables": null,
"requestContext": {
"path": "/dev",
"accountId": "001513791584",
"resourceId": "i6d2tyihx7",
"stage": "dev",
"requestId": "d58c5804-b6e5-11e7-8761-a9efcf8a8121",
"identity": {
"cognitoIdentityPoolId": null,
"accountId": null,
"cognitoIdentityId": null,
"caller": null,
"apiKey": "",
"sourceIp": "69.171.225.87",
"accessKey": null,
"cognitoAuthenticationType": null,
"cognitoAuthenticationProvider": null,
"userArn": null,
"userAgent": null,
"user": null
},
"resourcePath": "/",
"httpMethod": "POST",
"apiId": "ox53v9d8ug"
},
"body": "eyJvYmplY3QiOiJwYWdlIiwiZW50cnkiOlt7ImlkIjoiMTA3OTk2NDk2NTUxMDM1IiwidGltZSI6MTUwODY0ODM5MDE5NCwibWVzc2FnaW5nIjpbeyJzZW5kZXIiOnsiaWQiOiIxNDAzMDY4MDI5ODExODY1In0sInJlY2lwaWVudCI6eyJpZCI6IjEwNzk5NjQ5NjU1MTAzNSJ9LCJ0aW1lc3RhbXAiOjE1MDg2NDgzODk1NTUsIm1lc3NhZ2UiOnsibWlkIjoibWlkLiRjQUFBNHo5RmFDckJsYzdqVHMxZlFuT1daNXFaQyIsInNlcSI6MTY0MDAsInRleHQiOiJob2xhIn19XX1dfQ==",
"isBase64Encoded": true
}

my data was on body key, but is code64 encoded, How can I know this? I saw the key isBase64Encoded

I copy the value for body key and decode with This tool and "eureka", I get the values.

I hope this help you. :)

Blastocoel

Answer 5

You may have forgotten to define the Content-Type header. For example:

  return {
    statusCode: 200,
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ items }),
  }

Baynebridge

Answer 6

Getting json body in AWS Lambda (C#) via API Gateway and test with AWS .NET Mock Lambda Test Tool

If you're trying to debug your lambda functions locally with .NET Mock Lambda Test Tool or other tool such as Postman with the idea to expose your lambda functions through AWS API gateway with Lambda Proxy integration enabled, follow this steps:

1- Select "API Gateway AWS Proxy" under Example Requests in Mock Lambda Test Tool. You get a long json where you need to modify at least the body and path members

{
  "body": "{\"foo\":\"bar\"}",
  "path": "/YourLambdafunction",
  ...
}

2- Define your FunctionHandler with the APIGatewayProxyRequest data type for the first parameter and then you can strongly deserialize the parameter Body member

public async Task<APIGatewayProxyResponse> FunctionHandler(APIGatewayProxyRequest data, ILambdaContext context)
{
    MyCustomClass? myObj = JsonConvert.DeserializeObject<MyCustomClass>(data.Body);
}

Methodical

I wish I could give this 5000 upvotes. This led me to the fact that you needed to add the API Gateway Nugets which is not clear anywhere. – Dariusdarjeeling

@Dariusdarjeeling I wish too haha. You need actually Amazon.Lambda.APIGatewayEvents nuget lib – Methodical