Why does my AWS Lambda Function return "Invalid JSON" error?
Asked Answered
F

4

14

I have a lambda function that I wrote a few days ago that was acting totally fine when tested. After going to test it today (without changing any of the code), I receive the following error: "Invalid lambda function output : Invalid JSON".

Here is the function code (Node.js 10.x):

const AWS = require("aws-sdk");
const joi = require("@hapi/joi");

const Cognito = new AWS.CognitoIdentityServiceProvider();

exports.handler = async (event) => {
    // NOTE: Cognito expects Username to be the user's email

    // Vars
    const userPoolId = process.env.COGNITO_USER_POOL_ID;
    const {email : UNSAFE_EMAIL, language : UNSAFE_LANGUAGE = "en-US"} = event;

    // Normalize email and language
    const UNSAFE_TRIMMED_EMAIL = UNSAFE_EMAIL.trim();
    const UNSAFE_TRIMMED_LANGUAGE = UNSAFE_LANGUAGE.trim();

    // Validate UNSAFE_INPUTS
    const languageRegex = /^[a-z]{2}-[A-Z]{2}$/;

    const schema = joi.object().keys({
        email: joi.string().trim().email({minDomainSegments: 2}).required(),
        language: joi.string().trim().min(2).max(5).regex(languageRegex).required()
    });

    const validationResult = joi.validate({
        email: UNSAFE_TRIMMED_EMAIL,
        language: UNSAFE_TRIMMED_LANGUAGE
    }, schema);

    if(validationResult.error) {
        console.log(JSON.stringify(validationResult.error, null, 2));
        return {
            statusCode: 400,
            body: JSON.stringify({
                error: true,
                error_message: "Invalid"
            })
        }
    }

    // Validation successful, change variable names to reflect
    const VALIDATED_EMAIL = UNSAFE_TRIMMED_EMAIL;
    const VALIDATED_LANGUAGE = UNSAFE_TRIMMED_LANGUAGE;

    // Cognito params
    // Username is the user's email
    // email is also required in UserAttributes in order to send confirmation
    // DesiredDeliveryMediums is required to send confirmation
    const params = {
        UserPoolId: userPoolId,
        Username: VALIDATED_EMAIL,
        UserAttributes: [
            {
                Name: "email",
                Value: VALIDATED_EMAIL
            },
            {
                Name: "custom:language",
                Value: VALIDATED_LANGUAGE
            } 
        ],
        DesiredDeliveryMediums: ["EMAIL"]
    }

    // Attempt to create user in Cognito
    try {
        const authRes = await Cognito.adminCreateUser(params).promise();
        console.log("Success: ", JSON.stringify(authRes, null, 2));
        return {
            statusCode: 200,
            body: JSON.stringify({
                success: true
            })
        }
    } catch(err) {
        console.log("Error: ", JSON.stringify(err, null, 2));
        return {
            statusCode: 400,
            body: JSON.stringify({
                error: true,
                error_message: err.message
            })
        }
    }
};

Running the tests, I get the expected error message when passing in badly formatted event data, and I get a Cognito error when attempting to create a user with the same email twice. Again, this is expected. However, when passing in a valid email with no users in the user pool I get the following as my response (formatted for readability):

Response:
{
  "statusCode": 400,
  "body": {
    "error": true,
    "error_message": "Invalid lambda function output : Invalid JSON"
  }
}

Checking in the Cognito User Pool that this function connects to, I see that a user has been successfully created. Yet, no email has been sent to the email address as was happening a few days ago.

All that is logged is information saying that I have an invalid JSON error, there is no authRes logged at all. When removing the call to Cognito and the corresponding console.log call, the try block runs successfully. So the issue is with the call to Cognito.

But why is this code failing today when it was working perfectly a few days ago? That is the part that is making me very frustrated.

Francefrancene answered 21/8, 2019 at 1:19 Comment(0)
F
10

The issue wasn't with this lambda function at all. It was an issue with AWS and the lambda function I was using as a Custom Message Trigger for Cognito User Pools. Here is what went wrong:

Per the AWS docs, the event data provided to the Custom Message Trigger lambda is of the following form for the adminCreateUser function call:

{
  "version": 1,
  "triggerSource": "CustomMessage_AdminCreateUser",
  "region": "<region>",
  "userPoolId": "<userPoolId>",
  "userName": "<userName>",
  "callerContext": {
      "awsSdk": "<calling aws sdk with version>",
      "clientId": "<apps client id>",
      ...
  },
  "request": {
      "userAttributes": {
          "phone_number_verified": false,
          "email_verified": true,
           ...
      },
      "codeParameter": "####",
      "usernameParameter": "username"
  },
  "response": {
      "smsMessage": "<custom message to be sent in the message with code parameter and username parameter>"
      "emailMessage": "<custom message to be sent in the message with code parameter and username parameter>"
      "emailSubject": "<custom email subject>"
  }
}

And it is expected that the data returned from the Custom Message Trigger lambda is of the same form as the event - only with the response object changed.

So this is what I wrote for the lambda:

const email_message = require("./email_message");

exports.handler = async (event) => {
    // Vars
    const {codeParameter, usernameParameter} = event.request;
    console.log("Cognito Event: ", event);

    // Check that codeParameter equals "####" and usernameParameter equals "username"
    // This is to ensure that no compromised values are entered into the html
    if(!(codeParameter === "####" && usernameParameter === "username")) {
        return null;
    }


    const newRes = {
        smsMessage: `Welcome: confirmation code is ${codeParameter} and username is ${usernameParameter}`,
        emailMessage: email_message({codeParameter, usernameParameter}),
        emailSubject: "Welcome To Our Site"
    }

    return {...event, response: newRes};
};

And this worked when tested a few days ago because the event object was of the form above. What had happened is that AWS sneakily changed the content of the codeParameter and usernameParameter fields to the following:

{
    ...
    "codeParameter": "{####}",
    "usernameParameter": "{username}",
    ...
}

So the lambda function was returning null as these strings didn't pass validation - and null isn't valid JSON.

So the temporary fix is to validate these new strings instead. However, this raises some concerns. Why is AWS changing the event object without so much as an update to the docs all of a sudden? Second, how should I validate that these strings are safe to inject in a customer's email address? I know that I can sanitize the usernameParameter but how about the codeParameter since it could very likely contain dangerous characters such as < > & ' " since it is a password generated with random symbols? If generating the password myself I can be sure that it won't contain data from a malicious actor so there is no need to sanitize. But if it's coming from AWS, who's to say that somehow these values aren't compromised? Hence why I added the validation step in the first place that was supposed to fail in the case those values had been changed. Which is exactly what happened.

So in short, all of my code behaved as expected. AWS changed their event object all of a sudden without notice.

Francefrancene answered 21/8, 2019 at 3:38 Comment(2)
Hi, I am facing an identical issue and I've posted this question - #67161429 I'd like to know how you went about finding what changed within the response object. How do I troubleshoot my problem and identify the specific issue?Hour
Official AWS documentation for this is still terrible. This is the only place on the Internet with this info, thank you for beating your head against the desk for the rest of us!Rimma
E
6

For me the issue was caused by not using async function as the handler. As soon as I added async (or just returning a promise would work as well), the error disappeared

Enteric answered 9/5, 2023 at 12:8 Comment(2)
Oh seriously? WTF AWS.Pavis
Yes, this is legit when using ES Modules Lambda as in index.mjs and export const handler = async (event) { return { status: 200, body: "Hello Jello"}; }Scharf
S
1

For me, it turned out that one of my coworkers was experimenting with custom authorizers as that's a next step we need to take in our application. He took a Lambda that had nothing to do with custom authorization and added it to the Cognito User pool > General settings > Triggers > Pre authentication and Post authentication fields to watch that the lambda was being triggered in CloudWatch but then didn't remove those from the user pool.

Obviously, the lambda not providing any sort of pre/post authorization caused an invalid response sent back from Lambda, resulting in the "Invalid JSON" response.

Specifically, the response received in the developer console was "400 Bad Request" with a response of:

{
  "__type":"InvalidLambdaResponseException",
  "message":"Invalid lambda function output : Invalid JSON"
}

which clued me into digging around within Cognito after I verified no one had changed the built-in Amplify authentication lambda, thanks to the answer by the_new.

Seattle answered 12/10, 2021 at 4:28 Comment(0)
S
0

I suffered with a similar problem for a long time. I was confused that one lambda from the sam-application started without problems, and the other gave such an error. It turned out that you need to increase the timeout. The default timeout is 3 seconds. Try increasing it to 10 seconds or more. I put it with a margin of 20 seconds, since Lambda still dies after execution

Sheehan answered 18/12, 2021 at 2:15 Comment(1)
Any idea why this solves it? I increased it from 3 to 10, I hope this solves my issue. I would like to understand this better though, if you can clue me in further.Staats

© 2022 - 2024 — McMap. All rights reserved.