AWS Lambda: Async Calls outside handler (initialization section, invoke lambda)

Question

I would like to call an asynchronous function outside the lambda handler with by the following code:

var client;
(async () => {
    var result =  await initSecrets("MyWebApi");
    var secret = JSON.parse(result.Payload);
    client= new MyWebApiClient(secret.API_KEY, secret.API_SECRET); 
});

async function initSecrets(secretName) {
    var input = {
    "secretName" : secretName
    };
    var result = await lambda.invoke({
       FunctionName: 'getSecrets',
       InvocationType: "RequestResponse",
       Payload: JSON.stringify(input)
    }).promise();
    return result;
}

exports.handler = async function (event, context) {

    var myReq = await client('Request');
    console.log(myReq);
};

The 'client' does not get initialized. The same code works perfectly if executed within the handler. initSecrets contains a lambda invocation of getSecrets() which calls the AWS SecretsManager Has anyone an idea how asynchronous functions can be properly called for initialization purpose outside the handler?

Thank you very much for your support.

Skillet

@Dennis Bauszus has the right solution way below. You can get your promise in the section outside your lambda handler and then assign it to a new local variable and "await" on it. That way it only gets initialized once and the await will make sure it has completed. – Scud

Answer 1

I ran into a similar issue trying to get next-js to work with aws-serverless-express.

I fixed it by doing the below (using typescript so just ignore the :any type bits)

const appModule = require('./App');
let server: any = undefined;

appModule.then((expressApp: any) => {
  server = createServer(expressApp, null, binaryMimeTypes);
});

function waitForServer(event: any, context: any){
  setImmediate(() => {
    if(!server){
      waitForServer(event, context);
    }else{
      proxy(server, event, context);
    }
  });
}

exports.handler = (event: any, context: any) => {
  if(server){
    proxy(server, event, context);
  }else{
    waitForServer(event, context);
  }
}

So for your code maybe something like

var client = undefined;

initSecrets("MyWebApi").then(result => {
    var secret = JSON.parse(result.Payload);
    client= new MyWebApiClient(secret.API_KEY, secret.API_SECRET)
})

function waitForClient(){
  setImmediate(() => {
    if(!client ){
      waitForClient();
    }else{
      client('Request')
    }
  });
}

exports.handler = async function (event, context) {
  if(client){
    client('Request')
  }else{
    waitForClient(event, context);
  }
};

Corvette

Thank you very much, Will, for that proposal. That should make it. Meanwhile, I added var init = 0 in the global context to detect cold starts. – Skillet

No problem, interesting idea to detect cold starts versus a pre-warm invocation. – Corvette

Answer 2

client is being called before it has initialised; the client var is being "exported" (and called) before the async function would have completed. When you are calling await client() the client would still be undefined.

edit, try something like this

var client = async which => {
    var result =  await initSecrets("MyWebApi");
    var secret = JSON.parse(result.Payload);
    let api = new MyWebApiClient(secret.API_KEY, secret.API_SECRET); 
    return api(which) // assuming api class is returning a promise
}

async function initSecrets(secretName) {
    var input = {
    "secretName" : secretName
    };
    var result = await lambda.invoke({
       FunctionName: 'getSecrets',
       InvocationType: "RequestResponse",
       Payload: JSON.stringify(input)
    }).promise();
    return result;
}

exports.handler = async function (event, context) {

    var myReq = await client('Request');
    console.log(myReq);
};

Gonorrhea

Thank you very much, Tobin, for your explanation. So, how can I proceed to get the client initialized outside the handler, one time, at construction time of the module? – Skillet

Answer 3

This can be also be solved with async/await give Node v8+

You can load your configuration in a module like so...

const fetch = require('node-fetch');

module.exports = async () => {

  const config = await fetch('https://cdn.jsdelivr.net/gh/GEOLYTIX/public/z2.json');

  return await config.json();

}

Then declare a _config outside the handler by require / executing the config module. Your handler must be an async function. _config will be a promise at first which you must await to resolve into the configuration object.

const _config = require('./config')();

module.exports = async (req, res) => {

  const config = await _config;

  res.send(config);

}

Globefish

Answer 4

Ideally you want your initialization code to run during the initialization phase and not the invocation phase of the lambda to minimize cold start times. Synchronous code at module level runs at initialization time and AWS recently added top level await support in node14 and newer lambdas: https://aws.amazon.com/blogs/compute/using-node-js-es-modules-and-top-level-await-in-aws-lambda/ . Using this you can make the init phase wait for your async initialization code by using top level await like so:

const sleep = ms => new Promise(resolve => setTimeout(resolve, ms))

console.log("start init");
await sleep(1000);
console.log("end init");

export const handler = async (event) => {
    return {
        statusCode: 200,
        body: JSON.stringify('Hello from Lambda!'),
    };
};

This works great if you are using ES modules. If for some reason you are stuck using commonjs (e.g. because your tooling like jest or ts-node doesn't yet fully support ES modules) then you can make your commonjs module look like an es module by making it export a Promise that waits on your initialization rather than exporting an object. Like so:

const sleep = ms => new Promise(resolve => setTimeout(resolve, ms))

const main = async () => {
    console.log("start init");
    await sleep(1000);
    console.log("end init");

    const handler = async (event) => {
        return {
            statusCode: 200,
            body: JSON.stringify('Hello from Lambda!'),
        };
    };
    return { handler };
};

# note we aren't exporting main here, but rather the result 
# of calling main() which is a promise resolving to {handler}:
module.exports = main();

Aggy