AWS Lambda async concurrency limits

Question

I'm working on an AWS Lambda function that currently makes hundreds of API calls but when going into production it will make hundreds of thousands. The problem is that I can't test at that scale.

I'm using the async module to execute my api calls with async.eachLimit so that I can limit the concurrency (I currently set it a 300).

The thing that I don't understand is the limits on AWS Lambda. Here's what the docs say:

AWS Lambda Resource Limits per Invocation

• Number of file descriptors: 1,024
• Number of processes and threads (combined total): 1,024

As I understand it, Node.js is single threaded so I don't think I would exceed that limit. I'm not using child processes and the async library doesn't either so OK on that front too.

Now about those file descriptors, my function strictly calls the rest of AWS's API and I'm never writing to disk so I don't think I'm using them.

The other important AWS Lambda limits are execution time and memory consumed. Those are very clearly reported on each execution and I am perfectly aware when I'm close to reaching them or not, so let's ignore these for now.

A little bit of context:

The exact nature of my function is that every time a sports match starts I need to subscribe all mobile devices to the appropriate SNS topics, so basically I'm calling our own MySQL database and then the AWS SNS endpoint repeatedly.

So the question is...

How far can I push async's concurrency in AWS Lambda in this context? Are there any practical limits or something else that might come into play that I'm not considering?

Kidron

do you mean that a single lambda execution is going to make hundreds of thousands of calls? or multiple lambdas? – Umbra

@Umbra One Lambda execution makes thousands of calls inside it using async. So my backend calls the Lambda function and IT makes the thousands of API calls. – Kidron

node.js is not single threaded actually. The javascript is executed in a single thread but when you do IO, such as network requests, and disk operations, it will actually do that in a separate thread. So if you were to use async.map and for each item make a network request it will possibly create as many threads as requests you are making. – Uncertainty

Answer 1

As I understand it, Node.js is single threaded so I don't think I would exceed that limit. I'm not using child processes and the async library doesn't either so OK on that front too.

Node.js is event driven, not single threaded.
The Javascript engine runs on a single thread (the event loop) and delegates I/O operation to an internal library (libuv) which handles its thread pool and asynchronous operations.
async doesn't open a child process on its own, but behind the scenes, whether you're making an HTTP request or interacting with the file system, you're delegating these operations to libuv.

In other words, you've answered your own question well with the resources limits:

How far can I push async's concurrency in AWS Lambda in this context? Are there any practical limits or something else that might come into play that I'm not considering?

AWS Lambda Resource Limits per Invocation

• Number of file descriptors: 1,024
• Number of processes and threads (combined total): 1,024

It's hard to say whether libuv would open a new thread for each I/O operation, so you might get away with a little more than the numbers listed above. But you will probably run out or memory way before reaching those limits anyway.
The bottom line is no, you won't be able to make hundreds of thousands of calls in a single lambda execution.

Regarding the context of your function, depending on how often your job needs to run, you might want to refactor your lambda to multiple executions (it would also run faster), or have it on an EC2 with auto scaling triggered by lambda.

Umbra

I believe that these limits apply across executions of the same lambda function. So if you had two functions executing in parallel they would have a combined total of 1,024 still. I interpretted it as per-execution also but after a lot of testing the behavior is consistent with all lambdas of the same type sharing the limits. – Uncertainty