What happens to the 513th concurrent request to nginx?

Question

Nginx is event based and by default runs as 1 process supporting max 512 concurrent connections. Another process would allow another 512 concurrent connections.

I'm using nginx as a reverse proxy to a kestrel server. Kestrel supports unlimited connections.

Assuming 512 requests are concurrently processing, when the 513th request comes in then will nginx reject it or will it get added to a queue?

Binky

If all the HTTP requests come form different TCP connections (e.g. 512 requests and 512 connections), then the accepted answer is correct. Otherwise some of the HTTP requests share the same TCP connection (possible in HTTP/2) , that means the number of the TCP connections doesn't exceed the limit, in such case your nginx server can still accept new TCP connection. – Funny

Answer 1

According to the docs

Sets the maximum number of simultaneous connections that can be opened by a worker process.

and the source of ngx_get_connection() it'll simply emit a warning and won't open a new connection:

"%ui worker_connections are not enough"

so there doesn't appear do be any queueing.

And if the returned value is NULL (it is for this case), then it'll simply close the socket(if there's any), so won't handle the connection, but will kill it instead. The same behavior appears to be consistent between UDP and other types as well.

Regarding the "unlimited," I'd be highly skeptical about anything that mentions it. One type of the resources will eventually not be enough be it CPU utilization, memory, filesystem resources or whatever else, so if the library/application mentions "unlimited" go check what does it really mean and what happens when such "unlimited" thing can't be unlimited anymore. Then, you'll find the metrics according to which you can adjust Nginx config to match (or be a bit higher, so it can handle all incoming) connections.

I did a quick search and found microsoft.aspnetcore.server.kestrel.core.kestrelserverlimits.maxconcurrentconnections and its related ConnectionLimitMiddleware which if enabled seems to do some kind of queueing of connections and if disabled I'd guess it'll be limited by the OS's underlying socket limitations, however I found only very brief references to Windows' socket buffer size (TCP), thus you'll need to dig deeper.

For testing purposes, simply use the lowest possible value of worker_connections for Nginx until it starts and keeps running, similarly for Kestrel and see how it behaves when:

• Kestrel > Nginx
• Kestrel == Nginx
• Kestrel < Nginx

with opening sockets against the server and send a byte or two once in a while so the socket is busy.

Taps

I read some code in ngx_get_connection() and its callers and the directive worker_connections. Is this function invoked by all worker processes ? so when Nginx runs out of free connection objects, ngx_get_connection() simply returns NULL to the worker process currently invoking it, that matches the description in worker_connections of the documentation – Funny