What is optimal value for Phusion passenger PassengerMaxRequestQueueSize

Question

I know this depends on the box hardware, but for example if there are set 100 processes, the default queue is also 100. Does it makes sense to increase PassengerMaxRequestQueueSize to 200 or 300? Probably this depends on free memory. Thoughts?

The best answer will be explaining the setting and probably one or two examples, assuming the server process requests for 2-3 seconds.

Thanks in advance!

Huynh

Answer 1

Why you should limit queuing

Any requests that aren't immediately handled by an application process, are queued. Queuing is usually is bad: it often means that your server cannot handle the requests quickly enough.

A larger queue means that requests are less likely to be dropped. But this comes with a drawback: during busy times, the larger the queue, the longer your visitors have to wait before they see a response. This causes them to click reload, making the queue even longer (their previous request will stay in the queue; the OS does not know that they've disconnected until it tries to send data back to the visitor), or causes them to leave in frustration.

So having a limit on the queue is a good thing. It limits the impact of the above situation.

You should ensure that requests are queued as little as possible. That could mean:

• Making your app faster (if your workload is CPU bound).
• Upgrading to faster hardware (if your workload is CPU bound).
• Increasing your app's concurrency settings (if your workload is I/O bound), e.g. by increasing the number of processes or threads.

If you cannot prevent requests from being queued, then the next best thing to do is to keep the queue short, and to display a friendly error message upon reaching the queue limit. Something like, "We're sorry, a lot of people are visiting us right now. Please try again later." The documentation for PassengerMaxRequestQueueSize tells you how to do that.

Optimal value for the queue size

It's hard to say what the optimal queue size should be. A good rule of thumb is: set the request queue size to the maximum number of requests you can handle in one second. Depending on your situation you may have to tweak things a little bit.

This rule of thumb comes from the notion of expected burst traffic. How many simultaneous requests do you expect on your server?

Suppose that your queue size is 100, and that for whatever reason you receive 150 requests at the same time. Suppose that your server is fast enough to handle 150 requests in half a second, so you know it's not a performance problem. But if you have a request queue size of 100, then 50 of those requests will be dropped with a "Request queue full" error.

In such a situation, you should set the queue size to the maximum number of concurrent requests that you think you can safely handle without performance issues.

Oneman

Is there are any possibility to set a callback on an event when request queue size is larger than passenger_max_request_queue_size and starting sending the 503 error to the incoming requests? I would like to have notification when such events are occures (for example, someone is trying to DOS server with a ab utility). – Rusty

Not at this moment, but a hook can be introduced for this event. I've filed a feature request here: – Oneman

It's worth noting that these recommendations assume the primary driver of traffic to your app is humans. If you are running an API that has a more predictable/regular request load, you can make more accurate calculations based on the highest number of requests you expect to receive at once, how long they take to process, and what an acceptable wait time is for the longest request. Respectable machines will wait 30 seconds without reloading, if you ask them to ;) – Bromo

Answer 2

This SO question and the Passenger docs here talk more about working with this. If you want more information about why this is happening on your server you can try running passenger-status (usually you need to run this as root).

If you would like to set a custom error page when visitors see this issue you can use the following (in Apache) to set a custom error page:

PassengerErrorOverride on
ErrorDocument 503  /error503.html

As mentioned by Hongli you can also change the setting PassengerMaxRequestQueueSize to a higher number to queue more requests. You can also set this to 0 and disable it (for most situations this is not an optimal solution however).

For reference, the default error message a visitor to your site will see when bumping against this limit is:

This website is under heavy load

We're sorry, too many people are accessing this website at the same time. We're working on this problem. Please try again later.

Vesicle

I believe setting it to 0 does not disable it, it sets it to infinite. Per the doc at phusionpassenger.com/library/config/nginx/reference/… "A value of 0 means that the queue is unbounded." – Beneficial