IIS7 Integrated Pipeline: Interaction between maxConcurrentRequestsPerCPU and requestsQueueLimit settings

Question

Firstly there's a great overview of the IIS7 HTTP request lifecycle and various settings that affect performance here:

ASP.NET Thread Usage on IIS 7.0 and 6.0

Very specifically though, in dotNet 4 the defaults for maxConcurrentRequestsPerCPU and requestsQueueLimit are set to 5000. E.g. equivalent to: (in aspnet.config):

<system.web>
   <applicationPool 
      maxConcurrentRequestsPerCPU="5000" 
      maxConcurrentThreadsPerCPU="0" 
      requestQueueLimit="5000" /> (** see note below)
</system.web>

Seems to me that on a multi-CPU/core server the requestQueueLimit here will always be invoked well berfore the 'perCPU' limit. Thus, if a max of 5000 requests per CPU is what you actually want then I would expect that the requestQueueLimit needs to be increased to 5000 * CPUCount or just disabled altogether.

Is my interpretation correct? If so can I disable requestQueueLimit? (set it to zero?). The documentation on this setting doesn't appear to address this question (so maybe I'm missing something or misreading?)

** side note from the above article: The requestQueueLimit is poorly named. It actually limits the maximum number of requests that can be serviced by ASP.NET concurrently. This includes both requests that are queued and requests that are executing. If the "Requests Current" performance counter exceeds requestQueueLimit, new incoming requests will be rejected with a 503 status code)

Constrictor

I have asked Thomas (author of the post you point to) to comment on this. – Tifanie

Answer 1

***Is my interpretation correct?

Yes, if you want to execute more than 5000 requests concurrently, you'll need to increase the requestQueueLimit. The requestQueueLimit restricts the total number of requests in the system. Due to its legacy, it is actually the total number of requests in the system, and not the number of requests in some queue. It's goal is to prevent the server from toppling over due to lack of physical memory, virtual memory, etc. When the limit is reached, incoming requests will receive a quick 503 "Server Too Busy" response. By the way, the current number of requests in the system is exposed by the "ASP.NET\Requests Current" performance counter.

***can I disable requestQueueLimit? (set it to zero?)

You can effectively disable it by setting it to a large value, like 50000. You must set the value in the aspnet.config fileI doubt your server can handle 50000 concurrent requests, but if so, then double that. Setting it to zero will not disable it...oddly, it means no more than one request can execute concurrently.

By the way, it looks like there is a bug in v4. For integrated mode, it only successfully reads the value of requestQueueLimit if it is configured in the aspnet.config file as described on MSDN. For some reason, v4 was not reading it from machine.config when I experimented with it a little bit ago.

Aficionado

Thanks for this. Perhaps you could clarify one final point for me. If the total requests in-progress exceed requestQueueLimit then the client gets a '503 Server Too Busy' - OK. Will I also get a 503 if maxConcurrentRequestsPerCPU is exceeded /before/ requestQueueLimit? That is, are they both gating the number of requests more or less at the same point in the request handling lifecycle? Which from what I've read is a legacy of how IIS and ASP.NET have evolved. – Constrictor

You will not get a 503 if maxConcurrentRequestsPerCPU is exceeded before requestQueueLimit. If maxConcurrentRequestsPerCPU is exceeded before requestQueueLimit, the request will be added to a queue. It will be dequeued as soon as one of the executing requests completes. – Aficionado

Apologies but still not 100% clear... I got the impression from your blog that the CLR ThreadPool queue would queue requests if maxConcurrentRequestsPerCPU * CPUCount was greater than maxWorkerThreads * CPUCount. Hence, I assumed the ThreadPool Q /was/ the global Q you referred to in your blog. Seems that you are saying that there is another queue (legacy from IIS6?) apart from the ThreadPool Q? – Constrictor

In integrated mode, global queue is native strucutre. From beginning...HTTP receives request and posts to IIS CP. IIS picks up request on CP thread and calls into ASP.NET. ASP invokes QUWI to post to CLR TP, if requestQueueLimit not exceeded. A native callback (in webengine.dll) picks up request on CLR worker thread, we compare maxConcurrentRequestsPerCPU * CPUCount to total active requests. If we've exceeded limit, request is inserted in global queue (native code). Otherwise, it will be executed. If it was queued, it will be dequeued when one of the active requests completes. – Aficionado

Thanks Thomas, this has been extremely useful. We were also just wonderign about the minFreeThreads setting(s) that I believe no longer apply in the integrated pipeline - are there similar settings we should be setting or ways or do we not have to worry about thread exhaustion? My guess at this stage would be that maxConcurrentRequestsPerCPU should be set /lower/ than maxWorkerThreads to ensure ample threads for processign each request. However the defaults are maxConcurrentRequestsPerCPU=5000 and maxWorkerThreads is much lower (100 or 250 I think). – Constrictor

The minFreeThreads setting applies to classic mode. The analogy for integrated mode would be maxConcurrentThreadsPerCPU, which is disabled (i.e. has value of 0) by default in integrated mode. Please take this discussion over to my blog if you would like to continue. The default settings work for most folks, but not everyone. It depends on what you're doing. Ideally we would tune automatically, but we're not there yet. – Aficionado

Answer 2

You might want to check this application's source code. IIS tuner is an open source application which optimizes IIS settings for better performance. On the other hand this page could be useful for your questions.

Wandis