ASP.Net httpruntime executionTimeout not working (and yes debug=false)

Question

We just recently noticed that executionTimeout has stopped working on our website. It was definitely working ~last year ... hard to say when it stopped.

We are currently running on:

• Windows-2008x64
• IIS7
• 32bit binaries
• Managed Pipeline Mode = classic
• Framework version = v2.0

Web.Config has

<compilation defaultLanguage="vb" debug="false" batch="true">
<httpRuntime executionTimeout="90" />

Any hints on why we are seeing Timetaken all the way up to ~20 minutes. Would compilation options for DebugType (full vs pdbonly) have any effect?

datetime       timetaken httpmethod Status  Sent    Received<BR>
12/19/10 0:10  901338    POST       302 456 24273<BR>
12/19/10 0:18  1817446   POST       302 0   114236<BR>
12/19/10 0:16  246923    POST       400 0   28512<BR>
12/19/10 0:12  220450    POST       302 0   65227<BR>
12/19/10 0:22  400150    GET        200 180835  416<BR>
12/19/10 0:20  335455    POST       400 0   36135<BR>
12/19/10 0:57  213210    POST       302 0   51558<BR>
12/19/10 0:48  352742    POST       302 438 25802<BR>
12/19/10 0:37  958660    POST       400 0   24558<BR>
12/19/10 0:06  202025    POST       302 0   58349<BR>

Chalky

Is it possible another web.config or <location> tag deeper within your application is overriding this? What about HTTP modules that might be changing it programatically? – Medullated

Shot in the dark: Are there any other web.config files in the project. One in a child directory can override the one you are currently looking at. – Hard

Another idea: Try printing HttpContext.Current.Server.ScriptTimeout from an affected page and see if it matches what you expect. – Medullated

I'm having the exact same issue. And I've debugged into it to see that, yes, Server.ScriptTimeout does match what was set in the config. httpRuntime is not configured in any other web.config file. Requests are running until they hit the site wide connectionTimeout. – Infantine

Any luck on getting this fixed? I'm having a similar problem. I'm printing Server.ScriptTimeout on the page and it is indeed the value I have set in the Web.config, but the timeout still occurs after the "magical" 90 seconds. – Inconstant

Answer 1

Execution timeout and time-taken time two different things. Although, the size of the discrepancy is troubling.

time-taken includes all of the network time in the request/response (under certain conditions.). The network transfer time easily outstrips the amount of time a request really takes. Though, normally, I'm used to just seconds of difference not minutes.

Execution timeout refers only to the amount of time the worker process spent processing the request; which is just a subset of time-taken. It only applies if the debug attribute is set to false; which it looks like you have.

Of course, assuming the first request you listed took the full 90 seconds of allowed time out, that still leaves 13.5 minutes left in the time-taken window to transfer essentially 24k of data. That sounds like a serious network issue.

So, either you have a serious transport issue or there is another web.config file somewhere in the tree where the requests are being processed that either sets debug to true or increases the execution timeout to something astronomical.

Another possibility is that the page itself has either the debug attribute set or it's own timeout values.

Earleneearley

Based on Chris' comments and some MS articles, I see that [executiontimeout] is just time inside the CLR/ASP.NET. However (and this may just be semantics), I caught an error this weekend where the CLR had the request for ~30 minutes. We have a context wrapper that accompanies every request through the platform. Here are the timestamps from the log entries surrounding the threatabortexception. [Info] 07:53:48 MXLocale is US {USEnglish} [Err] 08:25:00 Exception()-> System.Threading.ThreadAbortException – Chalky

(continuation on above comment) TAE was thrown on this line: oRequestFormColl = Request.Form Any insight on Request.Form taking ~30 minutes? – Chalky

@cbcolin: The ONLY reason I can think for access to Request.Form to take such a long time is if fairly large files are being uploaded to the page. But that's grasping at straws as I would think the time would have been spent in a earlier process. What data is being sent to this page? – Earleneearley

Answer 2

I have a theory but I'm not sure how to prove it. I've done something similar to cbcolin and logged the time when the request starts from within the BeginRequest event handler. Then when the request times out (1 hour later in our case) it is logged in the database and a timestamp recorded.

So here is the theory: ASP.NET only counts time that the thread is actually executing, not time that it is asleep.

So after BeginRequest the thread goes to sleep until the entire POST body is received by IIS. Then the thread is woken up to do work and the executionTimeout clock starts running. So time spent in the network transmission phase is not counted against the executionTimeout. Eventually the site wide Connection Timeout is hit and IIS closes the connection, resulting in an exception in ASP.NET.

BeginRequest and even PreRequestHandlerExecute all get called before the POST body is transferred to the web server. Then there is a long gap before the request handler is called. So it may look like .NET had the request for 30 minutes but the thread wasn't running that long.

I'm going to start logging the time that the request handler actually starts running and see if it ever goes over the limit I set.

Now as to control how long a request can stay in the transmittions phase like this on a per URL basis I have no idea. On a global level we can set minBytesPerSecond in webLimits for the application. There is no UI for it that I can find. This should kick ultra slow clients in the transmission phase.

That still wont solve the problem for DoS attacks that actually send data.

Infantine

Answer 3

I came across this article 2 days ago when I had the same problem. I tried everything, it worked on my local machine but did not work on the production server. Today, I have a workaround that fixes the problem and would like to share. Microsoft seems to not apply timeout to IHttpAsyncHandler and I take advantage of that. On my system, I only have 1 handler that is time-consuming, so this solution works for me. My handler code looks like this:

public class Handler1 : IHttpAsyncHandler
{
    public bool IsReusable
    {
        get { return true; }
    }

    public void ProcessRequest(HttpContext context)
    { }

    public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
    {
        //My business logic is here

        AsynchOperation asynch = new AsynchOperation(cb, context, extraData);
        asynch.StartAsyncWork();
        return asynch;
    }

    public void EndProcessRequest(IAsyncResult result)
    { }
}

And my helper class:

class AsynchOperation : IAsyncResult
{
    private bool _completed;
    private Object _state;
    private AsyncCallback _callback;
    private HttpContext _context;

    bool IAsyncResult.IsCompleted { get { return _completed; } }
    WaitHandle IAsyncResult.AsyncWaitHandle { get { return null; } }
    Object IAsyncResult.AsyncState { get { return _state; } }
    bool IAsyncResult.CompletedSynchronously { get { return false; } }

    public AsynchOperation(AsyncCallback callback, HttpContext context, Object state)
    {
        _callback = callback;
        _context = context;
        _state = state;
        _completed = false;
    }

    public void StartAsyncWork()
    {
        _completed = true;
        _callback(this);
    }
}

In this approach, we actually do not do anything asynchronously. AsynchOperation is just a fake async task. All of my business logic is still executed on the main thread which does not change any behavior of the current code.

Castroprauxel

Answer 4

IIS execution timeouts stopped working in (pre-Core) ASP.NET after the internal HTTP operations became asynchronous. When HTTP operations were synchronous, the runtime just watched to see how long a request's thread lived and then killed it if it took too long, and that worked because one thread was always tied to one request. But once the framework evolved to where the HTTP operations became asynchronous, that timeout model no longer worked because one thread ends up handling multiple requests, so you can't just safely kill a thread after such and such a time.

https://serverfault.com/questions/1068281/asp-net-iis-requests-not-timing-out-after-executiontimeout-value/1128469

Fibrilliform