Owin on IIS web requests hang indefinitely

Question

We are running an Owin applications on IIS 8.5 on Win 2012 R2 behind a loadbalancer. One some occations, requests to certain URLs hang indefinitely. If the user selects cancel in the browser, and reloads the page, everything is OK and the server responds quickly.

In IIS manager, we can see the requests hanging: The hang seems to occur inside the Owin pipeline. We are running .NET 4.5.2 and the latest Owin packages.

Here's the code for the /api/whoami endpoint:

[Authorize]
public class WhoAmIController : ApiController
{
    [Route("whoami")]
    [HttpGet]
    public IHttpActionResult Whoami()
    {
        return Json(ClaimsPrincipal.Current.Identity.Name);
    }
}

An observation is that requests to this endpoint hangs in the AuthenticateRequest stage in the IIS pipeline, not PreExecuteRequestHandler which is the default IIS stage for the OWIN pipeline. Our only authentication method is cookie authentication, which executes in the AuthenticateRequest stage.

Any ideas?

Edit: adjusted screenshot

Blast

Are you using async await? – Gerome

@TomasJansson Yes, we use async await in the application. Whether it is used on the handling of the specific endpoints in question, I am not sure. The application is part custom code, and part IdentityServer3. – Blast

Ok, then you might have problem with the synchronization context. When doing an await you're not guaranteed to come back to the "correct" thread if you haven't specifed so. This talk from NNUG last year goes through the details: vimeo.com/108094560 – Gerome

The part about Synchronization context starts around 26:30, and basically what you need to do is probably to add ConfigureAwait(false). This causing the continuation after an await will continue on the thread pool thread. – Gerome

Answer 1

The problem was that we had code executing on the IIS event PreSendRequestHeaders, which apperently is bad according to http://www.asp.net/aspnet/overview/web-development-best-practices/what-not-to-do-in-aspnet,-and-what-to-do-instead#presend

Our intention was to adjust HTTP headers on the way out on all request. The fix was to move the code to the BeginRequest event.

Blast

Answer 2

Have you tried adding .ConfigureAwait(false) to your async calls?

Doing so will make the continuation continue on the "correct" thread.

Gerome

Thanks for the input. If there is an async/await, it is in the library code and not something we can get to. I'll update the question with a code sample. – Blast

@VidarKongsli I think you can configure the code when you call to the library code. It might be that this is not the issue though, but I I have experienced problems like this before (like this week) :) – Gerome

Anyway, thanks for the tip, altough that did not turn out to be the problem. We found some other weaknesses, scrutinizing the code for Task-related issues. – Blast