"Response has already started" exception in a bare-bones ASP.NET Core middleware project

Question

My question is basically this -- given the code shown here, what else runs in the pipeline that is trying to start a response to the client? I'm aware of the other questions about that exception, but it seems something in the pipeline which runs after my middleware is causing the exception -- it isn't caused by my middleware, which I think is the difference in my scenario.

This is a bare-bones ASP.NET Core 3.0 WebSocket echo server -- no SignalR, no MVC, no routing, no static page support, etc. Apart from handling the sockets, when the middleware sees a request for text/html it sends back a simple page (a hard-coded string) as the echo client.

The browser receives the content just fine, and my exception handler is not triggered (a crucial point), but after my middleware is done processing the request, ASP.NET Core logs the exception:

StatusCode cannot be set because the response has already started.

The code is quite minimal:

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddTransient<WebSocketMiddleware>();
    }

    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        var webSocketOptions = new WebSocketOptions()
        {
            KeepAliveInterval = TimeSpan.FromSeconds(120),
            ReceiveBufferSize = 4 * 1024
        };
        app.UseWebSockets(webSocketOptions);
        app.UseMiddleware<WebSocketMiddleware>();
    }
}

public class WebSocketMiddleware : IMiddleware
{
    // fields/properties omitted
    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        try
        {
            if (context.WebSockets.IsWebSocketRequest)
            {
                // omitted, socket upgrade works normally
            }
            else
            {
                if(context.Request.Headers["Accept"][0].Contains("text/html"))
                {
                    // this works but causes the exception later in the pipeline
                    await context.Response.WriteAsync(SimpleHtmlClient.HTML);
                }
                else
                {
                    // ignore other requests such as favicon
                }
            }
        }
        catch (Exception ex)
        { 
            // code omitted, never triggered 
        }
        finally
        {
            // exception happens here
            await next(context);
        }
    }
}

I thought the problem might be my use of WriteAsync but it seems to happen if I also set the HTTP status elsewhere, with no other output, like setting HTTP 500 in the catch block. If I step through a purposely-caused exception by adding a throw as the very first statement in the middleware, it gets to the finally where the "already started" exception occurs.

So what else tries to produce output in the pipeline given that Startup class?

Edit: Stack trace, line 91 referenced at the end is the await next(context) in the finally block.

System.InvalidOperationException: StatusCode cannot be set because the response has already started. at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ThrowResponseAlreadyStartedException(String value) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.set_StatusCode(Int32 value) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.Microsoft.AspNetCore.Http.Features.IHttpResponseFeature.set_StatusCode(Int32 value) at Microsoft.AspNetCore.Http.DefaultHttpResponse.set_StatusCode(Int32 value) at Microsoft.AspNetCore.Builder.ApplicationBuilder.<>c.b__18_0(HttpContext context) at KestrelWebSocketServer.WebSocketMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) in C:\Source\WebSocketExample\KestrelWebSocketServer\WebSocketMiddleware.cs:line 91 at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass5_1.<b__1>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

Deguzman

One question, why are u registering the middle-ware with the service-provider? Never seen anyone doing that before. Have you tried removing it from the configure-services delegate? learn.microsoft.com/en-us/aspnet/core/fundamentals/middleware/… – Magnificat

It's what they call "factory-activated middleware." I personally dislike "magic methods" and would rather have my middleware implement an interface explicitly (IMiddleware in this case). learn.microsoft.com/en-us/aspnet/core/fundamentals/middleware/… – Deguzman

Once you have written to the stream the other middleware wont be able to. You need to replace the stream if you want to write and pass one down the pipeline. – Cavendish

Why are you even passing down stream. if you are writing to the response, don't you mean to short circuit the pipeline? – Cavendish

@Cavendish I've been wondering if that was the problem -- that there shouldn't be any next in this case. But circling back to the main question, what is executing when I call next given I haven't added anything else to the pipeline. Obviously ASP.NET itself does more under the hood -- but what would be trying to write in this scenario? Also, do you have any links that can clarify your earlier comment about replacing the stream? I appreciate the feedback. – Deguzman

The framework would be adding their own handlers. In this case you need to change when you call next. If you are writing to the response you should end it there and not call the next in the pipeline. – Cavendish

I suppose I should precede await next with if(!context.Response.HasStarted) since my fall-through case could theoretically be handled elsewhere. – Deguzman

Answer 1

The comment from user @Nkosi was correct -- when my middleware is able to handle the request fully (either upgrading HTTP to WS, or sending back the echo client HTML) it should not hand off context to the next delegate. However, in this very simple example other requests (such as a browser trying to retrieve favicon) my middleware doesn't do anything, so the solution was this:

finally
{
    if(!context.Response.HasStarted)
        await next(context);
}

Deguzman

It is mentioned many times in the docs here as short-circuiting. There is also a specific warning about not calling next.Invoke after the response has been sent to the client. – Cavendish