Modify middleware response

Question

My requirement: write a middleware that filters all "bad words" out of a response that comes from another subsequent middleware (e.g. Mvc).

The problem: streaming of the response. So when we come back to our FilterBadWordsMiddleware from a subsequent middleware, which already wrote to the response, we are too late to the party... because response started already sending, which yields to the wellknown error response has already started...

So since this is a requirement in many various situations -- how to deal with it?

Unbrace

Answer 1

.NET Core 3+ solution with proper resource handling:

Replace response stream by MemoryStream to prevent its sending. Return the original stream after the response is modified:

public async Task Invoke(HttpContext context)
{
    var response = context.Response;

    //uncomment this line to re-read context.Request.Body stream
    //context.Request.EnableBuffering();
    
    var originBody = response.Body;
    using var newBody = new MemoryStream();
    response.Body = newBody;

    await _next(context);

    await ModifyResponseAsync(response);
    
    newBody.Seek(0, SeekOrigin.Begin);
    await newBody.CopyToAsync(originBody);
    response.Body = originBody;
}

private async Task ModifyResponseAsync(HttpResponse response)
{
    var stream = response.Body;
    
    //uncomment to re-read the response stream
    //stream.Seek(0, SeekOrigin.Begin);
    using var reader = new StreamReader(stream, leaveOpen: true);
    string originalResponse = await reader.ReadToEndAsync();
    
    //add modification logic

    string modifiedResponse = "Hello from Stackoverflow";
    stream.SetLength(0);
    using var writer = new StreamWriter(stream, leaveOpen: true);
    await writer.WriteAsync(modifiedResponse);
    await writer.FlushAsync();
    response.ContentLength = stream.Length;
}

Original .NET Core 1 answer

Replace response stream by MemoryStream to prevent its sending. Return the original stream after the response is modified:

    public async Task Invoke(HttpContext context)
    {
        bool modifyResponse = true;
        Stream originBody = null;

        if (modifyResponse)
        {
            //uncomment this line only if you need to read context.Request.Body stream
            //context.Request.EnableRewind();

            originBody = ReplaceBody(context.Response);
        }

        await _next(context);

        if (modifyResponse)
        {
            //as we replaced the Response.Body with a MemoryStream instance before,
            //here we can read/write Response.Body
            //containing the data written by middlewares down the pipeline 

            //finally, write modified data to originBody and set it back as Response.Body value
            ReturnBody(context.Response, originBody);
        }
    }

    private Stream ReplaceBody(HttpResponse response)
    {
        var originBody = response.Body;
        response.Body = new MemoryStream();
        return originBody;
    }

    private void ReturnBody(HttpResponse response, Stream originBody)
    {
        response.Body.Seek(0, SeekOrigin.Begin);
        response.Body.CopyTo(originBody);
        response.Body = originBody;
    }

It's a workaround and it can cause performance problems. I hope to see a better solution here.

Adjunct

This works, thanks! I found that in some cases attaching a callback to context.Response.OnStarting() works as well, but not when modifying responses. Also I don't like using OnStarting(), because it breaks the iterative middleware workflow. – Unbrace

For future reference, before writing to Response, it helps to set the Content Length property. – Lengthwise

In Dotnet Core 3, I am getting "System.InvalidOperationException: Response Content-Length mismatch" to solve this I added context.Response.ContentLength = json.Length; – Lxx

Thanks you all for noticing! I've added a relevant solution to the answer. Works fine on .NET 7. – Adjunct

Why the originalResponse is always an empty string? How can I modify, not the replace response body? – Spiker

I found solution: stream.Seek(0, SeekOrigin.Begin); should be added before using var reader = new StreamReader(stream, leaveOpen: true);. – Spiker

@Spiker this means the stream has already been read before getting to rewriting middleware, which does't happen by default. I've updated the answer with optional Seek call. Thanks for the comment, and sorry for the extremely late reply. – Adjunct

I also found it strange, but it didn't work without this line. Thanks for the answer! – Spiker

Answer 2

A simpler version based on the code I used:

/// <summary>
/// The middleware Invoke method.
/// </summary>
/// <param name="httpContext">The current <see cref="HttpContext"/>.</param>
/// <returns>A Task to support async calls.</returns>
public async Task Invoke(HttpContext httpContext)
{
    var originBody = httpContext.Response.Body;
    try
    {
        var memStream = new MemoryStream();
        httpContext.Response.Body = memStream;

        await _next(httpContext).ConfigureAwait(false);

        memStream.Position = 0;
        var responseBody = new StreamReader(memStream).ReadToEnd();

        //Custom logic to modify response
        responseBody = responseBody.Replace("hello", "hi", StringComparison.InvariantCultureIgnoreCase);

        var memoryStreamModified = new MemoryStream();
        var sw = new StreamWriter(memoryStreamModified);
        sw.Write(responseBody);
        sw.Flush();
        memoryStreamModified.Position = 0;

        await memoryStreamModified.CopyToAsync(originBody).ConfigureAwait(false);
    }
    finally
    {
        httpContext.Response.Body = originBody;
    }
}

Nasty

Unfortunately I do get a System.InvalidOperationException: Response Content-Length mismatch: too few bytes written. – Celeriac

@JonneKleijer use context.Response.ContentLength = responseBody.Length; after memoryStreamModified.Position = 0; like Oriel Dayanim suggested in the comment – Masonmasonic

Why are you using ".ConfigureAwait(false)"? The code seems to work with out it? – Bowne

Answer 3

Unfortunately I'm not allowed to comment since my score is too low. So just wanted to post my extension of the excellent top solution, and a modification for .NET Core 3.0+

First of all

context.Request.EnableRewind();

has been changed to

context.Request.EnableBuffering();

in .NET Core 3.0+

And here's how I read/write the body content:

First a filter, so we just modify the content types we're interested in

private static readonly IEnumerable<string> validContentTypes = new HashSet<string>() { "text/html", "application/json", "application/javascript" };

It's a solution for transforming nuggeted texts like [[[Translate me]]] into its translation. This way I can just mark up everything that needs to be translated, read the po-file we've gotten from the translator, and then do the translation replacement in the output stream - regardless if the nuggeted texts is in a razor view, javascript or ... whatever. Kind of like the TurquoiseOwl i18n package does, but in .NET Core, which that excellent package unfortunately doesn't support.

...

if (modifyResponse)
{
    //as we replaced the Response.Body with a MemoryStream instance before,
    //here we can read/write Response.Body
    //containing the data written by middlewares down the pipeline

    var contentType = context.Response.ContentType?.ToLower();
    contentType = contentType?.Split(';', StringSplitOptions.RemoveEmptyEntries).FirstOrDefault();   // Filter out text/html from "text/html; charset=utf-8"

    if (validContentTypes.Contains(contentType))
    {
        using (var streamReader = new StreamReader(context.Response.Body))
        {
            // Read the body
            context.Response.Body.Seek(0, SeekOrigin.Begin);
            var responseBody = await streamReader.ReadToEndAsync();

            // Replace [[[Bananas]]] with translated texts - or Bananas if a translation is missing
            responseBody = NuggetReplacer.ReplaceNuggets(poCatalog, responseBody);

            // Create a new stream with the modified body, and reset the content length to match the new stream
            var requestContent = new StringContent(responseBody, Encoding.UTF8, contentType);
            context.Response.Body = await requestContent.ReadAsStreamAsync();//modified stream
            context.Response.ContentLength = context.Response.Body.Length;
        }
    }

    //finally, write modified data to originBody and set it back as Response.Body value
    await ReturnBody(context.Response, originBody);
}
...

private Task ReturnBody(HttpResponse response, Stream originBody)
{
    response.Body.Seek(0, SeekOrigin.Begin);
    await response.Body.CopyToAsync(originBody);
    response.Body = originBody;
}

Hannis

What does ReturnBody do? – Terret

Sorry, added the RetunBody-function to the solution suggestion – Hannis

Answer 4

A "real" production scenario may be found here: tethys logging middeware

If you follow the logic presented in the link, do not forget to addhttpContext.Request.EnableRewind() prior calling _next(httpContext) (extension method of Microsoft.AspNetCore.Http.Internal namespace).

Fidel

Answer 5

If you are using MVC you can try filters. They seem to allow changing the response.

Pyjamas