How do I get the incoming request body and the outgoing response body in ASP.NET Core middleware?

Question

I'm writing some middleware for an ASP.NET Core 7 minimal API project.

At this stage, I would just like to be able to read the request body being sent in, and then the response body that is being sent out, and dump them to the console.

Reading the request seems easy enough. My middleware contains the following...

  public async Task InvokeAsync(HttpContext httpContext) {
    try {
      httpContext.Request.EnableBuffering();
      string requestBody = await new StreamReader(httpContext.Request.Body, Encoding.UTF8).ReadToEndAsync();
      httpContext.Request.Body.Position = 0;
      Console.WriteLine($"Request body: {requestBody}");
    } catch (Exception ex) {
      Console.WriteLine($"Exception reading request: {ex.Message}");
    }

    await _next(httpContext);
  }

This works fine. However, when I try to read the response body, I get an exception. I tried inserting the following (similar to what's found on many SO answers and blog posts)...

    try {
      using StreamReader reader = new(httpContext.Response.Body);
      httpContext.Response.Body.Seek(0, SeekOrigin.Begin);
      string responseBody = await reader.ReadToEndAsync();
      httpContext.Response.Body.Seek(0, SeekOrigin.Begin);
      Console.WriteLine($"Response body: {responseBody}");
    } catch (Exception ex) {
      Console.WriteLine($"Exception reading response: {ex.Message}");
    }

However, this gives an exception System.ArgumentException: Stream was not readable.

One thing that confuses me is that according to the Microsoft docs, middleware is called twice, once when the request comes in (before the specific endpoint code has been called), and a second time when the response is being sent out (after the endpoint code has done its stuff). I can see that my middleware is only being called when the request comes in, so even if the code above worked, I don't see how it would get the response, as that hasn't been generated by the endpoint code yet.

I've searched around, but haven't found anywhere that clarifies this point.

Anyone able to advise what I'm doing wrong? Thanks

Iover

Answer 1

middleware is called twice, once when the request comes in (before the specific endpoint code has been called), and a second time when the response is being sent out

While this is impression which can be formed based on the article and pictures in the docs, it is not fully correct one, middlewares are called only one time (per registration in the pipeline). More correct representation of the handling pipeline would be Russian nesting doll - the first registered middleware calls the second, etc. The await _next(httpContext); is the call to the following middleware in the pipeline, so if needed you can do stuff before and after it.

You can try something like this - substitute the response stream by one controlled by yourself and then write everything back (I use just Use to register middleware but you can adapt it to the approach with separate class):

app.Use(async (httpContext, next) =>
{
    try
    {
        httpContext.Request.EnableBuffering();
        string requestBody = await new StreamReader(httpContext.Request.Body, Encoding.UTF8).ReadToEndAsync();
        httpContext.Request.Body.Position = 0;
        Console.WriteLine($"Request body: {requestBody}");
    }
    catch (Exception ex)
    {
        Console.WriteLine($"Exception reading request: {ex.Message}");
    }

    Stream originalBody = httpContext.Response.Body;
    try
    {
        using var memStream = new MemoryStream();
        httpContext.Response.Body = memStream;

        // call to the following middleware 
        // response should be produced by one of the following middlewares
        await next(httpContext); 

        memStream.Position = 0;
        string responseBody = new StreamReader(memStream).ReadToEnd();

        memStream.Position = 0;
        await memStream.CopyToAsync(originalBody);
        Console.WriteLine(responseBody);
    }
    finally
    {
        httpContext.Response.Body = originalBody;
    }
});

Also I highly recommend to check out the HttpLoggingMiddleware from the framework-provided HTTP logging support which does exactly the same (and highly likely in more correct/performant way).

Spring

Thanks for the answer, this was the easiest one to implement, and works great. One question though, you clarified that middleware is only called once, not like the docs imply. If so, how is it possible to modify the request body (which presumably can only happen before the endpoint code has been called), as well as see/modify the response body (which presumably can only happen after the endpoint code is called)? – Iover

@AvrohomYisroel this is explained im my answer, your middleware calls the next one (await next(...)) and you can execute code before this call and after. One of the next midllewares will write the response body and you will be able to read it after await next(...). – Spring

Ah, that's why the call to next(httpContext) is after reading the request and before reading the response? I did wonder why you had it in the middle there, but didn't realise that was what was going on. Thanks for clarifying that. – Iover

Somehow I got response (in plain JSON) as "�\u0013\u0002\.... – Exothermic

@Exothermic without minimal reproducible example it is hard to tell. Maybe you have data compressed so you try reading request before decompressing it. – Spring

@GuruStron You right. It was due to ASP.NET Core using Content-Encoding:br by default...need to add the Brotli decompress – Exothermic

@Exothermic was glad to help! Feel free to upvote if answer itself have helped you ;) – Spring

Perfect just what I needed to finish my project – Always

Answer 2

this is what i wrote for response/request logging

startup:

app.UseRequestResponseLogging();

create a new file requestresponselogging.cs

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Microsoft.IO;
using System;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Common.API.Middleware
{

    /// <summary>
    /// 
    /// </summary>
    public class RequestResponseLoggingMiddleware
    {
        private readonly RequestDelegate _next;
        private readonly ILogger _logger;
        private readonly RecyclableMemoryStreamManager _recyclableMemoryStreamManager;
        internal RequestResponseData log { get; set; }

        /// <summary>
        /// 
        /// </summary>
        /// <param name="next"></param>
        /// <param name="loggerFactory"></param>
        public RequestResponseLoggingMiddleware(RequestDelegate next, ILoggerFactory loggerFactory)
        {
            _next = next;
            _logger = loggerFactory.CreateLogger<RequestResponseLoggingMiddleware>();
            _recyclableMemoryStreamManager = new RecyclableMemoryStreamManager();
        }

        /// <summary>
        /// 
        /// </summary>
        /// <param name="context"></param>
        /// <returns></returns>
        public async Task Invoke(HttpContext context)
        {
            //send to log if it real api call, ignore home page and swagger store only post put and delete
            if (context.Request.Path.ToString().Contains("/api/") && context.Request.Method != "GET")
            {

                log = new RequestResponseData()
                {
                    Application = Environment.GetEnvironmentVariable("Application"),
                    User = context.User.Identity.Name
                };
                await LogRequest(context);
                await LogResponse(context);
                _logger.LogInformation(log.ToString());

            }
            else
                await _next(context);
        }

        private async Task LogRequest(HttpContext context)
        {
            try
            {

                string headers = String.Empty;
                var requestBodyStream = new MemoryStream();

                foreach (var key in context.Request.Headers.Keys)
                {
                    headers += key + "=" + context.Request.Headers[key] + Environment.NewLine;
                }

                context.Request.EnableBuffering();

                await using var requestStream = _recyclableMemoryStreamManager.GetStream();
                await context.Request.Body.CopyToAsync(requestStream);

                log.RequestTimestamp = DateTime.Now;
                log.RequestUri = $"{context.Request.Scheme}://{context.Request.Host}{context.Request.Path}{context.Request.QueryString}";
                log.IpAddress = $"{context.Request.Host.Host}";
                log.RequestMethod = $"{context.Request.Method}";
                log.Machine = Environment.MachineName;
                log.RequestContentType = context.Request.ContentType;
                log.RequestBody = ReadStreamInChunks(requestStream);
                //log.RequestBody = FormatRequest(context.Request);
                log.RequestHeaders = headers;
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "request/response logging exception");
                throw;
            }
            context.Request.Body.Position = 0;

        }

        private async Task LogResponse(HttpContext context)
        {
            var originalBodyStream = context.Response.Body;

            await using var responseBody = _recyclableMemoryStreamManager.GetStream();
            context.Response.Body = responseBody;

            await _next(context);

            log.ResponseTimestamp = DateTime.Now;
            log.ResponseBody = await FormatResponse(context.Response);
            log.ResponseStatusCode = context.Response.StatusCode;

            await responseBody.CopyToAsync(originalBodyStream);
        }

        private static string ReadStreamInChunks(Stream stream)
        {
            const int readChunkBufferLength = 4096;
            stream.Seek(0, SeekOrigin.Begin);
            using var textWriter = new StringWriter();
            using var reader = new StreamReader(stream);
            var readChunk = new char[readChunkBufferLength];
            int readChunkLength;
            do
            {
                readChunkLength = reader.ReadBlock(readChunk, 0, readChunkBufferLength);
                textWriter.Write(readChunk, 0, readChunkLength);
            } while (readChunkLength > 0);
            return textWriter.ToString();
        }

        private string FormatRequest(HttpRequest request)
        {
            var body = request.Body;
            //request.EnableRewind();

            var buffer = new byte[Convert.ToInt32(request.ContentLength)];
            request.Body.ReadAsync(buffer, 0, buffer.Length);
            var bodyAsText = Encoding.UTF8.GetString(buffer);
            //request.Body = body;

            return $"{request.QueryString} {bodyAsText}";
        }

        private async Task<string> FormatResponse(HttpResponse response)
        {
            response.Body.Seek(0, SeekOrigin.Begin);
            var text = await new StreamReader(response.Body).ReadToEndAsync();
            response.Body.Seek(0, SeekOrigin.Begin);

            return $"{text}";
        }
    }

    #region Class 

    /// <summary>
    /// 
    /// </summary>
    public class RequestResponseData
    {
        /// <summary>
        /// 
        /// </summary>
        public string Application { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public DateTime? RequestTimestamp { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public string RequestUri { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public string RequestMethod { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public string RequestBody { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public string IpAddress { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public string RequestHeaders { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public string RequestContentType { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public DateTime? ResponseTimestamp { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public string ResponseBody { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public int ResponseStatusCode { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public string Machine { get; set; }
        /// <summary>
        /// 
        /// </summary>
        public string User { get; set; }

        /// <summary>
        /// 
        /// </summary>
        /// <returns></returns>
        public override string ToString()
        {
            string separator = string.Concat(Enumerable.Repeat("*", 50));
            var lb = "\r\n";
            return $"{lb}{separator}{lb}" +
                    $"Http Request Information:{lb}" +
                    $"Application:              {Application}{lb}" +
                    $"Machine:                  {Machine}{lb}" +
                    $"User:                     {User}{lb}" +
                    $"Request Timestamp:        {RequestTimestamp}{lb}" +
                    $"Request IpAddress:        {IpAddress}{lb}" +
                    $"Request Uri:              {RequestUri}{lb}" +
                    $"Request Method:           {RequestMethod}{lb}" +
                    $"RequestHeaders:           {RequestHeaders}{lb}" +
                    $"Request ContentType:      {RequestContentType}{lb}" +
                    $"Request ContentBody:      {RequestBody}{lb}" +
                    $"RequestHeaders:           {RequestHeaders}{lb}" +
                    $"{Environment.NewLine}Http Response Information:{Environment.NewLine}" +
                    $"ResponseStatusCode:   {ResponseStatusCode}{lb}" +
                    $"ResponseContentBody:  {ResponseBody}{lb}" +
                    $"{lb}{separator}{lb}{Environment.NewLine}";
        }
    }

    #endregion

    #region Middleware Extensions


    /// <summary>
    /// 
    /// </summary>
    public static class RequestResponseLoggingMiddlewareExtensions
    {
        /// <summary>
        /// 
        /// </summary>
        /// <param name="builder"></param>
        /// <returns></returns>
        public static IApplicationBuilder UseRequestResponseLogging(this IApplicationBuilder builder) => builder.UseMiddleware<RequestResponseLoggingMiddleware>();
    }
    #endregion

}

Irksome