Http Servlet request lose params from POST body after read it once
Asked Answered
U

13

110

I'm trying to access two http request parameters in a Java Servlet filter, nothing new here, but was surprised to find that the parameters have already been consumed! Because of this, they are not available in the filter chain anymore.

It seems that this only occurs when parameters come in a POST request body (a form submit, for example).

Is there a way to read the parameters and NOT consume them?

So far I've found only this reference: Servlet Filter using request.getParameter loses Form data.

Thanks!

Usual answered 18/4, 2012 at 13:40 Comment(2)
maybe show a code fragment of how you're doing it?Consignor
Did you get getInputStream() or getReader()? Seems like they are the ones which will interfere with the execution of getParameter()Paramatta
N
135

As an aside, an alternative way to solve this problem is to not use the filter chain and instead build your own interceptor component, perhaps using aspects, which can operate on the parsed request body. It will also likely be more efficient as you are only converting the request InputStream into your own model object once.

However, I still think it's reasonable to want to read the request body more than once particularly as the request moves through the filter chain. I would typically use filter chains for certain operations that I want to keep at the HTTP layer, decoupled from the service components.

As suggested by Will Hartung I achieved this by extending HttpServletRequestWrapper, consuming the request InputStream and essentially caching the bytes.

public class MultiReadHttpServletRequest extends HttpServletRequestWrapper {
  private ByteArrayOutputStream cachedBytes;

  public MultiReadHttpServletRequest(HttpServletRequest request) {
    super(request);
  }

  @Override
  public ServletInputStream getInputStream() throws IOException {
    if (cachedBytes == null)
      cacheInputStream();

      return new CachedServletInputStream(cachedBytes.toByteArray());
  }
    
  @Override
  public BufferedReader getReader() throws IOException{
    return new BufferedReader(new InputStreamReader(getInputStream()));
  }

  private void cacheInputStream() throws IOException {
    /* Cache the inputstream in order to read it multiple times. For
     * convenience, I use apache.commons IOUtils
     */
    cachedBytes = new ByteArrayOutputStream();
    IOUtils.copy(super.getInputStream(), cachedBytes);
  }

 
  /* An input stream which reads the cached request body */
  private static class CachedServletInputStream extends     ServletInputStream {

    private final ByteArrayInputStream buffer;

    public CachedServletInputStream(byte[] contents) {
      this.buffer = new ByteArrayInputStream(contents);
    }

    @Override
    public int read() {
      return buffer.read();
    }

    @Override
    public boolean isFinished() {
      return buffer.available() == 0;
    }

    @Override
    public boolean isReady() {
      return true;
    }

    @Override
    public void setReadListener(ReadListener listener) {
  throw new RuntimeException("Not implemented");
    }
  }
}

Now the request body can be read more than once by wrapping the original request before passing it through the filter chain:

public class MyFilter implements Filter {
  @Override
  public void doFilter(ServletRequest request, ServletResponse response,
        FilterChain chain) throws IOException, ServletException {

    /* wrap the request in order to read the inputstream multiple times */
    MultiReadHttpServletRequest multiReadRequest = new MultiReadHttpServletRequest((HttpServletRequest) request);

    /* here I read the inputstream and do my thing with it; when I pass the
     * wrapped request through the filter chain, the rest of the filters, and
     * request handlers may read the cached inputstream
     */
    doMyThing(multiReadRequest.getInputStream());
    //OR
    anotherUsage(multiReadRequest.getReader());
    chain.doFilter(multiReadRequest, response);
  }
}

This solution will also allow you to read the request body multiple times via the getParameterXXX methods because the underlying call is getInputStream(), which will of course read the cached request InputStream.

Edit

For newer version of ServletInputStream interface. You need to provide implementation of few more methods like isReady, setReadListener etc. Refer this question as provided in comment below.

Nauseous answered 16/6, 2013 at 0:50 Comment(17)
Is that true? The underlying call is getInputStream() on the original request, of which you'll already have read the bytes. The underlying request has no knowledge of your wrapper, so how would it know to call the wrapper's getInputStream()?Meier
To be precise getInputStream is called on my wrapper since this is the ServletRequest instance that I pass into the filter chain. If you're still in doubt, read the source code for ServletRequestWrapper and the ServletRequest interface.Nauseous
If I could make this +100, I would. I've been trying to get this to work right for 3-4 hrs. Thank-you for your clear example and explanation! I'm glad I found this post!Karelia
Any suggestions how to make this work with Servlet-api 3.0+ ? The ServletInputStream now has abstract isReady(). isFinished() and setReadListener() to deal with non-blocking IO which must be implemented. I'm thinking the ReadListener could be left blank, but not sure what to do about isFinished() and/or isReady().Gwen
Does it works in any environment? In case of Tomcat 7 the getParameter() methods don't call getInputStream(). At least it doesn't happen in my environment...Chin
The code works for servlet-api 2.5, I'm afraid you'll have to test 3.0 and Tomcat 7 (which also uses 3.0).Nauseous
The CachedServletInputStream extending ServletInputStream doesn't seems to implement the other methods as per the latest specificiation. could you post how would those methods should be implemented to make this wrapper workingAdrenal
I believe you should also overwrite getStream since this method is being used by readChunkedPostBody used by parseParameters which populates the parameters attribute of the coyote.Request object. The problem is that getStream doesn't throw an IOException but IOUtils.copy does. So you need to catch it there but there is no sensible thing to do with it.Boccie
I am working on a version for Tomcat 8. You have to overwrite readPostBody and readChunkedPostBody and use getStream so the flags 'usingInputStream' and 'usingReader' aren't set since they will intefere with Request#parseParameter(). You could also overwrite Request#parseParameter() but if you don't need to use the reader it overcomplicates things. I will try to formulate a full answer when my code works.Boccie
The problem with the wrapper solution is that modifying the behavior of getStream, readPostBody or readChunkedPostBody has no effect when these methods are called by parseParameter since parseParameter is executed in the Request and not the WrappedRequest. It's not inheritance... I noticed this when my implementation didn't work and the methods I had modified were never called. I believe this should be dealt with internally in Tomcat. But probably the servlet spec has to change first.Boccie
@EricB. wondering have you got a solution for Servlet-api 3.0+ in the end? I found many examples for the wrapper but they are all under the old api namely not implementing isReady(). isFinished() and setReadListener()Provender
@wuha - I honestly don't remember what I did. And unfortunately, I don't even remember what project this was for so I can't go dig up old source code.Gwen
@EricB. thanks anyway. I later found the solution for the newer api interface, just pasted here in case someone is interested. #29208956Provender
@Boccie - do you have a solution for the tomcat Request parseParameters issue? I've spent several hours on trying to solve this one (see #36522293)Pachisi
Thanks a lot @Nauseous for the solution, worked for me. Can you please also help with MultiReadHttpServletResponse ? I tried similar to MultiReadHttpServletRequest approach, but for some reason it doesn't cache.Fortepiano
Firstly, thank you! Secondly, Is there any DOWNSIDE to this approach? Higher mem usage or something? I ask, as it is baffling to me as to why this is so hard in Spring, like why wouldn't they just make this part of the framework itself? Im using spring to build a simple REST API, so there should be no risk of some binary file being held in memory if thats the concern.Fenland
@Nauseous I see inconsistency. You defined the constructor for CachedServletInputStream as taking one parameter - public CachedServletInputStream(byte[] contents), but when you create a new instance of CachedServletInputStream inside getInputStream() you call the default (i.e. no parameter) constructor, which was never defined.Cherub
P
46

I know I'm late, but this question was still relevant for me and this SO post was one of the top hits in Google. I'm going ahead and post my solution in the hopes that someone else might save couple of hours.

In my case I needed to log all requests and responses with their bodies. Using Spring Framework the answer is actually quite simple, just use ContentCachingRequestWrapper and ContentCachingResponseWrapper.

import org.springframework.web.util.ContentCachingRequestWrapper;
import org.springframework.web.util.ContentCachingResponseWrapper;

import javax.servlet.*;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;

public class LoggingFilter implements Filter {

    @Override
    public void init(FilterConfig filterConfig) throws ServletException {

    }

    @Override
    public void destroy() {

    }

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {

        ContentCachingRequestWrapper requestWrapper = new ContentCachingRequestWrapper((HttpServletRequest) request);
        ContentCachingResponseWrapper responseWrapper = new ContentCachingResponseWrapper((HttpServletResponse) response);

        try {
            chain.doFilter(requestWrapper, responseWrapper);
        } finally {

            String requestBody = new String(requestWrapper.getContentAsByteArray());
            String responseBody = new String(responseWrapper.getContentAsByteArray());
            // Do not forget this line after reading response content or actual response will be empty!
            responseWrapper.copyBodyToResponse();

            // Write request and response body, headers, timestamps etc. to log files

        }

    }

}
Palmetto answered 25/8, 2016 at 20:33 Comment(5)
This didn't work for me. Both requestBody and responseBody were empty stringsUhhuh
My mistake. I was doing a chain.doFilter(request, response); instead of a chain.doFilter(requestWrapper, responseWrapper);Uhhuh
The ContentCaching*Wrapper classes have the expensive price of consuming the input stream so the "caching" is done through the method getContentAsByteArray but this class is not caching the input stream which might be needed by other filters in the filter chain (which is my use case). Imho, this is a not expected behavior of a content caching class, therefore I raised this improvement in spring team jira.spring.io/browse/SPR-16028Greyson
You can use AbstractRequestLoggingFilter from Spring, where most of the work is already done by Spring and you only need to override 1 or 2 simple methods.Bauxite
This doesn't work for me as of spring-web-4.3.12.RELEASE. As I checked the source I found the variable cachedContent is used to store various contents such as request parameters and request inputStream. It's empty if you call getContentAsByteArray() solely. To obtain the request body you have to call getInputStream(). But again, this will make the inputStream unavailable to other filters and the handler.Acklin
P
11

So this is basically Lathy's answer BUT updated for newer requirements for ServletInputStream.

Namely (for ServletInputStream), one has to implement:

public abstract boolean isFinished();

public abstract boolean isReady();

public abstract void setReadListener(ReadListener var1);

This is the edited Lathy's object

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;

public class RequestWrapper extends HttpServletRequestWrapper {

    private String _body;

    public RequestWrapper(HttpServletRequest request) throws IOException {
        super(request);
        _body = "";
        BufferedReader bufferedReader = request.getReader();
        String line;
        while ((line = bufferedReader.readLine()) != null){
            _body += line;
        }
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {

        CustomServletInputStream kid = new CustomServletInputStream(_body.getBytes());
        return kid;
    }

    @Override
    public BufferedReader getReader() throws IOException {
        return new BufferedReader(new InputStreamReader(this.getInputStream()));
    }
}

and somewhere (??) I found this (which is a first-class class that deals with the "extra" methods.

import javax.servlet.ReadListener;
import javax.servlet.ServletInputStream;
import java.io.IOException;
import java.io.UnsupportedEncodingException;

public class CustomServletInputStream extends ServletInputStream {

    private byte[] myBytes;

    private int lastIndexRetrieved = -1;
    private ReadListener readListener = null;

    public CustomServletInputStream(String s) {
        try {
            this.myBytes = s.getBytes("UTF-8");
        } catch (UnsupportedEncodingException ex) {
            throw new IllegalStateException("JVM did not support UTF-8", ex);
        }
    }

    public CustomServletInputStream(byte[] inputBytes) {
        this.myBytes = inputBytes;
    }

    @Override
    public boolean isFinished() {
        return (lastIndexRetrieved == myBytes.length - 1);
    }

    @Override
    public boolean isReady() {
        // This implementation will never block
        // We also never need to call the readListener from this method, as this method will never return false
        return isFinished();
    }

    @Override
    public void setReadListener(ReadListener readListener) {
        this.readListener = readListener;
        if (!isFinished()) {
            try {
                readListener.onDataAvailable();
            } catch (IOException e) {
                readListener.onError(e);
            }
        } else {
            try {
                readListener.onAllDataRead();
            } catch (IOException e) {
                readListener.onError(e);
            }
        }
    }

    @Override
    public int read() throws IOException {
        int i;
        if (!isFinished()) {
            i = myBytes[lastIndexRetrieved + 1];
            lastIndexRetrieved++;
            if (isFinished() && (readListener != null)) {
                try {
                    readListener.onAllDataRead();
                } catch (IOException ex) {
                    readListener.onError(ex);
                    throw ex;
                }
            }
            return i;
        } else {
            return -1;
        }
    }
};

Ultimately, I was just trying to log the requests. And the above frankensteined together pieces helped me create the below.

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.security.Principal;
import java.util.Enumeration;
import java.util.LinkedHashMap;
import java.util.Map;

import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.io.IOUtils;

//one or the other based on spring version
//import org.springframework.boot.autoconfigure.web.ErrorAttributes;
import org.springframework.boot.web.servlet.error.ErrorAttributes;

import org.springframework.core.Ordered;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Component;
import org.springframework.web.context.request.ServletRequestAttributes;
import org.springframework.web.context.request.WebRequest;
import org.springframework.web.filter.OncePerRequestFilter;


/**
 * A filter which logs web requests that lead to an error in the system.
 */
@Component
public class LogRequestFilter extends OncePerRequestFilter implements Ordered {

    // I tried apache.commons and slf4g loggers.  (one or the other in these next 2 lines of declaration */
    //private final static org.apache.commons.logging.Log logger = org.apache.commons.logging.LogFactory.getLog(LogRequestFilter.class);
    private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(LogRequestFilter.class);

    // put filter at the end of all other filters to make sure we are processing after all others
    private int order = Ordered.LOWEST_PRECEDENCE - 8;
    private ErrorAttributes errorAttributes;

    @Override
    public int getOrder() {
        return order;
    }

    @Override
    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
            throws ServletException, IOException {

        String temp = ""; /* for a breakpoint, remove for production/real code */

        /* change to true for easy way to comment out this code, remove this if-check for production/real code */
        if (false) {
            filterChain.doFilter(request, response);
            return;
        }

        /* make a "copy" to avoid issues with body-can-only-read-once issues */
        RequestWrapper reqWrapper = new RequestWrapper(request);

        int status = HttpStatus.INTERNAL_SERVER_ERROR.value();
        // pass through filter chain to do the actual request handling
        filterChain.doFilter(reqWrapper, response);
        status = response.getStatus();

        try {
            Map<String, Object> traceMap = getTrace(reqWrapper, status);
            // body can only be read after the actual request handling was done!
            this.getBodyFromTheRequestCopy(reqWrapper, traceMap);
            
            /* now do something with all the pieces of information gatherered */
            this.logTrace(reqWrapper, traceMap);
        } catch (Exception ex) {
            logger.error("LogRequestFilter FAILED: " + ex.getMessage(), ex);
        }
    }

    private void getBodyFromTheRequestCopy(RequestWrapper rw, Map<String, Object> trace) {
        try {
            if (rw != null) {
                byte[] buf = IOUtils.toByteArray(rw.getInputStream());
                //byte[] buf = rw.getInputStream();
                if (buf.length > 0) {
                    String payloadSlimmed;
                    try {
                        String payload = new String(buf, 0, buf.length, rw.getCharacterEncoding());
                        payloadSlimmed = payload.trim().replaceAll(" +", " ");
                    } catch (UnsupportedEncodingException ex) {
                        payloadSlimmed = "[unknown]";
                    }

                    trace.put("body", payloadSlimmed);
                }
            }
        } catch (IOException ioex) {
            trace.put("body", "EXCEPTION: " + ioex.getMessage());
        }
    }

    private void logTrace(HttpServletRequest request, Map<String, Object> trace) {
        Object method = trace.get("method");
        Object path = trace.get("path");
        Object statusCode = trace.get("statusCode");

        logger.info(String.format("%s %s produced an status code '%s'. Trace: '%s'", method, path, statusCode,
                trace));
    }

    protected Map<String, Object> getTrace(HttpServletRequest request, int status) {
        Throwable exception = (Throwable) request.getAttribute("javax.servlet.error.exception");

        Principal principal = request.getUserPrincipal();

        Map<String, Object> trace = new LinkedHashMap<String, Object>();
        trace.put("method", request.getMethod());
        trace.put("path", request.getRequestURI());
        if (null != principal) {
            trace.put("principal", principal.getName());
        }
        trace.put("query", request.getQueryString());
        trace.put("statusCode", status);

        Enumeration headerNames = request.getHeaderNames();
        while (headerNames.hasMoreElements()) {
            String key = (String) headerNames.nextElement();
            String value = request.getHeader(key);
            trace.put("header:" + key, value);
        }

        if (exception != null && this.errorAttributes != null) {
            trace.put("error", this.errorAttributes
                    .getErrorAttributes((WebRequest) new ServletRequestAttributes(request), true));
        }

        return trace;
    }
}

Please take this code with a grain of salt.

The MOST important "test" is if a POST works with a payload. This is what will expose "double read" issues.

pseudo example code

import org.springframework.web.bind.annotation.*;

@RestController
@RequestMapping("myroute")
public class MyController {
    @RequestMapping(method = RequestMethod.POST, produces = "application/json")
    @ResponseBody
    public String getSomethingExample(@RequestBody MyCustomObject input) {

        String returnValue = "";

        return returnValue;
    }
}

You can replace "MyCustomObject" with plain ole "Object" if you just want to test.

This answer is frankensteined from several different SOF posts and examples..but it took a while to pull it all together so I hope it helps a future reader.

Please upvote Lathy's answer before mine. I could have not gotten this far without it.

Below is one/some of the exceptions I got while working this out.

getReader() has already been called for this request

Looks like some of the places I "borrowed" from are here:

http://slackspace.de/articles/log-request-body-with-spring-boot/

https://github.com/c0nscience/spring-web-logging/blob/master/src/main/java/org/zalando/springframework/web/logging/LoggingFilter.java

https://howtodoinjava.com/servlets/httpservletrequestwrapper-example-read-request-body/

https://www.oodlestechnologies.com/blogs/How-to-create-duplicate-object-of-httpServletRequest-object

https://github.com/c0nscience/spring-web-logging/blob/master/src/main/java/org/zalando/springframework/web/logging/LoggingFilter.java

January 2021 APPEND.

I have learned the hard way that the above code does NOT work for

x-www-form-urlencoded

Consider the example below:

   @CrossOrigin
    @ResponseBody
    @PostMapping(path = "/mypath", consumes = {MediaType.APPLICATION_FORM_URLENCODED_VALUE})
    public ResponseEntity myMethodName(@RequestParam Map<String, String> parameters
    ) {
        /* DO YOU GET ANY PARAMETERS HERE?  Or are they empty because of logging/auditing filter ?*/
        return new ResponseEntity(HttpStatus.OK);

    }

I had to go through several of the other examples here.

I came up with a "wrapper" that works explicitly for APPLICATION_FORM_URLENCODED_VALUE

import org.apache.commons.io.IOUtils;
import org.springframework.http.MediaType;
import org.springframework.web.util.ContentCachingRequestWrapper;

import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;

/**
 * Makes a "copy" of the HttpRequest so the body can be accessed more than 1 time.
 * WORKS WITH APPLICATION_FORM_URLENCODED_VALUE
 * See : https://mcmap.net/q/88787/-why-do-we-wrap-httpservletrequest-the-api-provides-an-httpservletrequestwrapper-but-what-do-we-gain-from-wrapping-the-request/44187955#44187955
 */
public final class AppFormUrlEncodedSpecificContentCachingRequestWrapper extends ContentCachingRequestWrapper {

    public static final String ERROR_MSG_CONTENT_TYPE_NOT_SUPPORTED = "ContentType not supported. (Input ContentType(s)=\"%1$s\", Supported ContentType(s)=\"%2$s\")";

    public static final String ERROR_MSG_PERSISTED_CONTENT_CACHING_REQUEST_WRAPPER_CONSTRUCTOR_FAILED = "AppFormUrlEncodedSpecificContentCachingRequestWrapper constructor failed";

    private static final org.slf4j.Logger LOGGER = org.slf4j.LoggerFactory.getLogger(AppFormUrlEncodedSpecificContentCachingRequestWrapper.class);

    private byte[] body;

    private ServletInputStream inputStream;

    public AppFormUrlEncodedSpecificContentCachingRequestWrapper(HttpServletRequest request) {
        super(request);
        super.getParameterMap(); // init cache in ContentCachingRequestWrapper.  THIS IS THE VITAL CALL so that "@RequestParam Map<String, String> parameters" are populated on the REST Controller.  See https://mcmap.net/q/88067/-http-servlet-request-lose-params-from-post-body-after-read-it-once/64924380#64924380

        String contentType = request.getContentType();
        /* EXPLICTLY check for APPLICATION_FORM_URLENCODED_VALUE and allow nothing else */
        if (null == contentType || !contentType.equalsIgnoreCase(MediaType.APPLICATION_FORM_URLENCODED_VALUE)) {
            IllegalArgumentException ioex = new IllegalArgumentException(String.format(ERROR_MSG_CONTENT_TYPE_NOT_SUPPORTED, contentType, MediaType.APPLICATION_FORM_URLENCODED_VALUE));
            LOGGER.error(ERROR_MSG_PERSISTED_CONTENT_CACHING_REQUEST_WRAPPER_CONSTRUCTOR_FAILED, ioex);
            throw ioex;
        }

        try {
            loadBody(request);
        } catch (IOException ioex) {
            throw new RuntimeException(ioex);
        }
    }

    private void loadBody(HttpServletRequest request) throws IOException {
        body = IOUtils.toByteArray(request.getInputStream());
        inputStream = new CustomServletInputStream(this.getBody());
    }

    private byte[] getBody() {
        return body;
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        if (inputStream != null) {
            return inputStream;
        }
        return super.getInputStream();
    }
}

Note Andrew Sneck's answer on this same page. It is pretty much this : https://programmersought.com/article/23981013626/

I have not had time to harmonize the two above implementations (my two that is).

So I created a Factory to "choose" from the two:

import org.springframework.http.MediaType;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;
import java.io.IOException;

/**
 * Factory to return different concretes of HttpServletRequestWrapper. APPLICATION_FORM_URLENCODED_VALUE needs a different concrete.
 */
public class HttpServletRequestWrapperFactory {

    public static final String ERROR_MSG_HTTP_SERVLET_REQUEST_WRAPPER_FACTORY_CREATE_HTTP_SERVLET_REQUEST_WRAPPER_FAILED = "HttpServletRequestWrapperFactory createHttpServletRequestWrapper FAILED";

    public static HttpServletRequestWrapper createHttpServletRequestWrapper(final HttpServletRequest request) {
        HttpServletRequestWrapper returnItem = null;

        if (null != request) {
            String contentType = request.getContentType();
            if (null != contentType && contentType.equalsIgnoreCase(MediaType.APPLICATION_FORM_URLENCODED_VALUE)) {
                returnItem = new AppFormUrlEncodedSpecificContentCachingRequestWrapper(request);
            } else {
                try {
                    returnItem = new PersistedBodyRequestWrapper(request);
                } catch (IOException ioex) {
                    throw new RuntimeException(ERROR_MSG_HTTP_SERVLET_REQUEST_WRAPPER_FACTORY_CREATE_HTTP_SERVLET_REQUEST_WRAPPER_FAILED, ioex);
                }
            }
        }

        return returnItem;
    }

}

Below is the "other" one that works with JSON, etc. It is the other concrete that the Factory can output. I put it here so that my Jan 2021 APPEND is consistent..I don't know if the code below is perfect consistent with my original answer:

import org.springframework.http.MediaType;

import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.charset.StandardCharsets;
import java.util.Map;

/**
 * Makes a "copy" of the HttpRequest so the body can be accessed more than 1 time.
 * See : https://mcmap.net/q/88787/-why-do-we-wrap-httpservletrequest-the-api-provides-an-httpservletrequestwrapper-but-what-do-we-gain-from-wrapping-the-request/44187955#44187955
 * DOES NOT WORK WITH APPLICATION_FORM_URLENCODED_VALUE
 */
public final class PersistedBodyRequestWrapper extends HttpServletRequestWrapper {

    public static final String ERROR_MSG_CONTENT_TYPE_NOT_SUPPORTED = "ContentType not supported. (ContentType=\"%1$s\")";

    public static final String ERROR_MSG_PERSISTED_BODY_REQUEST_WRAPPER_CONSTRUCTOR_FAILED = "PersistedBodyRequestWrapper constructor FAILED";

    private static final org.slf4j.Logger LOGGER = org.slf4j.LoggerFactory.getLogger(PersistedBodyRequestWrapper.class);

    private String persistedBody;

    private final Map<String, String[]> parameterMap;

    public PersistedBodyRequestWrapper(final HttpServletRequest request) throws IOException {
        super(request);

        String contentType = request.getContentType();
        /* Allow everything EXCEPT APPLICATION_FORM_URLENCODED_VALUE */
        if (null != contentType && contentType.equalsIgnoreCase(MediaType.APPLICATION_FORM_URLENCODED_VALUE)) {
            IllegalArgumentException ioex = new IllegalArgumentException(String.format(ERROR_MSG_CONTENT_TYPE_NOT_SUPPORTED, MediaType.APPLICATION_FORM_URLENCODED_VALUE));
            LOGGER.error(ERROR_MSG_PERSISTED_BODY_REQUEST_WRAPPER_CONSTRUCTOR_FAILED, ioex);
            throw ioex;
        }

        parameterMap = request.getParameterMap();
        this.persistedBody = "";
        BufferedReader bufferedReader = request.getReader();
        String line;
        while ((line = bufferedReader.readLine()) != null) {
            this.persistedBody += line;
        }
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        CustomServletInputStream csis = new CustomServletInputStream(this.persistedBody.getBytes(StandardCharsets.UTF_8));
        return csis;
    }

    @Override
    public BufferedReader getReader() throws IOException {
        return new BufferedReader(new InputStreamReader(this.getInputStream()));
    }

    @Override
    public Map<String, String[]> getParameterMap() {
        return this.parameterMap;
    }
}
Pneuma answered 18/1, 2019 at 17:4 Comment(3)
@KnockingHeads. Way way way too much voodoo. :(Pneuma
Thank you so so much for this, made me understand an important concept. i was just curious to know when we are writing a Request Wrapper do we also need to wrap the response ??Unconscious
I don't "super know" but I strongly sense "No to the RESPONSE". The response seems to be more "complete payload, non-fhir-hose-reader" type of object.Pneuma
L
9

The above answers were very helpful, but still had some problems in my experience. On tomcat 7 servlet 3.0, the getParamter and getParamterValues also had to be overwritten. The solution here includes both get-query parameters and the post-body. It allows for getting raw-string easily.

Like the other solutions it uses Apache commons-io and Googles Guava.

In this solution the getParameter* methods do not throw IOException but they use super.getInputStream() (to get the body) which may throw IOException. I catch it and throw runtimeException. It is not so nice.

import com.google.common.collect.Iterables;
import com.google.common.collect.ObjectArrays;

import org.apache.commons.io.IOUtils;
import org.apache.http.NameValuePair;
import org.apache.http.client.utils.URLEncodedUtils;
import org.apache.http.entity.ContentType;

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.nio.charset.Charset;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;

import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;

/**
 * Purpose of this class is to make getParameter() return post data AND also be able to get entire
 * body-string. In native implementation any of those two works, but not both together.
 */
public class MultiReadHttpServletRequest extends HttpServletRequestWrapper {
    public static final String UTF8 = "UTF-8";
    public static final Charset UTF8_CHARSET = Charset.forName(UTF8);
    private ByteArrayOutputStream cachedBytes;
    private Map<String, String[]> parameterMap;

    public MultiReadHttpServletRequest(HttpServletRequest request) {
        super(request);
    }

    public static void toMap(Iterable<NameValuePair> inputParams, Map<String, String[]> toMap) {
        for (NameValuePair e : inputParams) {
            String key = e.getName();
            String value = e.getValue();
            if (toMap.containsKey(key)) {
                String[] newValue = ObjectArrays.concat(toMap.get(key), value);
                toMap.remove(key);
                toMap.put(key, newValue);
            } else {
                toMap.put(key, new String[]{value});
            }
        }
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        if (cachedBytes == null) cacheInputStream();
        return new CachedServletInputStream();
    }

    @Override
    public BufferedReader getReader() throws IOException {
        return new BufferedReader(new InputStreamReader(getInputStream()));
    }

    private void cacheInputStream() throws IOException {
    /* Cache the inputStream in order to read it multiple times. For
     * convenience, I use apache.commons IOUtils
     */
        cachedBytes = new ByteArrayOutputStream();
        IOUtils.copy(super.getInputStream(), cachedBytes);
    }

    @Override
    public String getParameter(String key) {
        Map<String, String[]> parameterMap = getParameterMap();
        String[] values = parameterMap.get(key);
        return values != null && values.length > 0 ? values[0] : null;
    }

    @Override
    public String[] getParameterValues(String key) {
        Map<String, String[]> parameterMap = getParameterMap();
        return parameterMap.get(key);
    }

    @Override
    public Map<String, String[]> getParameterMap() {
        if (parameterMap == null) {
            Map<String, String[]> result = new LinkedHashMap<String, String[]>();
            decode(getQueryString(), result);
            decode(getPostBodyAsString(), result);
            parameterMap = Collections.unmodifiableMap(result);
        }
        return parameterMap;
    }

    private void decode(String queryString, Map<String, String[]> result) {
        if (queryString != null) toMap(decodeParams(queryString), result);
    }

    private Iterable<NameValuePair> decodeParams(String body) {
        Iterable<NameValuePair> params = URLEncodedUtils.parse(body, UTF8_CHARSET);
        try {
            String cts = getContentType();
            if (cts != null) {
                ContentType ct = ContentType.parse(cts);
                if (ct.getMimeType().equals(ContentType.APPLICATION_FORM_URLENCODED.getMimeType())) {
                    List<NameValuePair> postParams = URLEncodedUtils.parse(IOUtils.toString(getReader()), UTF8_CHARSET);
                    params = Iterables.concat(params, postParams);
                }
            }
        } catch (IOException e) {
            throw new IllegalStateException(e);
        }
        return params;
    }

    public String getPostBodyAsString() {
        try {
            if (cachedBytes == null) cacheInputStream();
            return cachedBytes.toString(UTF8);
        } catch (UnsupportedEncodingException e) {
            throw new RuntimeException(e);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    /* An inputStream which reads the cached request body */
    public class CachedServletInputStream extends ServletInputStream {
        private ByteArrayInputStream input;

        public CachedServletInputStream() {
            /* create a new input stream from the cached request body */
            input = new ByteArrayInputStream(cachedBytes.toByteArray());
        }

        @Override
        public int read() throws IOException {
            return input.read();
        }
    }

    @Override
    public String toString() {
        String query = dk.bnr.util.StringUtil.nullToEmpty(getQueryString());
        StringBuilder sb = new StringBuilder();
        sb.append("URL='").append(getRequestURI()).append(query.isEmpty() ? "" : "?" + query).append("', body='");
        sb.append(getPostBodyAsString());
        sb.append("'");
        return sb.toString();
    }
}
Lightheaded answered 14/4, 2016 at 10:9 Comment(4)
This is great! I've been trying to figure this out for days, and this works with servlet 3.1. One question: why do you do decode(getPostBodyAsString(), result); in getParameterMap()? That creates a parameter with key = request body and value = null, which is pretty odd.Cuspid
Rather than going through all the string parsing, why you don't call super.getParameterMap() in your getParameterMap? Which will give you a map of <String, String[]> anyways.Hoffman
So I hit some issues with paramMap as well. See my answer and the January 2021 "APPEND". there is a magic line there (from yet another answer on this question)..that may be the "trigger" for not having to manually keep the paramertMaps. The magic part of the code super.getParameterMap(); // init cache in ContentCachingRequestWrapper. THIS IS THE VITAL CALL so that "@RequestParam Map<String, String> parameters" are populated on the REST ControllerPneuma
In case of multipart/form-data, its giving org.springframework.web.multipart.support.MissingServletRequestPartException: Required request part 'file' is not present when it do caching by calling getParameterMap. and when i don't do caching then it works fine but gives empty string as request body.Bedbug
C
7

The only way would be for you to consume the entire input stream yourself in the filter, take what you want from it, and then create a new InputStream for the content you read, and put that InputStream in to a ServletRequestWrapper (or HttpServletRequestWrapper).

The downside is you'll have to parse the payload yourself, the standard doesn't make that capability available to you.

Addenda --

As I said, you need to look at HttpServletRequestWrapper.

In a filter, you continue along by calling FilterChain.doFilter(request, response).

For trivial filters, the request and response are the same as the ones passed in to the filter. That doesn't have to be the case. You can replace those with your own requests and/or responses.

HttpServletRequestWrapper is specifically designed to facilitate this. You pass it the original request, and then you can intercept all of the calls. You create your own subclass of this, and replace the getInputStream method with one of your own. You can't change the input stream of the original request, so instead you have this wrapper and return your own input stream.

The simplest case is to consume the original requests input stream in to a byte buffer, do whatever magic you want on it, then create a new ByteArrayInputStream from that buffer. This is what is returned in your wrapper, which is passed to the FilterChain.doFilter method.

You'll need to subclass ServletInputStream and make another wrapper for your ByteArrayInputStream, but that's not a big deal either.

Colenecoleopteran answered 18/4, 2012 at 14:16 Comment(1)
I can't manage to read the InputStream and restore it after, there are no get/set methods to direct access to the stream. Your proposal seems good, but I don't see how to implement it.Usual
T
7

I found good solution for any format of request body. I tested for application/x-www-form-urlencoded and application/json both worked very well. Problem of ContentCachingRequestWrapper that is designed only for x-www-form-urlencoded request body, but not work with e.g. json. I found solution for json link. It had trouble that it didn't support x-www-form-urlencoded. I joined both in my code:

import org.apache.commons.io.IOUtils;
import org.springframework.web.util.ContentCachingRequestWrapper;

import javax.servlet.ReadListener;
import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStreamReader;

public class MyContentCachingRequestWrapper extends ContentCachingRequestWrapper {

    private byte[] body;

    public MyContentCachingRequestWrapper(HttpServletRequest request) throws IOException {
        super(request);
        super.getParameterMap(); // init cache in ContentCachingRequestWrapper
        body = super.getContentAsByteArray(); // first option for application/x-www-form-urlencoded
        if (body.length == 0) {
          try {
            body = IOUtils.toByteArray(super.getInputStream()); // second option for other body formats
          } catch (IOException ex) {
            body = new byte[0];
          }
        }
    }

    public byte[] getBody() {
        return body;
    }

    @Override
    public ServletInputStream getInputStream() {
        return new RequestCachingInputStream(body);
    }

    @Override
    public BufferedReader getReader() throws IOException {
        return new BufferedReader(new InputStreamReader(getInputStream(), getCharacterEncoding()));
    }

    private static class RequestCachingInputStream extends ServletInputStream {

        private final ByteArrayInputStream inputStream;

        public RequestCachingInputStream(byte[] bytes) {
            inputStream = new ByteArrayInputStream(bytes);
        }

        @Override
        public int read() throws IOException {
            return inputStream.read();
        }

        @Override
        public boolean isFinished() {
            return inputStream.available() == 0;
        }

        @Override
        public boolean isReady() {
            return true;
        }

        @Override
        public void setReadListener(ReadListener readlistener) {
        }

    }

}
Tresatrescha answered 20/11, 2020 at 5:39 Comment(5)
Your call to "super.getParameterMap();" was the magic for me.Pneuma
I updated my answer...but your answer (which I upvoted) had the magic line .. that assisted my code. and a well placed comment next to your code. thanks. super.getParameterMap(); // init cache in ContentCachingRequestWrapperPneuma
@Pneuma you should click trought to super method and you see what is doingTresatrescha
@AndrewSneck Thank you very much! It works fine for both application/x-www-form-urlencoded and application/jsonPeltier
It is not working for multipart/form-data.Bedbug
K
6

I too had the same issue and I believe the code below is more simple and it is working for me,

public class MultiReadHttpServletRequest extends  HttpServletRequestWrapper {

 private String _body;

public MultiReadHttpServletRequest(HttpServletRequest request) throws IOException {
   super(request);
   _body = "";
   BufferedReader bufferedReader = request.getReader();           
   String line;
   while ((line = bufferedReader.readLine()) != null){
       _body += line;
   }
}

@Override
public ServletInputStream getInputStream() throws IOException {
   final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(_body.getBytes());
   return new ServletInputStream() {
       public int read() throws IOException {
           return byteArrayInputStream.read();
       }
   };
}

@Override
public BufferedReader getReader() throws IOException {
   return new BufferedReader(new InputStreamReader(this.getInputStream()));
}
}

in the filter java class,

HttpServletRequest properRequest = ((HttpServletRequest) req);
MultiReadHttpServletRequest wrappedRequest = new MultiReadHttpServletRequest(properRequest);
req = wrappedRequest;
inputJson = IOUtils.toString(req.getReader());
System.out.println("body"+inputJson);

Please let me know if you have any queries

Kana answered 5/8, 2015 at 11:36 Comment(1)
Firstly, thank you! Secondly, Is there any DOWNSIDE to this approach? Higher mem usage or something? I ask, as it is baffling to me as to why this is so hard in Spring, like why wouldn't they just make this part of the framework itself? Im using spring to build a simple REST API, so there should be no risk of some binary file being held in memory if thats the concern.Fenland
C
3

Spring has built-in support for this with an AbstractRequestLoggingFilter:

@Bean
public Filter loggingFilter(){
    final AbstractRequestLoggingFilter filter = new AbstractRequestLoggingFilter() {
        @Override
        protected void beforeRequest(final HttpServletRequest request, final String message) {

        }

        @Override
        protected void afterRequest(final HttpServletRequest request, final String message) {

        }
    };

    filter.setIncludePayload(true);
    filter.setIncludeQueryString(false);
    filter.setMaxPayloadLength(1000000);

    return filter;
}

Unfortunately you still won't be able to read the payload directly off the request, but the String message parameter will include the payload so you can grab it from there as follows:

String body = message.substring(message.indexOf("{"), message.lastIndexOf("]"));

Conifer answered 26/2, 2019 at 12:33 Comment(1)
I was hoping to use your solution to generate an audit log but I need to record whether the request was successful, can I hook into the http response and get the code within this class.Quaternion
C
2

Just overwriting of getInputStream() did not work in my case. My server implementation seems to parse parameters without calling this method. I did not find any other way, but re-implement the all four getParameter* methods as well. Here is the code of getParameterMap (Apache Http Client and Google Guava library used):

@Override
public Map<String, String[]> getParameterMap() {
    Iterable<NameValuePair> params = URLEncodedUtils.parse(getQueryString(), NullUtils.UTF8);

    try {
        String cts = getContentType();
        if (cts != null) {
            ContentType ct = ContentType.parse(cts);
            if (ct.getMimeType().equals(ContentType.APPLICATION_FORM_URLENCODED.getMimeType())) {
                List<NameValuePair> postParams = URLEncodedUtils.parse(IOUtils.toString(getReader()), NullUtils.UTF8);
                params = Iterables.concat(params, postParams);
            }
        }
    } catch (IOException e) {
        throw new IllegalStateException(e);
    }
    Map<String, String[]> result = toMap(params);
    return result;
}

public static Map<String, String[]> toMap(Iterable<NameValuePair> body) {
    Map<String, String[]> result = new LinkedHashMap<>();
    for (NameValuePair e : body) {
        String key = e.getName();
        String value = e.getValue();
        if (result.containsKey(key)) {
            String[] newValue = ObjectArrays.concat(result.get(key), value);
            result.remove(key);
            result.put(key, newValue);
        } else {
            result.put(key, new String[] {value});
        }
    }
    return result;
}
Chin answered 25/9, 2014 at 8:54 Comment(4)
Jetty has this issue unfortunately, grepcode.com/file/repo1.maven.org/maven2/org.eclipse.jetty/…Expressionism
You are probably using Tomcat 7 or up with Servlet 3.0? Do you have the code for the other 3 methods as well?Boccie
Other 3 methods just call getParameterMap() and fetch the value needed.Chin
So I hit some issues with paramMap as well. See my answer and the January 2021 "APPEND". there is a magic line there (that came from yet another answer on this question)..that may be the "trigger" for not having to manually keep the paramertMaps. The magic part of the code super.getParameterMap(); // init cache in ContentCachingRequestWrapper. THIS IS THE VITAL CALL so that "@RequestParam Map<String, String> parameters" are populated on the REST ControllerPneuma
T
1

The method getContentAsByteArray() of the Spring class ContentCachingRequestWrapper reads the body multiple times, but the methods getInputStream() and getReader() of the same class do not read the body multiple times:

"This class caches the request body by consuming the InputStream. If we read the InputStream in one of the filters, then other subsequent filters in the filter chain can't read it anymore. Because of this limitation, this class is not suitable in all situations."

In my case more general solution that solved this problem was to add following three classes to my Spring boot project (and the required dependencies to the pom file):

CachedBodyHttpServletRequest.java:

public class CachedBodyHttpServletRequest extends HttpServletRequestWrapper {

    private byte[] cachedBody;

    public CachedBodyHttpServletRequest(HttpServletRequest request) throws IOException {
        super(request);
        InputStream requestInputStream = request.getInputStream();
        this.cachedBody = StreamUtils.copyToByteArray(requestInputStream);
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        return new CachedBodyServletInputStream(this.cachedBody);
    }

    @Override
    public BufferedReader getReader() throws IOException {
        // Create a reader from cachedContent
        // and return it
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(this.cachedBody);
        return new BufferedReader(new InputStreamReader(byteArrayInputStream));
    }
}

CachedBodyServletInputStream.java:

public class CachedBodyServletInputStream extends ServletInputStream {

    private InputStream cachedBodyInputStream;

    public CachedBodyServletInputStream(byte[] cachedBody) {
        this.cachedBodyInputStream = new ByteArrayInputStream(cachedBody);
    }

    @Override
    public boolean isFinished() {
        try {
            return cachedBodyInputStream.available() == 0;
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        return false;
    }

    @Override
    public boolean isReady() {
        return true;
    }

    @Override
    public void setReadListener(ReadListener readListener) {
        throw new UnsupportedOperationException();
    }

    @Override
    public int read() throws IOException {
        return cachedBodyInputStream.read();
    }
}

ContentCachingFilter.java:

@Order(value = Ordered.HIGHEST_PRECEDENCE)
@Component
@WebFilter(filterName = "ContentCachingFilter", urlPatterns = "/*")
public class ContentCachingFilter extends OncePerRequestFilter {

    @Override
    protected void doFilterInternal(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, FilterChain filterChain) throws ServletException, IOException {
        System.out.println("IN  ContentCachingFilter ");
        CachedBodyHttpServletRequest cachedBodyHttpServletRequest = new CachedBodyHttpServletRequest(httpServletRequest);
        filterChain.doFilter(cachedBodyHttpServletRequest, httpServletResponse);
    }
}

I also added the following dependencies to pom:

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>5.2.0.RELEASE</version>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>javax.servlet-api</artifactId>
    <version>4.0.1</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.10.0</version>
</dependency>

A tuturial and full source code is located here: https://www.baeldung.com/spring-reading-httpservletrequest-multiple-times

Tiga answered 20/12, 2019 at 22:33 Comment(0)
P
0

If you have control over the request, you could set the content type to binary/octet-stream. This allows to query for parameters without consuming the input stream.

However, this might be specific to some application servers. I only tested tomcat, jetty seems to behave the same way according to https://mcmap.net/q/88788/-why-is-httpservletrequest-inputstream-empty.

Pullulate answered 27/8, 2014 at 14:3 Comment(0)
A
-1

you can use servlet filter chain, but instead use the original one, you can create your own request yourownrequests extends HttpServletRequestWrapper.

Ada answered 30/6, 2013 at 1:48 Comment(1)
It seems link to tutorial contains a virus now.Ethelind
I
-3

First of all we should not read parameters within the filter. Usually the headers are read in the filter to do few authentication tasks. Having said that one can read the HttpRequest body completely in the Filter or Interceptor by using the CharStreams:

String body = com.google.common.io.CharStreams.toString(request.getReader());

This does not affect the subsequent reads at all.

Instead answered 23/12, 2016 at 16:46 Comment(2)
yes it does. If you do this once, request.getReader() will return a reader that only contains an empty string on the subsequent reads.Stockbroker
I would work in case of overwrite getReader() and getInputStream() methods to use this new body as source.Harpole

© 2022 - 2024 — McMap. All rights reserved.