How to read request.getInputStream() multiple times

Question

I have this code:

@Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
        throws IOException, ServletException {
    logger.info("Filter start...");

    HttpServletRequest httpRequest = (HttpServletRequest) request;
    HttpServletResponse httpResponse = (HttpServletResponse) response;

    String ba = getBaId(getBody(httpRequest));

    if (ba == null) {
        logger.error("Wrong XML");
        httpResponse.setStatus(HttpServletResponse.SC_BAD_REQUEST);
    } else {      

        if (!clients.containsKey(ba)) {
            clients.put(ba, 1);
            logger.info("Client map : init...");
        } else {
            clients.put(ba, clients.get(ba).intValue() + 1);
            logger.info("Threads for " + ba + " = " + clients.get(ba).toString());
        }

        chain.doFilter(request, response);
    }
}

and this web.xml (packages are shortened and names changed, but it looks the same)

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app>
  <filter>
    <filter-name>TestFilter</filter-name>
    <filter-class>pkg.TestFilter</filter-class>
  </filter>
  <filter-mapping>
    <filter-name>TestFilter</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>

  <context-param>
    <param-name>contextConfigLocation</param-name>
    <param-value>WEB-INF/applicationContext.xml</param-value>
  </context-param>

  <listener>
    <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
  </listener>

  <servlet>
    <servlet-name>Name</servlet-name>
    <display-name>Name</display-name>
    <servlet-class>pkg.Name</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>Name</servlet-name>
    <url-pattern>/services/*</url-pattern>
  </servlet-mapping>
</web-app>

I want to invoke the Servlet after the Filter. I was hoping chain.doFilter(...) could do the trick, but i always get this error on the line with chain.doFilter(...):

java.lang.IllegalStateException: getInputStream() can't be called after getReader()
at com.caucho.server.connection.AbstractHttpRequest.getInputStream(AbstractHttpRequest.java:1933)
at org.apache.cxf.transport.http.AbstractHTTPDestination.setupMessage(AbstractHTTPDestination.java:249)
at org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:82)
at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:283)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:166)
at org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:174)
at org.apache.cxf.transport.servlet.AbstractCXFServlet.doPost(AbstractCXFServlet.java:152)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:153)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
at pkg.TestFilter.doFilter(TestFilter.java:102)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:187)
at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:265)
at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:273)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:682)
at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:743)
at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:662)
at java.lang.Thread.run(Thread.java:619)

Godfrey

Yes, it should work. Does the servlet work without the filter? – Unlatch

The servlet works without the filter and the filter without chain.doFilter() works too – Godfrey

putting it outside if..else didn't help – Godfrey

Also, i notice you're trying to read XML using a Reader, which in my experience is usually wrong. You should be using an XML parser on an InputStream. I sure hope you're not pulling values out of XML using regular expressions or anything like that? Can we see your getBaId() and getBody() methods? – Biographer

I need only one parameter and I think that regular expressions may be faster and more efficient than parsing the whole XML. – Godfrey

Answer 1

You probably start consuming the HttpServletRequest using getReader() in :

String ba = getBaId(getBody(httpRequest)); 

Your servlet tries to call getInputStream() on the same request, which is not allowed. What you need to do is use a ServletRequestWrapper to make a copy of the body of the request, so you can read it with multiple methods. I dont have the time to find a complete example right know ... sorry ...

Luisluisa

Having a copy of the request and using getReader() doesn't help. The exception will be getReader() has already been called for this request – Ticonderoga

Answer 2

Working code based on the accepted answer.

public class CustomHttpServletRequestWrapper extends HttpServletRequestWrapper {

private static final Logger logger = Logger.getLogger(CustomHttpServletRequestWrapper.class);
private final String body;

public CustomHttpServletRequestWrapper(HttpServletRequest request) {
    super(request);

    StringBuilder stringBuilder = new StringBuilder();  
    BufferedReader bufferedReader = null;  

    try {  
        InputStream inputStream = request.getInputStream(); 

        if (inputStream != null) {  
            bufferedReader = new BufferedReader(new InputStreamReader(inputStream));  

            char[] charBuffer = new char[128];  
            int bytesRead = -1;  

            while ((bytesRead = bufferedReader.read(charBuffer)) > 0) {  
                stringBuilder.append(charBuffer, 0, bytesRead);  
            }  
        } else {  
            stringBuilder.append("");  
        }  
    } catch (IOException ex) {  
        logger.error("Error reading the request body...");  
    } finally {  
        if (bufferedReader != null) {  
            try {  
                bufferedReader.close();  
            } catch (IOException ex) {  
                logger.error("Error closing bufferedReader...");  
            }  
        }  
    }  

    body = stringBuilder.toString();  
}

@Override  
public ServletInputStream getInputStream () throws IOException {          
    final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(body.getBytes());

    ServletInputStream inputStream = new ServletInputStream() {  
        public int read () throws IOException {  
            return byteArrayInputStream.read();  
        }  
    };

    return inputStream;  
} 
}

Godfrey

Thank you. Working with Spring, I needed to override getReader() too: public BufferedReader getReader() throws IOException { return new BufferedReader(new StringReader(body)); } – Selfabasement

Does this need the filter as well, or can I directly use this class and create a new object and then convert to string? PS: i am very new to java and need to do this to print my JSON from request. – Plast

Answer 3

This worked for me. It implements getInputStream.

private class MyHttpServletRequestWrapper extends HttpServletRequestWrapper {

    private byte[] body;

    public MyHttpServletRequestWrapper(HttpServletRequest request) {
        super(request);
        try {
            body = IOUtils.toByteArray(request.getInputStream());
        } catch (IOException ex) {
            body = new byte[0];
        }
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        return new ServletInputStream() {
            ByteArrayInputStream bais = new ByteArrayInputStream(body);

            @Override
            public int read() throws IOException {
                return bais.read();
            }
        };
    }

}

Then you use in your method:

//copy body
servletRequest = new MyHttpServletRequestWrapper(servletRequest);

Enneahedron

How do we handle if Request has Multipart file upload request.? – Dele

this will work with any content. The disadvantage is: all content ist kept in the memory and thus the memory limit may be a problem. So only create the wrapper where needed. e.g. pass the original request if multipart and do not ready anything. – Gunter

Answer 4

For Servlet 3.1

class MyHttpServletRequestWrapper extends HttpServletRequestWrapper {

    private byte[] body;

    public MyHttpServletRequestWrapper(HttpServletRequest request) {
        super(request);
        try {
            body = IOUtils.toByteArray(request.getInputStream());
        } catch (IOException ex) {
            body = new byte[0];
        }
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {

        return new DelegatingServletInputStream(new ByteArrayInputStream(body));


    }

}


public class DelegatingServletInputStream extends ServletInputStream {

    private final InputStream sourceStream;

    private boolean finished = false;


    /**
     * Create a DelegatingServletInputStream for the given source stream.
     *
     * @param sourceStream the source stream (never {@code null})
     */
    public DelegatingServletInputStream(InputStream sourceStream) {
        this.sourceStream = sourceStream;
    }

    /**
     * Return the underlying source stream (never {@code null}).
     */
    public final InputStream getSourceStream() {
        return this.sourceStream;
    }


    @Override
    public int read() throws IOException {
        int data = this.sourceStream.read();
        if (data == -1) {
            this.finished = true;
        }
        return data;
    }

    @Override
    public int available() throws IOException {
        return this.sourceStream.available();
    }

    @Override
    public void close() throws IOException {
        super.close();
        this.sourceStream.close();
    }

    @Override
    public boolean isFinished() {
        return this.finished;
    }

    @Override
    public boolean isReady() {
        return true;
    }

    @Override
    public void setReadListener(ReadListener readListener) {
        throw new UnsupportedOperationException();
    }

}

Crockery

What will happen in case of async request. Because you are throwing UnsupportedOperationException from setReadListener. – Unblock

@MaheshBhuva you can implement setReadListener yourself. – Toddy

Can you please provide reference for custom implementation of setReadListener? And If I leave setReadListener implementation blank, then servlet async feature won't work. Am I right? – Unblock

I have also one question. If we modify inputstream by adding some fields (in case of json body), will it cause any issue in case of multipart request? – Unblock

@MaheshBhuva you show modify the json body only if the request is json request, you can use content-type header to verify if this request is json request – Toddy

Answer 5

inputStream in servlet request can only be used once because of it is stream,you can store it and then get it from a byte array,this can resolve.

public class HttpServletRequestWrapper extends javax.servlet.http.HttpServletRequestWrapper {

private final byte[] body;

public HttpServletRequestWrapper(HttpServletRequest request)
        throws IOException {
    super(request);
    body = StreamUtil.readBytes(request.getReader(), "UTF-8");
}

@Override
public BufferedReader getReader() throws IOException {
    return new BufferedReader(new InputStreamReader(getInputStream()));
}

@Override
public ServletInputStream getInputStream() throws IOException {
    final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(body);
    return new ServletInputStream() {

        @Override
        public int read() throws IOException {
            return byteArrayInputStream.read();
        }

        @Override
        public boolean isFinished() {
            return false;
        }

        @Override
        public boolean isReady() {
            return false;
        }

        @Override
        public void setReadListener(ReadListener arg0) {
        }
    };
}
}

in filter:

ServletRequest requestWrapper = new HttpServletRequestWrapper(request);

Hakodate

But then will it work for async servlet as we are leaving setReadListener method empty?. – Unblock

can you blindly always return false? Wouldnt it have side effects? – Atomizer

Answer 6

request.getInputStream() is allowed to read only one time. In order to use this method many times, we need to do extra the custom task to HttpServletReqeustWrapper class. see my sample wrapper class below.

public class MultiReadHttpServletRequest extends HttpServletRequestWrapper {
    private ByteArrayOutputStream cachedBytes;

    public MultiReadHttpServletRequest(HttpServletRequest request) {
        super(request);
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        if (cachedBytes == null)
            cacheInputStream();

        return new CachedServletInputStream();
    }

    @Override
    public BufferedReader getReader() throws IOException {
        return new BufferedReader(new InputStreamReader(getInputStream()));
    }

    private void cacheInputStream() throws IOException {
        /*
         * Cache the inputstream in order to read it multiple times. For convenience, I use apache.commons IOUtils
         */
        cachedBytes = new ByteArrayOutputStream();
        IOUtils.copy(super.getInputStream(), cachedBytes);
    }

    /* An inputstream which reads the cached request body */
    public class CachedServletInputStream extends ServletInputStream {
        private ByteArrayInputStream input;

        public CachedServletInputStream() {
            /* create a new input stream from the cached request body */
            input = new ByteArrayInputStream(cachedBytes.toByteArray());
        }

        @Override
        public int read() throws IOException {
            return input.read();
        }
    }
}

In my case, I trace all incoming requests into the log. I created a Filter

public class TracerRequestFilter implements Filter {
    private static final Logger LOG = LoggerFactory.getLogger(TracerRequestFilter.class);

    @Override
    public void destroy() {

    }

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException,
        ServletException {
        final HttpServletRequest req = (HttpServletRequest) request;

        try {
            if (LOG.isDebugEnabled()) {
                final MultiReadHttpServletRequest wrappedRequest = new MultiReadHttpServletRequest(req);
                // debug payload info
                logPayLoad(wrappedRequest);
                chain.doFilter(wrappedRequest, response);
            } else {
                chain.doFilter(request, response);
            }
        } finally {
            LOG.info("end-of-process");
        }
    }

    private String getRemoteAddress(HttpServletRequest req) {
        String ipAddress = req.getHeader("X-FORWARDED-FOR");
        if (ipAddress == null) {
            ipAddress = req.getRemoteAddr();
        }
        return ipAddress;
    }

    private void logPayLoad(HttpServletRequest request) {
        final StringBuilder params = new StringBuilder();
        final String method = request.getMethod().toUpperCase();
        final String ipAddress = getRemoteAddress(request);
        final String userAgent = request.getHeader("User-Agent");
        LOG.debug(String.format("============debug request=========="));
        LOG.debug(String.format("Access from ip:%s;ua:%s", ipAddress, userAgent));
        LOG.debug(String.format("Method : %s requestUri %s", method, request.getRequestURI()));
        params.append("Query Params:").append(System.lineSeparator());
        Enumeration<String> parameterNames = request.getParameterNames();

        for (; parameterNames.hasMoreElements();) {
            String paramName = parameterNames.nextElement();
            String paramValue = request.getParameter(paramName);
            if ("password".equalsIgnoreCase(paramName) || "pwd".equalsIgnoreCase(paramName)) {
                paramValue = "*****";
            }
            params.append("---->").append(paramName).append(": ").append(paramValue).append(System.lineSeparator());
        }
        LOG.debug(params.toString());
        /** request body */

        if ("POST".equals(method) || "PUT".equals(method)) {
            try {
                LOG.debug(IOUtils.toString(request.getInputStream()));
            } catch (IOException e) {
                LOG.error(e.getMessage(), e);
            }
        }
        LOG.debug(String.format("============End-debug-request=========="));
    }

    @Override
    public void init(FilterConfig arg0) throws ServletException {

    }
}

It works for me both Servlet 2.5 and 3.0. I see all request params both form-encoded and request json body.

Mesopause