How to decode Gzip compressed request body in Spring MVC
Asked Answered
S

3

19

I have a client that sends data with

CONTENT-ENCODING deflate

I have code like this

@RequestMapping(value = "/connect", method = RequestMethod.POST)
@ResponseBody
public Map onConnect(@RequestBody String body){}

Currently 'body' prints out the garbled, compressed data. Is there any way to make Spring MVC automatically uncompress it?

Skylar answered 19/5, 2013 at 19:0 Comment(2)
Just a guess, but doesn't Spring support HTTP filters?Enlarger
not sure why there are 5 questions asking the same thing on SO, but heres one solution: https://mcmap.net/q/507563/-http-request-compressionGarceau
J
2

You don't handle it in Spring. Instead you use a filter so that the data arrives in Spring already deflated.

Hopefully these two links can get you started.

Jassy answered 19/5, 2013 at 19:13 Comment(2)
The question is asked about the gzip request instead response.Mersey
@Erik Nedwidek How do we add this Gzip filter in Spring Web Flux (with Netty) as I don't have ServletRequest in there?Hemocyte
C
34

You'll need to write you own filter to unzip body of gzipped requests. Since you will be reading the whole input stream from request you need to override parameters parsing method too. This is the filter I'm using in my code. Supports only gzipped POST requests, but you can update it to use other types of requests if needed. Also beware to parse parameters I'm using guava library, you can grab yours from here: http://central.maven.org/maven2/com/google/guava/guava/

public class GzipBodyDecompressFilter extends Filter {
    
    @Override
    public void init(FilterConfig filterConfig) throws ServletException {
        
    }
    /**
     * Analyzes servlet request for possible gzipped body.
     * When Content-Encoding header has "gzip" value and request method is POST we read all the
     * gzipped stream and is it haz any data unzip it. In case when gzip Content-Encoding header
     * specified but body is not actually in gzip format we will throw ZipException.
     *
     * @param servletRequest  servlet request
     * @param servletResponse servlet response
     * @param chain           filter chain
     * @throws IOException      throws when fails
     * @throws ServletException thrown when fails
     */
    @Override
    public final void doFilter(final ServletRequest servletRequest,
                               final ServletResponse servletResponse,
                               final FilterChain chain) throws IOException, ServletException {
        HttpServletRequest request = (HttpServletRequest) servletRequest;
        HttpServletResponse response = (HttpServletResponse) servletResponse;
        boolean isGzipped = request.getHeader(HttpHeaders.CONTENT_ENCODING) != null
                && request.getHeader(HttpHeaders.CONTENT_ENCODING).contains("gzip");
        boolean requestTypeSupported = HttpMethods.POST.equals(request.getMethod());
        if (isGzipped && !requestTypeSupported) {
            throw new IllegalStateException(request.getMethod()
                    + " is not supports gzipped body of parameters."
                    + " Only POST requests are currently supported.");
        }
        if (isGzipped && requestTypeSupported) {
            request = new GzippedInputStreamWrapper((HttpServletRequest) servletRequest);
        }
        chain.doFilter(request, response);

    }

    /**
     * @inheritDoc
     */
    @Override
    public final void destroy() {
    }

    /**
     * Wrapper class that detects if the request is gzipped and ungzipps it.
     */
    final class GzippedInputStreamWrapper extends HttpServletRequestWrapper {
        /**
         * Default encoding that is used when post parameters are parsed.
         */
        public static final String DEFAULT_ENCODING = "ISO-8859-1";

        /**
         * Serialized bytes array that is a result of unzipping gzipped body.
         */
        private byte[] bytes;

        /**
         * Constructs a request object wrapping the given request.
         * In case if Content-Encoding contains "gzip" we wrap the input stream into byte array
         * to original input stream has nothing in it but hew wrapped input stream always returns
         * reproducible ungzipped input stream.
         *
         * @param request request which input stream will be wrapped.
         * @throws java.io.IOException when input stream reqtieval failed.
         */
        public GzippedInputStreamWrapper(final HttpServletRequest request) throws IOException {
            super(request);
            try {
                final InputStream in = new GZIPInputStream(request.getInputStream());
                bytes = ByteStreams.toByteArray(in);
            } catch (EOFException e) {
                bytes = new byte[0];
            }
        }


        /**
         * @return reproduceable input stream that is either equal to initial servlet input
         * stream(if it was not zipped) or returns unzipped input stream.
         * @throws IOException if fails.
         */
        @Override
        public ServletInputStream getInputStream() throws IOException {
            final ByteArrayInputStream sourceStream = new ByteArrayInputStream(bytes);
            return new ServletInputStream() {
                public int read() throws IOException {
                    return sourceStream.read();
                }

                public void close() throws IOException {
                    super.close();
                    sourceStream.close();
                }
            };
        }

        /**
         * Need to override getParametersMap because we initially read the whole input stream and
         * servlet container won't have access to the input stream data.
         *
         * @return parsed parameters list. Parameters get parsed only when Content-Type
         * "application/x-www-form-urlencoded" is set.
         */
        @Override
        public Map getParameterMap() {
            String contentEncodingHeader = getHeader(HttpHeaders.CONTENT_TYPE);
            if (!Strings.isNullOrEmpty(contentEncodingHeader)
                    && contentEncodingHeader.contains("application/x-www-form-urlencoded")) {
                Map params = new HashMap(super.getParameterMap());
                try {
                    params.putAll(parseParams(new String(bytes)));
                } catch (UnsupportedEncodingException e) {
                    e.printStackTrace();
                }
                return params;
            } else {
                return super.getParameterMap();
            }
        }

        /**
         * parses params from the byte input stream.
         *
         * @param body request body serialized to string.
         * @return parsed parameters map.
         * @throws UnsupportedEncodingException if encoding provided is not supported.
         */
        private Map<String, String[]> parseParams(final String body)
                throws UnsupportedEncodingException {
            String characterEncoding = getCharacterEncoding();
            if (null == characterEncoding) {
                characterEncoding = DEFAULT_ENCODING;
            }
            final Multimap<String, String> parameters = ArrayListMultimap.create();
            for (String pair : body.split("&")) {
                if (Strings.isNullOrEmpty(pair)) {
                    continue;
                }
                int idx = pair.indexOf("=");

                String key = null;
                if (idx > 0) {
                    key = URLDecoder.decode(pair.substring(0, idx), characterEncoding);
                } else {
                    key = pair;
                }
                String value = null;
                if (idx > 0 && pair.length() > idx + 1) {
                    value = URLDecoder.decode(pair.substring(idx + 1), characterEncoding);
                } else {
                    value = null;
                }
                parameters.put(key, value);
            }
            return Maps.transformValues(parameters.asMap(),
                    new Function<Collection<String>, String[]>() {
                        @Nullable
                        @Override
                        public String[] apply(final Collection<String> input) {
                            return Iterables.toArray(input, String.class);
                        }
                    });
        }
    }
}
Compositor answered 6/10, 2014 at 23:9 Comment(3)
I had to add charset=utf-8 to my content-type header on the request and then this worked beautifully! 'Content-Type' : 'application/json; charset=utf-8'Elwood
getParameter map shouldn't be needed as it's not called as part of the filter. Also requestTypeSupported is redundant as spring will return error 415 unsupported request if other mapping isn't implemented.Wimble
Nice filter :) Here's an updated version for 2020 for Java 11 & Guava 29: gist.github.com/tadhgpearson/0a778e1340c8280fc396d06c9b419163 It's worth mentioning that performance-wise, Undertow has native support to decompress the POST body and is about an order of magnitude faster at doing it. If it's an option, I encourage you to use spring-boot-starter-undertow instead.Hakon
H
8

This should be handled by the server, not the application.

As far as I know, Tomcat doesn't support it, though you could probably write a filter.

A common way to handle this is to put Tomcat ( or whatever Java container you're using) behind an Apache server that is configured to handle compressed request bodies.

Hebbe answered 19/5, 2013 at 19:19 Comment(2)
like this: serverfault.com/questions/56700/…Bazan
What if for example you want to decompress in order to log the message? That case should be handled by the application fiigure...Interference
J
2

You don't handle it in Spring. Instead you use a filter so that the data arrives in Spring already deflated.

Hopefully these two links can get you started.

Jassy answered 19/5, 2013 at 19:13 Comment(2)
The question is asked about the gzip request instead response.Mersey
@Erik Nedwidek How do we add this Gzip filter in Spring Web Flux (with Netty) as I don't have ServletRequest in there?Hemocyte

© 2022 - 2024 — McMap. All rights reserved.