Spring boot response compression not working
Asked Answered
P

8

28

I have some javascript bundled file that is pretty big, ~1MB. I'm trying to turn on response compression with the following application properties in my yml file:

server.compression.enabled: true
server.compression.mime-types: application/json,application/xml,text/html,text/xml,text/plain,application/javascript,text/css

But it doesn't work. No compression is happening.

Request headers:

Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
Accept: */*
Accept-Encoding: gzip, deflate, sdch, br

Response headers

Cache-Control:no-cache, no-store, max-age=0, must-revalidate
Connection:keep-alive
Content-Length:842821
Content-Type:application/javascript;charset=UTF-8

There's no content encoding header in the response.

I'm using spring boot version 1.3.5.RELEASE

What am I missing?

=== EDIT 4 === I was planning to create a stand alone app to investigate further why content compression properties weren't working. But all of sudden it started working and I haven't changed any thing configuration-wise, not POM file change, not application.yml file change. So I don't know what has changed that made it working...

===EDIT 3=== follow @chimmi's suggestions further. I've put break points in the suggested places. It looks like requests to static resources (js files) never stopped at those break points. Only rest API requests do. And for those request, the content-length was zero for some reason which causes the content compression to be skipped.

enter image description here

===EDIT 2=== I've put a break point at line 180 of o.s.b.a.w.ServerProperties thanks to @chimmi's suggestion and it shows that all the properties are set but somehow the server doesn't honor the setting... :(

Printing the Compression object at o.s.b.a.w.ServerProperties line 180

===EDIT 1===

not sure if it matters, but I'm pasting my application main and configuration code here:

Application.java:

@SpringBootApplication
public class TuangouApplication extends SpringBootServletInitializer {

    public static void main(String[] args) throws Exception {
        SpringApplication.run(TuangouApplication.class, args);
    }

    // this is for WAR file deployment
    @Override
    protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
        return application.sources(TuangouApplication.class);
    }

    @Bean
    public javax.validation.Validator localValidatorFactoryBean() {
       return new LocalValidatorFactoryBean();
    }
}

Configuration:

@Configuration
public class TuangouConfiguration extends WebSecurityConfigurerAdapter {

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        // @formatter:off   
        http.antMatcher("/**").authorizeRequests().antMatchers("/", "/login**").permitAll()
            .and().antMatcher("/**").authorizeRequests().antMatchers("/api/**").permitAll()
            .and().exceptionHandling().authenticationEntryPoint(new LoginUrlAuthenticationEntryPoint("/"))
            .and().formLogin().loginPage("/login").failureUrl("/login?error").permitAll()
            .and().logout().logoutSuccessUrl("/").permitAll()
            .and().csrf().csrfTokenRepository(csrfTokenRepository())
            .and().addFilterAfter(csrfHeaderFilter(), CsrfFilter.class)
            .headers().defaultsDisabled().cacheControl();
        // @formatter:on
    }

    @Order(Ordered.HIGHEST_PRECEDENCE)
    @Configuration
    @EnableGlobalMethodSecurity(prePostEnabled=true)
    protected static class AuthenticationSecurity extends GlobalAuthenticationConfigurerAdapter {

        @Override
        public void init(AuthenticationManagerBuilder auth) throws Exception {
          auth.userDetailsService(userDetailsService()).passwordEncoder(new BCryptPasswordEncoder());
        }

        @Bean
        public UserDetailsService userDetailsService() {
            return new DatabaseUserServiceDetails();
        }
    }

    private Filter csrfHeaderFilter() {
        return new OncePerRequestFilter() {
            @Override
            protected void doFilterInternal(HttpServletRequest request,
                    HttpServletResponse response, FilterChain filterChain)
                            throws ServletException, IOException {
                CsrfToken csrf = (CsrfToken) request
                        .getAttribute(CsrfToken.class.getName());
                if (csrf != null) {
                    Cookie cookie = WebUtils.getCookie(request, "XSRF-TOKEN");
                    String token = csrf.getToken();
                    if (cookie == null
                            || token != null && !token.equals(cookie.getValue())) {
                        cookie = new Cookie("XSRF-TOKEN", token);
                        cookie.setPath("/");
                        response.addCookie(cookie);
                    }
                }
                filterChain.doFilter(request, response);
            }
        };
    }

    private CsrfTokenRepository csrfTokenRepository() {
        HttpSessionCsrfTokenRepository repository = new HttpSessionCsrfTokenRepository();
        repository.setHeaderName("X-XSRF-TOKEN");
        return repository;
    }
}

Resource server config:

@Configuration
@EnableResourceServer
public class ResourceServerConfiguration extends ResourceServerConfigurerAdapter{

    @Autowired
    private TokenStore tokenStore;

    @Override
    public void configure(ResourceServerSecurityConfigurer resources)
            throws Exception {
        resources.tokenStore(tokenStore);
    }

    @Override
    public void configure(HttpSecurity http) throws Exception {
        // @formatter:off
        http.antMatcher("/**").authorizeRequests().antMatchers("/api/**").permitAll();
        // @formatter:on
    }
}
Pyrography answered 18/7, 2016 at 10:51 Comment(16)
Have you checked this SO -#21410817Bayern
@Bayern yes i did. I based my solution on answer #2 from that link.Pyrography
Your request headers have keep-alive, which indicate HTTP 1.0, but the referenced solution uses AbstractHttp11Protocol . Maybe you need to be sending HTTP 1.1 requests?Daw
What server are you using ? Tomcat ? Jetty ? ....Lunation
@Lunation The code is deployed to AWS tomcat8Pyrography
@Daw The front end is written in Angular.js. Does it not send http 1.1 request? I'm not familiar with frond end stuff...Pyrography
No, your browser will default to http 1.1 unless you have some strange configurationLunation
Could you try to add a custom tomcat customizer, like described github.com/spring-projects/spring-boot/issues/2031Lunation
If you look at response headers from stackoverflow.com do you see the gzip compression enabled ?Lunation
Are you sure you dont have server.compression.min-response-size set to something ridiculous somewhere?Nymphalid
You can verify actual properties you get at runtime by debugging line 180 at o.s.b.a.w.ServerPropertiesNymphalid
Since all properties are in place I can only suggest more debugging. You can go from TomcatEmbeddedServletContainerFactory.customizeCompression to AbstractHttp11Protocol.configureProcessor and end up at AbstractHttp11Processor around line 1458Nymphalid
@Nymphalid It looks like request for static js files never reached TomcatEmbeddedServletContainerFactory.customizeCompression. Only dynamic api result (application/json) stopped there. But compression was skipped because somehow contentLength was determined to be zero. see my screen shot update. Why would requests to static files bypass the mentioned code and why would content-length be zero for application/json response I returned? I used ResponseEntity.ok(obj) to build the API response.Pyrography
@Nymphalid your suggestions took me the furthest. I can't award you the 50 points because you didn't create an answer. Could you please create an answer instead of commenting on my question directly so that I can give you 50 points? Thanks.Pyrography
I will not do that since what i said is not an answer in any way. Instead let's try some more to tackle this thing. I tried to figure out where this 0 comes from, and it looks like there are two possibilities. 1. java.io.File#length returns 0 when path is invalid. 2. Maybe some implementations of java.net.URLConnection#getContentLength can return 0 sometimes (there are a lot of them and i dont know which one is used). Please take a look at what is going on at o.s.c.i.AbstractFileResolvingResource#contentLengthNymphalid
Btw it is starting to feel like resources location misconfiguration.Nymphalid
I
7

Maybe the problem is with YAML configuration. If you use ‘Starters’ SnakeYAML will be automatically provided via spring-boot-starter. If you don't - you must use properties convention in application.properties. Using YAML instead of Properties

EDIT: Try with this in your yml file:

server:
      compression:
        enabled: true
        mime-types: text/html,text/xml,text/plain,text/css,application/javascript,application/json
        min-response-size: 1024
Inextinguishable answered 17/8, 2016 at 10:12 Comment(1)
I tried the break point approach @Nymphalid suggested and it shows that all properties are read by spring boot container (I've updated my original post with the screenshot) but they are ignored somehow...Pyrography
G
5

Never had much luck with the Spring Boot compression. A simple solution could be to use a third party library like ziplet.

Add to pom.xml

<dependency>
    <groupId>com.github.ziplet</groupId>
    <artifactId>ziplet</artifactId>
    <version>2.0.0</version>
    <exclusions>
        <exclusion>
            <artifactId>servlet-api</artifactId>
            <groupId>javax.servlet</groupId>
        </exclusion>
    </exclusions>
</dependency>

Add to your @Config class :

@Bean
public Filter compressingFilter() {
    return new CompressingFilter();
}
Grishilda answered 18/8, 2016 at 21:22 Comment(5)
why javax.servlet.servlet-api need to exclude ? if we exclude then What will be package for Filter class ?Edmundedmunda
Spring already has a servlet library included, that's why you exclude this one. Also this post is 4 years old, I'm sure compression in Spring Boot works properly now..Grishilda
Ohh, Got it. Thanks. How we can provide configuration property to compressing filter? like change min response size, mime type etc.. in ziplet readme mentioned config property but not mentioned example i.e how to add/configureEdmundedmunda
Look at this class for hints - you'll want to create a FilterConfig and pass it to the constructor of the CompressingServlet - github.com/ziplet/ziplet/blob/master/src/main/java/com/github/…Grishilda
Note from the future: Spring Boot compression still fails even on modern versions, so you may want to take a serious look at ziplet.Magnate
I
2

If you use non-embedded Tomcat you should add this to your server.xml:

compression="on" 
compressionMinSize="2048" 
compressableMimeType="text/html,text/xml,application/javascript"

More tomcat 8 config variables

Inextinguishable answered 16/8, 2016 at 20:28 Comment(3)
I guess he doesn't have a server.xml since he uses spring bootLunation
I am having the same issue... I deploy my Spring Boot REST application to AWS Elastic Beanstalk (Tomcat instance), so there is no server.xml. Even though I specify Accept-Encoding: gzip, deflate, I don't see a Content-Encoding in the response. How do I work around this?Mcglynn
my solution was Accept-Encoding on the request header ThanksOversweet
A
2

When upgrading to Spring Boot 2.6 I had to add the following to my application.yml to allow static resources to be compressed:

spring:
  web:
    resources:
      chain:
        compressed: true

The server.compression settings seem not to impact this.

Adrianaadriane answered 29/3, 2022 at 12:26 Comment(0)
I
1

Did you try with different browsers? That could be because of antivirus which is unzipping the file as mentioned in the SO post Spring boot http response compression doesn't work for some User-Agents

Intyre answered 18/8, 2016 at 19:43 Comment(1)
tried chrom, firefox, safari. all same behavior, no content compression. :(Pyrography
R
1

you have to enable ssl like http2 mode, response compression (Content-Encoding) can work, when ssl mode is configured. Content-Encoding:gzip

response compression

application.yml

server:
  compression:
     enabled: true
     mime-types: text/html,text/xml,text/plain,text/css, application/javascript, application/json
     min-response-size: 1024
  ssl:
   enabled: true
   key-store: keystore.p12
   key-store-password: pass
   keyStoreType: PKCS12
   keyAlias: name


spring:
  resources:   
      chain:
        gzipped: true
Rivarivage answered 20/8, 2018 at 7:58 Comment(0)
S
1

I was having the same issue on production. My configuration file is as follows:

server:
    port: 8080
    compression:
        enabled: true
        mime-types: text/html,text/xml,text/plain,text/css, application/javascript, application/json
        min-response-size: 1024

Since i was not using the Embedded tomcat, I had to enable compression on the Tomcat itself by editing the following section of the server.xml file:

<Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" compression="on"/>

Make sure that this section has the compression="on" part and the issue should be resolved

Serle answered 24/2, 2022 at 8:44 Comment(0)
G
1

I ran into the same issue and spent many hours stepping through Apache and Spring code around this. I finally tracked it down to org.apache.coyote.CompressionConfig::useCompression() and found something surprising. If you're generating ETags and they're strong (that is, they don't start with W/) then compression is disabled.

I'm using Spring's ShallowEtagHeaderFilter, which. by default, generates strong ETags. My fix was to change that via setWriteWeakETag(true).

Gyrostatic answered 2/8, 2023 at 17:20 Comment(1)
Had the same issue here, had to find it by myself. Too bad no one upvoted this.Charlatan

© 2022 - 2024 — McMap. All rights reserved.