YSlow gives F grade to files compressed with mod_deflate

Question

I'm using mod_deflate on Apache 2.2 and the compression level is set to 9. I've fine tuned every possible aspects of the site based on the recommendations of YSlow (v2) and have managed to get an overall A grade (Total Score: 91) as well as on all categories except for:

• Make fewer HTTP requests (Grade C - I'm still working on further unification of images)
• Compress components with gzip (Grade F)

YSlow still reports back with a F and tells me to use gzip on my CSS and JS files. Here's a screenshot of the YSlow report (the domain has been blurred out for the sake of privacy):

However, sites like GIDNetwork GZIP Test reports perfect compression!!

mod_deflate section of my .htaccess

# Below uses mod_deflate to compress text files. Never compress binary files.
<IfModule mod_deflate.c>
SetOutputFilter DEFLATE

# compress content with type html, text, js, and css
AddOutputFilterByType DEFLATE text/html text/plain text/css text/javascript text/xml image/svg+xml application/javascript application/x-javascript application/atom_xml application/rss+xml application/xml application/xhtml+xml application/x-httpd-php application/x-httpd-fastphp

# Properly handle old browsers that do not support compression  
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

# Explicitly exclude binary files from compression just in case
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.avi$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.mov$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.mp3$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.mp4$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.rm$ no-gzip dont-vary

# properly handle requests coming from behind proxies
Header append Vary User-Agent env=!dont-vary
</IfModule>

Can anyone please point out where I'm going wrong?

Thanks, m^e

Adame

Answer 1

It's possible that mod_deflate has been configured incorrectly.

A typical mod_deflate configuration may be excluding certain browsers based on user-agent strings, and may only be configured to compress certain file types - identified by their MIME type as registered on the server.

You should be compressing all of your HTML, CSS and Javascript files, but not your PNG, GIF or JPEG files, and there are bugs with Netscape 4 you may or may not want to account for. Try using the sample code from the documentation:

<Location />
    # Insert filter
    SetOutputFilter DEFLATE

    # Netscape 4.x has some problems...
    BrowserMatch ^Mozilla/4 gzip-only-text/html

    # Netscape 4.06-4.08 have some more problems
    BrowserMatch ^Mozilla/4\.0[678] no-gzip

    # MSIE masquerades as Netscape, but it is fine
    BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
    # Don't compress images
    SetEnvIfNoCase Request_URI \
    \.(?:gif|jpe?g|png)$ no-gzip dont-vary

    # Make sure proxies don't deliver the wrong content
    Header append Vary User-Agent env=!dont-vary
</Location> 

Note too that the GIDZipTest GZIP test you posted does not test associated Javascript and CSS files, whereas YSlow does - in GIDZipTest GZIP test you'd need to test these individually.

I guess it is also possible that your ISP is using a caching proxy - transparent or not - which is mangling or removing your Accept-Encoding: header. To rule this out as the cause you could get someone to test it from outside of your ISP.

Another thing to note is that when compressing files using gzip you are trading bandwidth for CPU time. Above the lower compression strengths you will see diminishing returns in bandwidth savings, but huge increases in CPU time required. Unfortunately with a compression strength as high as 9, you are almost certainly wasting too much CPU time for very little improved compression - I would always recommend using strength of 1.

Imperceptible

I've added the deflate portion of my htaccess above... it more or less matches the example you've shown here. As for the CPU time, we're running on a dedicated server with no other processes (except for MySQL) running along with Apache. Should the CPU utilization be of concern in such a scenario ? – Adame

As for the GIDZipTest, I followed your suggestion and checked for compression on the JS and CSS files individually instead of the whole page... and it reported a compression of 70% and 77.3% respectively. – Adame

Is it possible that on Firefox, you are accessing the site through a proxy? It's possible that a proxy could be dropping the Accept-Encoding: header that your browser sends, in order to cache it (the proxy could also be transparent). Getting someone from outsite your ISP to check it with their own Firefox would confirm or rule that out as the cause. – Imperceptible

I'd completely forgotten about this post... as midway my problem got solved. Indeed it was due to a proxy employed at my workplace. They got rid of it and all was fine :) Thanks for all the help.. – Adame

Answer 2

this website http://www.rubyrobot.org/article/5-tips-for-faster-loading-web-sites told me that AddOutputFilterByType will not work in .htaccess

Newsom

That's only partially true. AddOutputFilterByType is available in Apache 2.0.33 and later, even 2.2 still has it (it's deprecated since 2.1, though). And it's available in all contexts, including .htaccess. – Fourfold