How to address weak etags conversion by nginx on gzip compression

Question

I am trying to compress response bodies for certain endpoints using gzip in nginx. The problem is with nginx marking the upstream apps generated etags as weak(prefixed with "W/"). The upstream apps don't have weak etag support yet(spring version < 4.3). When clients send back weak etag, it wont match with app computed strong etag, I don't see 304 status but a 200 with body. Even if apps have weak etag, its easier to manage compression in one layer than modify all the apps, upgrade them and enable weak tags for now.

I am trying two options:

1. When upstream server sends a strong etag and nginx gzip modifies it to a weak one, try nginx lua API to modify it back to strong.

2. When clients send weak etags back, strip off the weak etag identifier("W/") and forward the request to apps.

I must be doing something wrong in the nginx config and lua API usage that I am not able to achieve this. This issue is about option 1.

Nginx config:

  location /test/compression {
  proxy_pass              http://upstream_server:8080/someapi;
  proxy_redirect          default;
  proxy_set_header        X-Real-IP               $remote_addr;
  proxy_set_header        X-Forwarded-For         $proxy_add_x_forwarded_for;

  include compression.conf;

  header_filter_by_lua_block {
          ngx.header["ETag"] = string.substring(ngx.header["ETag"], 2);
      }
  }

compression.conf

gzip on;
gzip_http_version 1.0;
gzip_proxied any;
gzip_types application/json application/octet-stream;
gzip_min_length 10000;
gzip_comp_level 7;

Actual result: Error in nginx log:

nginx  | 2019/03/21 14:11:06 [error] 38#38: *8 failed to run header_filter_by_lua*: header_filter_by_lua:2: attempt to call field 'substring' (a nil value)
nginx  | stack traceback:
nginx  |    header_filter_by_lua:2: in function <header_filter_by_lua:1> while reading response header from upstream, client: 127.0.0.1, server: _, request: "GET /test/compression HTTP/1.1", upstream: "http://upstream_server:8080/someapi", host: "localhost:9696"

Expected result: Strong ETag in the response to client

Also tried another way to retrieve the ETag header after going through this: nginx - read custom header from upstream server

  location /test/compression {
  proxy_pass              http://upstream_server:8080/someapi;
  proxy_redirect          default;
  proxy_set_header        X-Real-IP               $remote_addr;
  proxy_set_header        X-Forwarded-For         $proxy_add_x_forwarded_for;

  include compression.conf;
  set $etag $upstream_http_etag

  header_filter_by_lua_block {
          ngx.header["ETag"] = string.substring(ngx.var.etag, 2);
      }
  }

Same error.

Wootan

Error log you have attached indicates that it cannot find substring function in string. The reason is - it does not exist. Substring function is just sub, so string.sub(...). Let's start with that and we'll see where we go from there. – Oshiro

Thanks. That error was gone after I used the correct sub string function. But I still have the problem modifying the etag header value. From my analysis so far, nginx gzip runs after "header_filter_by_lua_block" and overrides the etag header value to prefix with "W/". I tried with other upstream header values and they all got modified except for ETag. Lets say upstream server responds with etag value "A", I override it to "B" in lua block and the client response has W\"B" in the etag header. If I remove the lua block I get W/"A". I just want "B" as etag header in the response to client. – Wootan

Related: javorszky.co.uk/2019/03/28/etag-if-match-nginx-and-you – Agnesagnese

Answer 1

Finally, I ended up modifying the "If-None-Match" request header from client instead.

1. If client sent a weak etag in "If-None-Match" request header, modify it to strong etag in nginx rewrite phase using lua or set nginx directive.
2. If client sent a strong etag in "If-None-Match" request header, leave it as is.
3. In case of a 304 response from upstream, nginx gzip module isn't invoked and strong etag goes back to client. Ti handle this I left the header_filter_by_lua_block block as is so that strong to weak etag conversion is done for 304 responses.

This solution worked for me. Any suggestion to do it in better way is welcome.

Wootan

2. how to leave etag as is? – Mcwhorter

Answer 2

Nginx "weakens" the ETag response header whenever it modifies the response body (see this code search), this includes compressing it.

To prevent this from happening in the first place, you can disable the gzip module (gzip off), or try excluding specific content types if you have that pre-knowledge.

Another option is a bit more hacky - Try adding Content-Encoding: identity to the response headers (only if there is no content encoding header in the first place). I did not try it, but reading the source code it should work:

ngx_http_gzip_filter_module.c L220:

    if (!conf->enable
    || (r->headers_out.status != NGX_HTTP_OK
        && r->headers_out.status != NGX_HTTP_FORBIDDEN
        && r->headers_out.status != NGX_HTTP_NOT_FOUND)
    || (r->headers_out.content_encoding
        && r->headers_out.content_encoding->value.len)
    || (r->headers_out.content_length_n != -1
        && r->headers_out.content_length_n < conf->min_length)
    || ngx_http_test_content_type(r, &conf->types) == NULL
    || r->header_only)
{
    return ngx_http_next_header_filter(r);
}

Manrope