http_sub_module / sub_filter of nginx and reverse proxy not working

Question

I am trying to reverse proxy my website and modify the content. To do so, I compiled nginx with sub_filter. It now accepts the sub_filter directive, but it does not work somehow.

server {
    listen       8080;
    server_name  www.xxx.com;

    access_log  /var/log/nginx/www.goparts.access.log  main;
    error_log  /var/log/nginx/www.goparts.error.log;
    root   /usr/share/nginx/html;
    index  index.html index.htm;

    ## send request back to apache1 ##
    location / {
       sub_filter <title> '<title>test</title>';
 sub_filter_once on;


     proxy_pass  http://www.google.fr;
     proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
     proxy_redirect off;
     proxy_buffering off;
     proxy_set_header        Host            $host;
     proxy_set_header        X-Real-IP       $remote_addr;
     proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

   }
}

Please help me

Allowedly

Why do you intend to modify content within Nginx configuration? Do you want to use Nginx in forward-proxy mode? – Impetrate

Because I intend to add a header and footer to the website this way. I want to use it in reverse proxy mode. – Allowedly

I even tried Httpsubs and subs_filter, it doesn't work either. Anyone has an idea ? – Allowedly

Based on proxy_pass google.fr it seems you try to build forward proxy but not reverse one? – Impetrate

Yes, google just to try it out. I now tried to reverse proxy an apache directory listing default page and it works. I read somewhere that httpsubs had problems with gzip-served pages so I added proxy_set_header Accept-Encoding ""; to try to disable gzip from server but no way :/ – Allowedly

sub_filter doesn't have a problem with gzip content, it just doesn't designed to work with that, the empty header Accept-Encoding helps to let backend send non-gzipped content to Nginx. – Impetrate

Answer 1

Check if the upstream source has gzip turned on, if so you need

proxy_set_header Accept-Encoding "";

so the whole thing would be something like

location / {
    proxy_set_header Accept-Encoding "";
    proxy_pass http://upstream.site/;
    sub_filter_types text/css;
    sub_filter_once off;
    sub_filter .upstream.site special.our.domain;
}

Check these links

• https://www.ruby-forum.com/topic/178781
• https://forum.nginx.org/read.php?2,226323,226323
• http://www.serverphorums.com/read.php?5,542078

Salford

I thought this was an ugly fix since it would disable gzip between origin server and public server (more back-end traffic), but it is Nginx official solution for this: nginx.com/resources/wiki/modules/substitutions – Gesticulative

I would also add that in my case, it only worked once I set the proxy_set_header Accept-Encoding ""; inside location directive, it would not work under server. – Gesticulative

@Gesticulative - Most likely that is because any usage of proxy_set_header in a block requires repeating any desired usage from parent blocks. So if your location block had already used proxy_set_header and you tried to set it globally in the server block, it would have no effect within your location block unless you repeated it. This is just an nginx-ism and not specific to the features used here. – Halfsole

When I add Accept-Encoding ""; it gives me a gateway timeout, but when I do the request to the upstream server with that header it works fine, any idea how it could be? – Tumer