502 Bad Gateway error - upstream sent too big header while reading response header from upstream

Question

I'm using wordpress in Google App Engine. When I make a POST API, getting an Error : "upstream sent too big header while reading response header from upstream". which returns 502, Bad Gateway, nginx

The data(JSON) i'm sending in POST API is around 4kb. If the Data is sent below 2kb API returns success.

I tried App Engine Standard and Flexible Environment, but facing the same issue.

As per this link: upstream sent too big header while reading response header from upstream

Modifying the nginx-app.config file with buffers will fix this issue. But it's not working.

In App Engine, the default nginx-app.config looks like.

location / {
    try_files $uri /index.php?q=$uri&$args;
}

location ~ ^/wp-admin {
    try_files $uri $uri/index.php?$args;
}

Not sure where I need to add these proxy buffers.

I tried adding the proxy buffers inside the default locations as shown below, but it didn't help.

location / {
    try_files $uri /index.php?q=$uri&$args;
    fastcgi_buffers 16 16k;
    fastcgi_buffer_size 32k;
    proxy_buffer_size 128k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;
}

Kindly help me to fix this issue.

Nessy

Answer 1

This is expected behavior as the nginx config for App Engine is using the default proxy_buffer_size which is 4K.

The workaround is to emit HTTP headers that are cumulatively under this limit. There is an open feature request to increase this however it seems it is unlikely to be increased to 128K. If increasing to 128K is the only solution for your use case I suggest creating a request for it with this issue tracker and include the business impact to illustrate the need for the feature.

Infect

Thanks Julie Wang for your reply. Sorry I couldn't understand your workaround. I have even tried reducing the buffer values to 16k, but it didn't work. – Nessy

As mentioned, you will need to reduce HTTP headers that are as a whole under this 4K limit. The limit seems to be with the proxy_buffer_size so reducing to to 4K should allow you complete the API call. There is a feature in the works that would increase the limit 8k to allow for larger header blocks however it will likely be over a month (or more) before it is rolled out so I suggest using the workaround until then. – Infect

I have two more clarification. 1) Actually data i'm sending is not in HTTP Header, it is sent in POST API Body. 2) In Flexible Environment i'm getting this error even for data around 3kb. In Standard Environment even for data greater than 1kb i'm getting the error. – Nessy

Can you provide the new buffer configurations you are using if it differs from the original post? Perhaps something else about the configuration is the issue. Also I noticed that most other examples of the error is longer than provided, if you are still experiencing the error can you provide the full error message – Infect

Actually the issue i'm getting is related with http response. I misunderstood it with http request data. Thanks for your support Julie Wang. – Nessy

Answer 2

Wordpress reverse proxy

Put it under the server group (it is a server property):

server {
        listen 443 ssl http2;

        server_name wordpress.site.com;
 
        fastcgi_buffers 16 16k;
        fastcgi_buffer_size 32k;
        proxy_buffer_size   128k;
        proxy_buffers   4 256k;
        proxy_busy_buffers_size   256k;     

       location ~ \.php$ {     
            ... 
             }
 ...
}

And the problem will be solved!

Grainfield

Answer 3

The root cause of this problem is that the upstream server set a header that is larger than what Nginx is configured to handle. This may be for instance if the upstream server sets a large cookie header. There are two ways to fix it:

1. Decrease the size of the headers that your upstream server sets (e.g., decrease the size of the cookie)
2. If you think the larger header is OK, configure Nginx to handle larger headers as below

Configuring Nginx to handle larger headers can be achieved by setting the proxy buffer size in the Nginx configuration under server or http (not under location):

# To prevent error "upstream sent too big header while reading response header from upstream" on /my-endpoint-here
proxy_buffer_size           16k;
proxy_buffers               8 16k;
proxy_busy_buffers_size     16k;

The default for the proxy buffer is either 4k or 8k, see documentation. Note that you also need to set the proxy_buffers configuration because it will give the validation error "proxy_busy_buffers_size" must be less than the size of all "proxy_buffers" minus one buffer.

If this is indeed about a large cookie, you probably also need to expand the buffer for large client headers:

large_client_header_buffers  4 16k;

The default for the large client header buffer is 8k, see documentation.

Samaria