Upstream too big - nginx + codeigniter

Question

I am getting this error from Nginx, but can't seem to figure it out! I am using codeigniter and am using the database for sessions. So I'm wondering how the header can ever be too big. Is there anyway to check what the header is? or potentially see what I can do to fix this error?

Let me know if you need me to put up any conf files or whatever and I'll update as you request them

2012/12/15 11:51:39 [error] 2007#0: *5778 upstream sent too big header while reading response header from upstream, client: 24.63.77.149, server: jdobres.xxxx.com, request: "POST /main/login HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "jdobres.xxxxx.com", referrer: "http://jdobres.xxxx.com/"

UPDATE

I added the following into conf:

proxy_buffer_size   512k;
proxy_buffers   4 512k;
proxy_busy_buffers_size   512k;

And now I still get the following:

2012/12/16 12:40:27 [error] 31235#0: *929 upstream sent too big header while reading response header from upstream, client: 24.63.77.149, server: jdobres.xxxx.com, request: "POST /main/login HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "jdobres.xxxx.com", referrer: "http://jdobres.xxxx.com/"

Hutton

possible duplicate of upstream sent too big header while reading response header from upstream – Suber

#31808900 – Mcmanus

Hey another possible issue could be that you have not restarted your php5-fpm service. Just for others as a possible solution. – Chaldea

Answer 1

Add this to your http {} of the nginx.conf file normally located at /etc/nginx/nginx.conf:

proxy_buffer_size   128k;
proxy_buffers   4 256k;
proxy_busy_buffers_size   256k;

Then add this to your php location block, this will be located in your vhost file look for the block that begins with location ~ .php$ {

fastcgi_buffer_size 128k;
fastcgi_buffers 4 256k;
fastcgi_busy_buffers_size 256k;

Scalar

What's the difference between the two? – Hutton

For more information on the HttpProxyModule module visit wiki.nginx.org/HttpProxyModule and for the HttpFastcgiModule visit wiki.nginx.org/HttpFastcgiModule – Scalar

Thanks. From the looks of the docs I should be able to add it to the http block as well. I'm wanting to do it that way as I have multiple subdomain servers setup, so doing it in 1 place would be nicer. Have you run into issues with putting it in http instead of in location? – Hutton

The fastcgi_buffers worked. I was able to just put it under server and it worked there. – Hutton

Note: The lines starting with proxy_ need to be inserted into the http{} section of nginx.conf. – Elastance

Thsnks! Didn't notice for first time that instructions in blocks are different proxy_* vs fastcgi_*. – Majolica

Can you explain what fastcgi_buffers 4 256k actually means? I'm finding it hard to grok the NGINX documentation. – Cowan

Could you explain why did you set the fastcgi_buffer_size to 128k but not other number? How can I measure this in my server for nginx to work properly? – Capers

"proxy_busy_buffers_size" must be less than the size of all "proxy_buffers" minus one buffer – Kindergartner

You could also just add it to the location ~ .php$ block; you don't have to add it both places. – Unders

An old answer but still relevant today. The proxy_* parameters helped with my Kubernetes nginx ingresses and fastcgi for the proxy sitting in front of my php-fpm instances. Thanks! – Elayne

Answer 2

Modify your nginx configuration and change/set the following directives:

    proxy_buffer_size   128k;
    proxy_buffers   4 256k;
    proxy_busy_buffers_size   256k;

Toluol

Where would I put this? Inside of the http section or the server section? I'm assumign http? – Hutton

/etc/nginx/nginx.conf. http block – Toluol

Awesome. I'm testing it out now. I increased all 3 to 512k instead just in case. I'm having a few people test to see if it works. =D – Hutton

Maybe you have circular redirects? – Toluol

I'm not seeing any redirects in my access log before the failures occur =/ – Hutton

I get this error using this config: "proxy_busy_buffers_size" must be less than the size of all "proxy_buffers" minus one buffer – Kindergartner

Answer 3

using nginx + fcgiwrap + request too long

I had the same problem because I use a nginx + fcgiwrap configuration:

location ~ ^.*\.cgi$ {
    fastcgi_pass  unix:/var/run/fcgiwrap.sock;
    fastcgi_index index.cgi;
    fastcgi_param SCRIPT_FILENAME /opt/nginx/bugzilla/$fastcgi_script_name;
    include /etc/nginx/fastcgi_params;
    # attachments can be huge
    client_max_body_size 0;
    client_body_in_file_only clean;
    # this is where requests body are saved
    client_body_temp_path /opt/nginx/bugzilla/data/request_body 1 2;
}

and the client was doing a request with a URL that was about 6000 characters (a bugzilla request).

debugging...

location ~ ^.*\.cgi$ {
    error_log /var/log/nginx/bugzilla.log debug;
    # ...
}

This is what I got in the logs:

2015/03/18 10:24:40 [debug] 4625#0: *2 upstream split a header line in FastCGI records
2015/03/18 10:24:40 [error] 4625#0: *2 upstream sent too big header while reading response header from upstream, client: 10....

can I have "414 request-uri too large" instead of "502 bad gateway"?

Yes you can! I was reading How to set the allowed url length for a nginx request (error code: 414, uri too large) before because I thought "hey the URL's too long" but I was getting 502's rather than 414's.

large_client_header_buffers

Try #1:

# this goes in http or server block... so outside the location block
large_client_header_buffers 4 8k;

This fails, my URL is 6000 characters < 8k. Try #2:

large_client_header_buffers 4 4k;

Now I don't see a 502 Bad Gateway anymore and instead I see a 414 Request-URI Too Large

"upstream split a header line in FastCGI records"

Did some research and found somewhere on the internet:

• http://forum.nginx.org/read.php?2,4704,4704
• https://www.ruby-forum.com/topic/4422529
• http://mailman.nginx.org/pipermail/nginx/2009-August/014709.html
• http://mailman.nginx.org/pipermail/nginx/2009-August/014716.html

This was sufficient for me:

location ~ ^.*\.cgi$ {
    # holds request bigger than 4k but < 8k
    fastcgi_buffer_size 8k;
    # getconf PAGESIZE is 4k for me...
    fastcgi_buffers 16 4k;
    # ...
}

Suber

Answer 4

I have proven that this is also sent when an invalid header is transmitted. Invalid characters or formatting of the HTTP headers, cookie expiration set back by more than a month, etc will all cause: upstream sent too big header while reading response header from upstream

Sanguineous

I had the case that an error in the PHP file (could not write to errorlog) caused this nginx error message incl. "Bad Gateway" result. – Clown

Answer 5

I encountered this problem in the past (not using codeigniter but it happens whenever the responses contain a lot of header data) and got used to tweaking the buffers as suggested here, but recently I got bitten by this issue again and the buffers were apparently okay.

Turned out it was spdy's fault which I was using on this particular project and solved by enabling spdy headers compression like this:

spdy_headers_comp 6;

Gerson

Answer 6

upstream sent too big header while reading response header from upstream

I got this nginx erorr when trying to set a cookie in upstream Apache server (by PHP), the cookie being over 4096 bytes caused this error in my case.

I initially thought the user submitted form was too long, but I was saving a large user form field in a cookie and that can cause the error.

Drouin

Answer 7

Problem: upstream sent too big header while reading 
response header from upstream Nginx with Magento 2

Solution: Replace given below setting into /nginx.conf.sample File

• fastcgi_buffer_size 4k;

with

• fastcgi_buffer_size 128k;
• fastcgi_buffers 4 256k;
• fastcgi_busy_buffers_size 256k;

Jenn

Please read How do I write a good answer? – Majors