Flush output buffer in Apache/Nginx setup

Question

I would like to have page content for a web page I am developing appear on screen as it is downloaded. In my test/development environment this works as expected using the PHP flush() command.

However, my production setup (WPEngine) uses an Nginx proxy in front of Apache and flush() no longer works (nor do any of the other output buffering commands). I have been able to get the desired behaviour by deliberately filling up the buffer when I want to flush by sending 4k worth of whitespace.

However, that feels like a hack and the page in question needs to be flushed 100 times or more so this adds a considerable amount to the total data downloaded.

Is there a way to signal to Nginx to flush the buffer (or not buffer at all) by sending control characters and/or setting HTTP headers so I can avoid sending otherwise unnecessary whitespace?

Since WPEngine is a managed hosting environment, I am not able to make any changes to the server setup. So, for example, turning off Nginx buffering by adding a directive to the nginx server config is not an option.

The way I am currently doing this is as follows:-

<?php
//turn off server content compression for this page
header('Content-Encoding: none;');

//turn off PHP output buffering
ob_end_flush();

//make padding to fill buffer
$buffer = str_repeat(" ", 4096*8);

$start = time();

do
{
    printf( 'Time: %s secs<br>', time() - $start );
    echo $buffer;
    sleep(1);
} while( (time() - $start) < 10 );
?>

Supra

great! the trick is the ob_end_flush() at the start of file + setting fastcgi_buffering off; in nginx conf (if you use PHP-FPM). Don't need to overflow the buffer, remove that, just add a flush() after printf. – Polder

Answer 1

You should turn off buffering in nginx:

proxy_buffering off;

Reference: http://nginx.org/r/proxy_buffering

Apace

Thanks, but is there any way to do this without updating the server's Nginx config? – Supra

Have you read the docs on the link I provided? A quote from it: Buffering can also be enabled or disabled by passing “yes” or “no” in the “X-Accel-Buffering” response header field. – Apace

Yes, but <blush> I completely missed that bit. Unfortunately, though, adding header('X-Accel-Buffering: no;'); doesn't seem to turn buffering off, which suggests that WPEngine uses proxy_ignore_headers to prohibit that :-( – Supra

Try without ";" at the end of no, I mean: header('X-Accel-Buffering: no'); – Apace

The ; made no difference on that count or with other headers which are working. WPEngine claims they aren't preventing it with proxy_ignore_headers, but when I cURL --head to a test page while other headers I set come through fine, the X-Accel-Buffering header is absent. – Supra

Maybe they use gzip compression (it also buffers response)? Try to check without the Accept-Encoding header. – Apace

Double <blush>... I re-ran my test script and it now works as expected. I had forgotten to reinsert flush() instead of echo $buffer;. The header('X-Accel-Buffering: no'); was all that was needed. Thank you for your help. – Supra

Answer 2

While there's already an accepted answer that's somewhat correct, it's worth noting that if you're using php-fpm, fastcgi_buffering is also an issue.

The documentation is unclear, but emitting the X-Accel-Buffering: no header in your response will disable fastcgi_buffering, only in >= nginx 1.5.6. The header actually seems to affect both proxy_buffering and fastcgi_buffering, which isn't clear from the docs.

Since most distros are still running nginx 1.4 series, and a lot of people are using php-fpm, this is a potential tripping point (i.e. it hung me up for an hour).

Paraphernalia

Adding header X-Accel-Buffering: no works like a charm when you want to disable buffering for a specific request. – Schnauzer

Answer 3

It might be an nginx issue, use this in the location that is being buffered

fastcgi_keep_conn on;

If you need help knowing where exactly you need to put that then share your nginx config file.

Pigeonhole

Nginx in front of Apache doesn't use fastcgi at all. – Apace