What are the "serious performance implications" of implicit_flush?

Question

My site's admin section has a bunch of very slow report-generating scripts that echo output line by line as it is generated. To have this output flushed immediately to the browser, instead of the user having to wait for minutes before they see any response, we have output_buffering disabled and we call ob_implicit_flush at the beginning of such scripts.

For convenience, I was considering just turning on the implicit_flush setting in php.ini instead of adding ob_implicit_flush() calls to every script that would benefit from it.

However, the documentation contains the following scary but unexplained remark:

implicit_flush

...

When using PHP within an web environment, turning this option on has serious performance implications and is generally recommended for debugging purposes only.

What are these "serious performance implications", and do they justify the manual's recommendation?

Whiskey

Although I've self-answered this with one answer I was able to uncover, I remain interested to learn if there are others. It is quite possible that the performance trap I describe in my answer is in fact not the one that the manual is alluding to. – Whiskey

Answer 1

It may or may not be what the manual is hinting at, but one context in which either turning on implicit_flush or calling ob_implicit_flush() has serious performance implications is when using PHP with Apache through mod_php with mod_deflate enabled.

In this context, flush() calls are able to push output all the way through mod_deflate to the browser. If you have any scripts that echo large amounts of data in small chunks, flushing every chunk will cripple mod_deflate's ability to compress your output, quite possibly resulting in a 'compressed' form that is larger than the original content.

As an extreme example, consider this simple script which echoes out a million random numbers:

<?php
    header('Content-Type: text/plain');

    for ($i=0; $i < 1000000; $i++) { 
        echo rand();
        echo "\n";
    }
?>

With output_buffering off and implicit_flush also off (for now), let's hit this in Chrome with the dev tools open:

Note the Size/Content column; the decompressed output is 10.0MB in size, but thanks to mod_deflate's gzip compression, the entire response was compressed down to 4.8MB, roughly halving it in size.

Now hitting exactly the same script with implicit_flush set to On:

Once again, the 'decompressed' output is 10.0MB in size. This time, though, the size of the HTTP response was 28.6MB - mod_deflate's 'compression' has actually trebled the size of the response.

This, for me, is more than enough reason to heed the PHP manual's advice of leaving the implicit_flush config option off, and only using ob_implicit_flush() (or manual flush() calls) in contexts where doing so actually serves a purpose.

Whiskey

Do you know what’s happening on the TCP layer? Maybe multiple reassembled TCP packets for the latter scenario? – Danzig

@Danzig No idea; I'm afraid that's closer to the metal than I've ever needed to know or think about professionally. Now that you've asked, though, I'll try to find out. – Whiskey