Event Source -> Server returns event stream in bulk rather then returning in chunk
Asked Answered
C

3

7

I have a php script that import large data from csv files with validations.
For that I need to show progress to the user. I have used Event Streaming for that.
When I echo something, I want it to be transferred to client one by one instead of server sent whole output in bulk.
I had already played around with ob_start(), ob_implicit_flush() & ob_flush(), but they didn't work.
My script is working perfect on another server. Below server configurations are given:

Server configuration on which the code is not responding as desired, i.e.

OS: Linux
PHP Version 5.4.36-0+deb7u3
Server API: CGI/FastCGI 
Memory_limit: 128M
output_buffering: no value

As I have said, the code is working properly on another server which has the almost same configuration, i.e.

OS: Linux
PHP Version 5.4.37
Server API: CGI/FastCGI 
Memory_limit: 256MB
output_buffering: no value

Below is my sample code for sending event:

<?php
header("Content-Type: text/event-stream");
header("Cache-Control: no-cache");
header("Access-Control-Allow-Origin: *");

$lastEventId = floatval(isset($_SERVER["HTTP_LAST_EVENT_ID"]) ? $_SERVER["HTTP_LAST_EVENT_ID"] : 0);
if ($lastEventId == 0) {
    $lastEventId = floatval(isset($_GET["lastEventId"]) ? $_GET["lastEventId"] : 0);
}

echo ":" . str_repeat(" ", 2048) . "\n"; // 2 kB padding for IE
echo "retry: 2000\n";

// event-stream
$i = $lastEventId;

while ($i <= 100) {
    if($i==100){
        echo "data: stop\n";
        ob_flush();
        flush();
        break;
    } else {
        echo "id: " . $i . "\n";
        echo "data: " . $i . ";\n\n";
        ob_flush();
        flush();
        sleep(1);
    }
    $i++;
}
?>

Below is my client page on which I need response:

<!DOCTYPE html>
<html>

<head>
    <meta charset="utf-8" />
    <title>EventSource example</title>
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <script src="../jquery/eventsource.js"></script>
    <script>
        var es = new EventSource("events.php");
        var listener = function(event) {
            console.log(event.data);
            var type = event.type;
            if (event.data == 'stop') {
                es.close();
            } else {
                var div = document.createElement("div");
                div.appendChild(document.createTextNode(type + ": " + (type === "message" ? event.data : es.url)));
                document.body.appendChild(div);
            }
        };
        var errlistener = function(event) {
            es.close();
        }
        es.addEventListener("open", listener);
        es.addEventListener("message", listener);
        es.addEventListener("error", errlistener);
    </script>
</head>

<body>
</body>

</html>
Commuter answered 19/2, 2015 at 13:31 Comment(2)
you mean your live http strimming is working on other server ????Synthiasyntonic
Yes, it is working on another server... but not with the new oneCommuter
L
6

Your best method to return chucked data to the browser is to use Web Sockets get the client to open a socket to your file reader then you can chunk the data to the browser without a problem.

Then once it has finished you can close the socket.

a good tutorial for web sockets http://www.phpbuilder.com/articles/application-architecture/optimization/creating-real-time-applications-with-php-and-websockets.html

with this method you can then if you wanted implement verification so the server is not just sending chunks it's sends the chunks on request by javascript

So your Client could say i need chunk 5 and your server implement something like

$requestedChunk = 5; // this would be set by the javascript sending the request
$chunkSize = 256; // this would be your chunk size;

$readPossition = $requestedChunk * $chunkSize;

Link no longer works so here is one built on Ratchet: https://blog.samuelattard.com/the-tutorial-for-php-websockets-that-i-wish-had-existed/

Leukocyte answered 1/12, 2015 at 11:36 Comment(2)
Linked page is 404.Philina
it's actually a 403 just showing as 404 looks at the address you are directed to but use google "php WebSocket server example" and you will find plentyLeukocyte
A
0

I had a similar problem. Event streams were working as expected (returning chunks) on a server using the Apache 2.0 Handler but not on a server using FastCGI (returning it in bulk). I assumed that something in FastCGI is the culprit and so tried to resolve the problem by switching to CGI. Now the event stream works as expected.

Whether using CGI or FastCGI the Server API shows up as CGI/FastCGI so I assume that the server it works on for you is running CGI and the server it doesn't work on for you is running FastCGI. Try changing the non-working server to CGI.

As for why it doesn't work in FastCGI I'm not entirely sure but unless it's a necessary requirement and CGI isn't possible then the above solution should work.

Arsenide answered 27/11, 2015 at 16:57 Comment(1)
it's not just FastCGI / even mod_php had this problemLeukocyte
P
0

Many things can prevent chunked response such as but not limited to;

  • Proxy or any other buffering mechanism on web server
  • When "Output buffering" is "on" in php.ini (you should explictly set it to off)
  • When gzip is enabled on web server

You should check these at first.

Paletot answered 4/12, 2015 at 15:16 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.