PHP Event Source keeps executing

Question

I started using push in HTML5 using the JavaScript EventSource object. I was totally happy with a working solution in PHP:

$time = 0;
while(true) {
    if(connection_status() != CONNECTION_NORMAL) {
        mysql_close()
        break;
    }
    $result = mysql_query("SELECT `id` FROM `table` WHERE UNIX_TIMESTAMP(`lastUpdate`) > '".$time."'");
    while($row = mysql_fetch_array($result)) {
        echo "data:".$row["id"].PHP_EOL;
        echo PHP_EOL;
        ob_flush();
        flush();
    }
    $time = time();
    sleep(1);
}

But suddenly my WebApp wasn't reachable anymore with an MySQL error "too many connections".

It turned out that the MySQL connection doesn't close after closing the event source in JavaScript:

window.onload = function() {
    sse = new EventSource("push.php");
    sse.onmessage = function(event) {
        console.log(event.data.split(":")[1]);
    }
}
window.onbeforeunload = function() {
    sse.close();
}

So I guess that the PHP script does not stop to execute. Is there any way to call in function (like die();) before the clients connection disconnects? Why doesn't my script terminate after calling .close(); on the EventSource?!

Thanks for help! —

Discontinuous

It seems that every client session starts a new server side infinite loop with its own mysql connection, doesn't it? I see not the connection start on the example you posted, so I'm not sure. Anyway, That's why "too many connections" error appears. Server side service which serves the data could use (share) a single db connection (and a single infinite loop either). I'm not posting this as an answer because I haven't got an example at my hand. Haven't been using PHP since couple of years now. Though I see on php.net mysql_query is a bit outdated. They advice to use MySQLi or PDO_MySQL instead. – Tatterdemalion

Kind of. Each client session got it's own mysql connection. The connection starts before the infinite loop. The real problem is, that calling EventSource.close() does not close or abort the connection itself! – Discontinuous

As I understand served content through push.php is the same for each client, isn't it? Why then connect to database and query per each client session instead of sharing the same SQL results among all client sessions? Despite it's more optimal solution it will also get around your problem. – Tatterdemalion

Sounds like I should try it... I need my script to push informations to the client as another client changes the database. How can I share the results to all clients without establishing a DB connection per each client?! I need to PUSH information immediately. – Discontinuous

I know no exact answer without further research. I used PHP years ago. But try to search for things like 'application scope', 'stateful service' and 'cron jobs' adding key word PHP. Maybe you can stick together something. – Tatterdemalion

Answer 1

First off: when you wrap your code in a huge while(true) loop, your script will never terminate. The connection with the DB would have been closed when your script "ran out of code to execute", but since you've written a deadlock, that's not going to happen... ever.
It's not an EventSource issue, that merely does what it's supposed to do. It honestly, truly, and sadly is your fault.

The thing is: a user connects to your site, and the EventSource object is instantiated. A connection to the server is established and a request for the return value of push.php is requested. Your server does as requested, and runs the script, that -again- is nothing but a deadlock. There are no errors, so it can just keep on running, for as long as the php.ini allows it to run. The .close() method does cancel the stream of output (or rather it should), but your script is so busy either performing its infinite loop, or sleeping. Assuming the script to be stopped because a client disconnects is like assuming that any client could interfere with what the server does. Security-wise this would be the worst that could happen. Now, to answer your actual question: What causes the issue, how to fix it?
The answer: HEADERS

Look at this little PHP example: the script isn't kept alive server-side (by infinite loops), but by setting the correct headers.

Just as a side-note: Please, ditch mysql_* ASAP, it's being deprecated (finally), use PDO or mysqli_* istead. It's not that hard, honestly.

Asberry

To follow your arguments: calling close(); doesn't change the connection state returned by connection_status();? Since when can you edit headers from JavaScript?! When I terminate the script from PHP, new data will never be pushed to the client. When setting the header to "text/event-stream" (as I do) would never tell mysql to check for updates, right? – Discontinuous

@Julian: invoking .close() does change the connecion_status. You can set the request headers in JS (just look at ajax calls: setRequestHeader), but I was referring to PHP's header('some header');. You don't have to explicitly terminate the PHP script either. Just look at the link I provided, it contains more details and provides quite clear examples on how to get this working – Asberry

I know the link. To which paragraph do re refer? I need the PHP script to check continuously my database. In my PHP I wrote the connection status every second into a text file. When I invoke .colse(), the status does NOT change. Unfortunately. I can't change the header in PHP, because the script has to run continuously until the client disconnects! while(connection_staus() == CONNECTION_NORMAL) {} does not work either... – Discontinuous

@Julian: Well, there's a PHP sample in the link I posted. Because the headers will be set to a keep-alive and event-stream header the connection to your DB will persist until the .close() method is invoked... That's it, really. You don't need any loop wizardry, not continuous checks. Its behaviour is comparable to node.js: sending chunked data from a server that has listeners and spends most of its time idling... Just copy paste the code example from the link (first snippet under title Server Samples) and use that to test – Asberry

Well... That does not work. I'm using "text/event-stream" since I started working on push. When I remove the while loop, the Event Source object returnes the on error function and closes the connection. When I have no loop, the script finishes execution and terminates. The stream is closed then. The header seams to doesn't bother it. Invoking .close() still does NOT change the connection_status. Can't imagine why... – Discontinuous

@Julian: What is the readyState in the error callback? is it CLOSED? in which case: make sure you don't have a mysql_close or $pdo = null; somewhere in your php code, also check the JS source of this example, maybe there's some inspiration to be gotten from there – Asberry

The readyState is 2 (aka CLOSED). I don't have mysql_close or something in my code. I wanted to implement it after the loop... I know this example too. They do nothing different than I do. It's kind of the same – Discontinuous

@Julian: I just realized something: you're flushing the buffer inside the loop, EventSource isn't the same as a socket stream, you should either return the whole output in one lump or use a websocket for continuous data transfer – Asberry

let us continue this discussion in chat – Discontinuous

@EliasVanOotegem, the example you referenced in your link is not kept alive by Headers. In fact, it actually dies and the client re-opens the connection after three seconds. – Tropaeolin

Answer 2

I had exactly the same issue and as far as I understood it the reason was that apache didn't terminate the php script until I tried to write data trough the closed socket connection again.

I didn't have any changes in my database so there was no output.

To fix this I simply echo a event source comment in my while loop like:

echo ": heartbeat\n\n";
ob_flush();
flush();

This causes the script to be terminated if the socket connection is closed.

Nicolettenicoli

Yeah, this will work. But this prodces a "massive" netwok overhead in comparison. This is not much better than polling constantly... – Discontinuous

Depends on how often you send it. You don't need to do it on every loop. You can do this every 15 minutes or so... The important thing is that the script terminates and not really if it terminates immediately. – Nicolettenicoli

Agree, your totally right. But I always try to make things perfect ;) So you understand why this doesn't "taste" so well :D But yeah, thanks for your suggestion, this definitely makes it doable, even if not perfect. I just think PHP's not the right thing for making this perfect. – Discontinuous

I like to do things perfect too I get what you mean. Being able to do this with php like the whole rest of my application makes it a lot easier though. If I would have to do the event source part with something else I'd maybe switch to WebSockets. – Nicolettenicoli

Answer 3

you might want to try to add this to the loop:

if(connection_status() != CONNECTION_NORMAL)
{
    break;
}

but php should stop when the client disconnected.

Vanegas

I was assuming PHP should stop too. I tried your solution with the if statement, then mysql_close(); then break;. It doesn't help. When I go to my MySQL client and type SHOW PROCESSLIST my WebApps user is still connected. Every time I reload the page it connects one time more… Maybe it's JS EventSource.clos(); issue??? – Discontinuous

Would if(connection_aborted()) be a better check? – Jac

Answer 4

1.

window.onbeforeunload = function() {
    sse.close();
}

this is not required, EventSource will be closed at page unloading time.

1. even if you can not find a way to detect disconnected client on PHP, you can just stop execution after N seconds, EventSource will reconnect automatically.

http://www.php.net/manual/ru/function.connection-aborted.php comments on this page says, that u should use "flush" before connection_status anyway, you should drop connection after N seconds, because u cant detect "bad" disconnects.

3. echo "retry:1000\n"; // use this to tell EventSource the reconnection delay in ms

Obreption

Thanks, thats kind of a workaround. But it doesn't seem to be the right way either, since EventStream gets an error when the PHP Script stops execution. – Discontinuous

and? this is ok to get this error, you should check readyState in error handler, if it is EventSource.CONNECTED - then ES will try to reconnect – Obreption

Answer 5

$time = 0;
while(true) { 
    if(connection_status() != CONNECTION_NORMAL) {
        mysql_close()
        break;
    }
    $result = mysql_query("SELECT id FROM table WHERE UNIX_TIMESTAMP(lastUpdate) > '".$time."'");
    while($row = mysql_fetch_array($result)) {
        $rect[] = $row;
    }
    for($i-0;$i echo "data:".implode('',$disp)."\n\n"; $time = time(); sleep(1);
}

Katlynkatmai

Answer 6

As Elias Van Ootegem pointed out, your script never terminates and hence you have a bunch of MySQL connections active. More concerningly, you have a bunch of resources being used in the form of PHP loops running with no end in sight!

Unfortunately, the solution to keeping the is not "headers". In the example at this link referenced by Elias, the php script terminates, and the client javascript actually has to re-open a connection. You can test this by using the following code

source.addEventListener('open', function(e) {
  // Connection was opened.
    console.log("Opening new connection");
}, false);

If you follow his example on github, in implementation the author actually employs a loop that terminates after X seconds. See the do loop and comments at bottom of the script.

The solution is to give your loop a lifespan.

By defining a limit to the loop, IE X itterations or after X amount of time to trigger the end of the loop or die();, you will make it so that if the client disconnects there is a limit to how long a script will remain active. If the client is still there after X amount of time, they will simply re connect.

Tropaeolin