Best approach for (cross-platform) real-time data streaming in PHP?

Question

I've been wondering how to do "true" (semi) real-time data streaming with PHP.

Possible applications: chat rooms, auctions, games, etc.

By "true", I mean the data is not just written somewhere and polled continuously, but actually streaming to the client somehow.

By "semi", I mean it's okay if only the stream from the server to the client is real-time, and messages from the client to the server are not.

For the communication between the client and server, I'd like to stick with plain HTTP (AJAX) rather than some other protocol.

Streaming to the client with HTTP is possible by manually flushing the output buffer.

The question is what to connect that script to on the server-side?

And once it's connected, to do a blocking read, rather than polling for changes.

The shared memory (shmop) extension would work, but it's not cross-platform.

Perhaps memcached would work? But I'm not sure if there's a way to do a blocking read, so it comes down to polling again - although I'm sure memcached is pretty fast, I just don't like the idea of continuous polling.

Any ideas?

Clearheaded

I don't mean push - I'm looking for some way to open a stream/pipe/port/channel/something on the server-side and write to it, and any other script currently connected to that stream/thing can read from it in blocking mode, and then write results to a continuously running HTTP request and flush. – Clearheaded

Answer 1

PHP is not well suited for implementing realtime data streaming. PHP is very slow, and is not designed to build multi-threaded applications. You'd be better off implementing a full blown socket server in a language like Python or Java.

That said, I would highly recommend checking out NodeJS: http://nodejs.org/

It uses an asynchronous event based model for I/O, instead of having threads block in an event loop. NodeJS servers are written in Javascript. NodeJS is fast, scales, and has a low learning curve.

Clients would connect to a NodeJS HTTP server using long polling Ajax requests. PHP could connect to NodeJS directly and push notifications. Or PHP could write to a message queue, or database, memcache etc, and NodeJS would poll those data stores for updates, and send new messages to clients.

You would possibly need to write your own daemon to serve as a go between from NodeJS to MySQL, memcached, etc. when polling for updates. NodeJS would keep a socket open with a daemon process. The daemon process would poll data stores for updates, and send the updates to NodeJS. A NodeJS HTTP server would then send those updates to clients.

See this tutorial for implementing a realtime Twitter stream: http://net.tutsplus.com/tutorials/javascript-ajax/learning-serverside-javascript-with-node-js/

Copalite

this is exactly what node.js is for – Canicular

You're wrong about PHP being slow. But I have to agree, it was probably not designed for parallel/multi-threaded applications of this type. Answer accepted. – Clearheaded

nodejs is not single threaded rickgaribay.net/archive/2012/01/28/… – Clearheaded

+1 on the "wrong about PHP being slow". It's not. It does have limitations around threading and it's definitely not tailored to persistent connections, but it's not slow. – Erective

Look at the date of the post. PHP is certainly slower than Java, Python, and JS in V8. And was even more so back in 2011. – Copalite

The answer is dangerously outdated. PHP 5->7 came with a massive boost of performance and 7->8 came with another increase plus JIT compiler (which needs proper typing usage). PHP 8 is 3-4 times faster than Python typically, when optimized for performance likely quite a bit more than that. In many tasks JAVA is noticeable faster than PHP today, though the new JIT compiler might close the gap. In terms of memory PHP is much more efficient than JAVA. PHP can be used as high performance high level language, including multi-threading and non blocking IO. – Lowell

Answer 2

If you're using HTML and Javascript, then you want WebSockets. If it's Flash or anything else, then normal TCP sockets.

The idea for either is that you run a server file (written in PHP), that waits for connections. Once it's connected to one or more clients, data can be pushed both ways. There are a few PHP WebSocket projects out there. Check out this:

http://code.google.com/p/phpwebsocket

There's also a framework called Skeleton that I've been contributing to that has a WebSocket server library built in. Still in the unstable stages, though.

http://code.google.com/p/skeleton

Unfortunately, WebSockets are still a new technology, so they're not supported universally. As @Christian mentioned, you may want to use the Socket.IO library.

Motto

I recently came across Ratchet - a very nice, clean, simple implementation of a web-sockets server in PHP. – Clearheaded

I take that back - Ratchet is a nice idea, but PHP as such is unsuitable for real-time applications as a socket server - this article explains why. – Clearheaded

Answer 3

If you want to communicate between PHP and another language (for instance a C++ application) you might want to look into Apache Thrift (http://thrift.apache.org/). Apache Thrift is widely used at Facebook for "scalable cross-language services development".

Edit: I would probably use Apache Thrift to communicate with a Twisted application listening on port 80 and have the browsers connect to the Twisted application using either long polling or a websocket. You might want to also look into Socket.IO, it's a cross-browser web socket implementation which is made for realtime applications.

Basically you would have your application push to your Twisted web server using Thrift and you would then pass the message to any open connections.

• Christian

Blasted

I'm not convinced it's worth the effort - PHP simply was not designed for this sort of task. It was designed with the HTTP life cycle in mind: a stateless request/response model. NodeJS is a much safer bet for real-time/socket applications, and the language barrier is not so bad now, with all the great languages that compile to JavaScript and simply use Node/V8 as a VM. – Clearheaded

Answer 4

<?php
//B"H

    header('X-Accel-Buffering: no');
    header('Cache-Control: no-cache');

    set_time_limit(0);              // making maximum execution time unlimited
    ob_implicit_flush(1);           // Send content immediately to the browser on every statement which produces output
    ob_end_flush();                 // deletes the topmost output buffer and outputs all of its contents
    $waitTime = 1;
    sleep($waitTime);
    for($i = 0; $i < 8; $i++) {
        echo json_encode(['data' => 'hi '.$i]);
        echo "<br>";
        sleep($waitTime);
    }

    die(1);
?>

You definitely do not need web-sockets. You can stream data using the X-Accel-Buffering: no header (if using nginx) and turning on "implicit flush" to send data immediately. That's all there is to it.

Wasteful