Using comet with PHP?

Question

I was thinking of implementing real time chat using a PHP backend, but I ran across this comment on a site discussing comet:

My understanding is that PHP is a terrible language for Comet, because Comet requires you to keep a persistent connection open to each browser client. Using mod_php this means tying up an Apache child full-time for each client which doesn’t scale at all. The people I know doing Comet stuff are mostly using Twisted Python which is designed to handle hundreds or thousands of simultaneous connections.

Is this true? Or is it something that can be configured around?

Dukie

you can run php as fastcgi... – Lymphocyte

Use nodeJS as a server to keep client connections, websockets in javascript to connect to server from browser. PHP in this sense can be a priviledged client that will connect to nodejs, push some service data that will be treated somehow on client side. – Highline

@ArtjomKurapov You can make PHP into a web server thus bypassing Apache's method of handling requests - think of it like a real PHP server taking care comet requests only. – Stonecrop

@Stonecrop if you meant built-in php webserver since 5.4 then its only for development and using it on production is a bad idea – Highline

@ArtjomKurapov No, I meant writing a real PHP server, by using PHP sockets to listen to port 80 and block on input indefinitely - effectively how servers work. This can be already seen in action in projects like phpwebsocket. – Stonecrop

Answer 1

Agreeing/expanding what has already been said, I don't think FastCGI will solve the problem.

Apache

Each request into Apache will use one worker thread until the request completes, which may be a long time for COMET requests.

This article on Ajaxian mentions using COMET on Apache, and that it is difficult. The problem isn't specific to PHP, and applies to any back-end CGI module you may want to use on Apache.

The suggested solution was to use the 'event' MPM module which changes the way requests are dispatched to worker threads.

This MPM tries to fix the 'keep alive problem' in HTTP. After a client completes the first request, the client can keep the connection open, and send further requests using the same socket. This can save signifigant overhead in creating TCP connections. However, Apache traditionally keeps an entire child process/thread waiting for data from the client, which brings its own disadvantages. To solve this problem, this MPM uses a dedicated thread to handle both the Listening sockets, and all sockets that are in a Keep Alive state.

Unfortunately, that doesn't work either, because it will only 'snooze' after a request is complete, waiting for a new request from the client.

PHP

Now, considering the other side of the problem, even if you resolve the issue with holding up one thread per comet request, you will still need one PHP thread per request - this is why FastCGI won't help.

You need something like Continuations which allow the comet requests to be resumed when the event they are triggered by is observed. AFAIK, this isn't something that's possible in PHP. I've only seen it in Java - see the Apache Tomcat server.

Edit:

There's an article here about using a load balancer (HAProxy) to allow you to run both an apache server and a comet-enabled server (e.g. jetty, tomcat for Java) on port 80 of the same server.

Urgent

I realise this doesn't exactly provide a solution :/ – Urgent

+1 because Apache/PHP are not good options for scaling out a comet solution. Options for PHP users are 1) as you mentioned, crazy configurations of additional servers and proxies or 2) using a SaaS solution and offloading the comet stuff via something like WebSync On-Demand. – Glossology

This is wrong in several aspects. If one wants to leave the one-thread-per-user method, it can be easily achieved by removing Apache as an intermediary and let PHP handle these requests. Of course, Apache works better at serving content, so I would run this apache-less PHP sever on a subdomain that does not serve any content. – Stonecrop

@MikeHouston What about trying comet with php in IIS.? – Diaphoretic

@ravz, there's some stuff here about IIS and comet: https://mcmap.net/q/243923/-comet-programming-in-iis, but I suspect the fast-cgi PHP module has the same limitations as with apache. It mentions a single threaded environment here: microsoft.com/web/platform/phponwindows.aspx - I don't use Windows servers myself, so I'm not sure about the threading model. – Urgent

The HAProxy advice is excellent. HAProxy advertises 30000 sessions per GB of memory, and the comet/longpoll can be redirected to a custom process/cometd-server. Using just apache/mod_php for comet, a 1GB server would scale to about 50 simultaneous connections considering apache's memory footprint. NGINX+php-fpm is no better, I tested. – Monocyte

Answer 2

You could use Nginx and JavaScript to implement a Comet based chat system that is very scalable with little memory or CPU utilization.

I have a very simple example here that can get you started. It covers compiling Nginx with the NHPM module and includes code for simple publisher/subscriber roles in jQuery, PHP, and Bash.

http://blog.jamieisaacs.com/2010/08/27/comet-with-nginx-and-jquery/

Undulation

Answer 3

PHP

I found this funny little screencasts explaining simple comet. As a side note I really think this is going to kill your server on any real load. When just having a couple of users, I would say to just go for this solution. This solution is really simple to implement(screencasts only takes 5 minutes of your time :)). But as I was telling previously I don't think it is good for a lot of concurrent users(Guess you should benchmark it ;)) because:

1. It uses file I/O which is much slower then just getting data from memory. Like for example the functions filemtime(),
2. Second, but I don't think least PHP does not a have a decent thread model. PHP was not designed for this anyway because of the share nothing model. Like the slides says "Shared data is pushed down to the data-store layer" like for example MySQL.

Alternatives

I really think you should try the alternatives if you want to do any comet/long polling. You could use many languages like for example:

• Java/JVM: Jetty continuations.
• Python: Dustin's slosh.
• Erlang: Popular language for comet/etc.
• Lua, Ruby, C, Perl just to name a few.

Just performing a simple google search, will show you a lot alternatives also PHP(which I think on any big load will kill your server).

Donets

Answer 4

mod_php is not the only way to use PHP. You can use fastcgi. PHP must be compiled with --enable-fastcgi.

PHP as FastCGI: http://www.fastcgi.com/drupal/node/5?q=node/10

Hartwell

Answer 5

You may also try https://github.com/reactphp/react

React is a low-level library for event-driven programming in PHP. At its core is an event loop, on top of which it provides low-level utilities, such as: Streams abstraction, async dns resolver, network client/server, http client/server, interaction with processes. Third-party libraries can use these components to create async network clients/servers and more.

The event loop is based on the reactor pattern (hence the name) and strongly inspired by libraries such as EventMachine (Ruby), Twisted (Python) and Node.js (V8).

The introductory example shows a simple HTTP server listening on port 1337:

<?php

$i = 0;

$app = function ($request, $response) use (&$i) {
    $i++;

    $text = "This is request number $i.\n";
    $headers = array('Content-Type' => 'text/plain');

    $response->writeHead(200, $headers);
    $response->end($text);
};

$loop = React\EventLoop\Factory::create();
$socket = new React\Socket\Server($loop);
$http = new React\Http\Server($socket);

$http->on('request', $app);

$socket->listen(1337);
$loop->run();

Dispatcher

Answer 6

I'm having a similar issue. One option I'm finding interesting is to use an existing Comet server, like cometd-java or cometd-python, as the core message hub. Your PHP code is then just a client to the Comet server -- it can post or read messages from channels, just like other clients.

There's an interesting code snippet linked here: http://morglog.org/?p=22=1 that implements part of this method (although there are bits of debug code spread around, too).

Waylin

Answer 7

You'll have a hard time implementing comet in PHP, just because of it's inherent single-threaded-ness.

Check out Websync On-Demand - the service lets you integrate PHP via server-side publishing, offloading the heavy concurrent connection stuff, and will let you create a real-time chat app in no time.

Glossology

Answer 8

I'm current implementing a scalable PHP Comet server using socket functions. It is called 'phet' ( [ph]p com[et] )

Project page: http://github.com/Tim-Smart/phet

Free free to join in on development. I have currently managed to get most of the server logic done, just need to finish off the client side stuff.

EDIT: Recently added 'Multi-threading' capabilities using the pcntl_fork method :)

Bestir

There are no examples readily available for how to use this library. – Amblygonite

Answer 9

A new module just came out for the nginx web server that'll allow Comet with any language, including PHP.

http://www.igvita.com/2009/10/21/nginx-comet-low-latency-server-push/

Ermey

Answer 10

You will have to create your own server in PHP. Using Apache/mod_php or even fastcgi will not scale at all. A few years old, but can get you started:

PHP-Comet-Server: http://sourceforge.net/projects/comet/

Shame

Answer 11

I think this is more an issue that having a lot of apache threads running all the time is a problem. That will existing with any language if it works via apache in the same way as PHP (usually) does.

Fasces

I think the point is that you usually run php in a process per request, rather than a thread per request. – Tannatannage