What are Long-Polling, Websockets, Server-Sent Events (SSE) and Comet?

Question

I have tried reading some articles, but I am not very clear on the concepts yet.

Would someone like to take a shot at explaining to me what these technologies are:

1. Long Polling
2. Server-Sent Events
3. Websockets
4. Comet

One thing that I came across every time was, the server keeps a connection open and pushes data to the client. How is the connection kept open, and how does the client get the pushed data? (How does the client use the data, maybe some code might help?)

Now, which one of them should I use for a real-time app. I have been hearing a lot about websockets (with socket.io [a node.js library]) but why not PHP?

Tactless

Realtime websocket or webrtc? There is a library for websocket in php, you do need to write extra code in order for it to work using ZMQ or just socket programming, nodeJs is built for this so its easily available. The reason websocket is not readily available in php is that you have to run an extra terminal and kept it running so that websocket server is readily available, you will have two servers bottom line. and the structure, php is not an event structure like javascript so there's that, websocket uses a event structure in order to catch and send messages. – Limnetic

Additionally: Comet and ServerSent Events are PHP's workaround of achieving almost realtime(not really) without creating 2 servers. – Limnetic

Answer 1

In the examples below the client is the browser and the server is the webserver hosting the website.

Before you can understand these technologies, you have to understand classic HTTP web traffic first.

Regular HTTP:

1. A client requests a webpage from a server.
2. The server calculates the response
3. The server sends the response to the client.

Ajax Polling:

1. A client requests a webpage from a server using regular HTTP (see HTTP above).
2. The client receives the requested webpage and executes the JavaScript on the page which requests a file from the server at regular intervals (e.g. 0.5 seconds).
3. The server calculates each response and sends it back, just like normal HTTP traffic.

Ajax Long-Polling:

1. A client requests a webpage from a server using regular HTTP (see HTTP above).
2. The client receives the requested webpage and executes the JavaScript on the page which requests a file from the server.
3. The server does not immediately respond with the requested information but waits until there's new information available.
4. When there's new information available, the server responds with the new information.
5. The client receives the new information and immediately sends another request to the server, re-starting the process.

HTML5 Server Sent Events (SSE) / EventSource:

1. A client requests a webpage from a server using regular HTTP (see HTTP above).
2. The client receives the requested webpage and executes the JavaScript on the page which opens a connection to the server.

3. The server sends an event to the client when there's new information available.

   • Real-time traffic from server to client, mostly that's what you'll need
   • You'll want to use a server that has an event loop
   • Connections with servers from other domains are only possible with correct CORS settings
   • If you want to read more, I found these very useful: (article), (article), (article), (tutorial).

HTML5 Websockets:

1. A client requests a webpage from a server using regular http (see HTTP above).
2. The client receives the requested webpage and executes the JavaScript on the page which opens a connection with the server.

3. The server and the client can now send each other messages when new data (on either side) is available.

   • Real-time traffic from the server to the client and from the client to the server
   • You'll want to use a server that has an event loop
   • With WebSockets it is possible to connect with a server from another domain.
   • It is also possible to use a third party hosted websocket server, for example Pusher or others. This way you'll only have to implement the client side, which is very easy!
   • If you want to read more, I found these very useful: (article), (article) (tutorial).

Comet:

Comet is a collection of techniques prior to HTML5 which use streaming and long-polling to achieve real time applications. Read more on wikipedia or this article.

---

Now, which one of them should I use for a realtime app (that I need to code). I have been hearing a lot about websockets (with socket.io [a node.js library]) but why not PHP ?

You can use PHP with WebSockets, check out Ratchet.

Janeanjaneczka

This is awesome! I am reading up on SSE and found this article, it's very nice - like I've now compared stuff, can you also include SSE here so we can also cross-check it's difference with Websocket? – Undersize

@Janeanjaneczka Oh was that it? I thought SSE meant Server-Sent Events. Anyway, thanks, I see it now. – Undersize

Q: in php let's say you were using websocket would every client connected to my server using ws: would have one thread allocated to him/her and its size would be ~2mb as is the case with normal requests? how would that differ in nodejs? How many concurrent clients can nodejs handle and when it breaks what happens? – Teocalli

@Janeanjaneczka what's the difference between SSE and long pulling? From your words, they are almost the same. – Extender

You can accomplish the same with both solutions but the mechanism is different. Long-polling uses 'regular' http data, SSE uses a different underlying protocol and needs a different server setup compared to long-polling. – Janeanjaneczka

Why do you say to not use apache ? – Igorot

Well you could use apache if you want. But a lot of people use Node.js because it has an event loop. But for Apache, see #12203943 – Janeanjaneczka

Can you suggest me which Comet Protocol will be use for Spring MVC. – Pushcart

Gerat answer. But why would I want to use websockets with a server that has an event loop? – Gosport

It would be nice if this answer was more explicit about the underlying technology. Which TCP connections are used? Are second connections established over the first one used to make the original request? How is HTTP pipelining involved? – Ossian

I agree, do you have this knowledge or do you know a user who does? – Janeanjaneczka

Just to add to what @Janeanjaneczka has already answered, the limitations of both SSE and WS can be handled as follows: WS reconnect issue can be solved by using an appropriate client library, for example awesome-websocket. SSE cross-origin server issue can be resolved by sending CORS headers in the response – Branca

I couldn't understand differences of comet and ajax long polling. In both techniques, we are (as client) sending an ajax request and waiting with an open HTTP request for an answer from server. @Janeanjaneczka – Perissodactyl

@Eray, did you read the last paragraph and followed the url's to more info about comet? – Janeanjaneczka

I guess Long Polling (and probably comet too) will be limited by network infrastructure that enforce short timeout on HTTP. – Hohenlohe

@Janeanjaneczka I was lost with your explanation at the point where you were explaining about Ajax Polling and you said "A client requests a webpage from a server using regular HTTP (see HTTP above) and then The requested webpage executes JavaScript..." How does a regular web server (e.g. apache) " executes JavaScript " ? – Unearthly

I think it should be: "the client executes the JavaScript on the page" so this is the browser, not the sever. Clear? – Janeanjaneczka

@Janeanjaneczka I know 2013 is a long time ago, but I'd like to point out SSE does not use a different protocol. I'ts just a variation of HTTP chunked encoding, and browsers accomodate it by setting TCP keepalive on the socket. It works with HTTP/2 as well, in contrast to websockets. – Zenda

Answer 2

Tieme put a lot of effort into his excellent answer, but I think the core of the OP's question is how these technologies relate to PHP rather than how each technology works.

PHP is the most used language in web development besides the obvious client side HTML, CSS, and Javascript. Yet PHP has 2 major issues when it comes to real-time applications:

1. PHP started as a very basic CGI. PHP has progressed very far since its early stage, but it happened in small steps. PHP already had many millions of users by the time it became the embed-able and flexible C library that it is today, most of whom were dependent on its earlier model of execution, so it hasn't yet made a solid attempt to escape the CGI model internally. Even the command line interface invokes the PHP library (libphp5.so on Linux, php5ts.dll on Windows, etc) as if it still a CGI processing a GET/POST request. It still executes code as if it just has to build a "page" and then end its life cycle. As a result, it has very little support for multi-thread or event-driven programming (within PHP userspace), making it currently unpractical for real-time, multi-user applications.

Note that PHP does have extensions to provide event loops (such as libevent) and threads (such as pthreads) in PHP userspace, but very, very, few of the applications use these.

2. PHP still has significant issues with garbage collection. Although these issues have been consistently improving (likely its greatest step to end the life cycle as described above), even the best attempts at creating long-running PHP applications require being restarted on a regular basis. This also makes it unpractical for real-time applications.

PHP 7 will be a great step to fix these issues as well, and seems very promising as a platform for real-time applications.

Ignacioignacius

One small correction: PHP was always written in C, as can be seen here: museum.php.net/php1 Also, "lesser used (but immensely more popular)" is rather self-contradictory; maybe what you mean is "more fashionable"? – Timid

@Timid - Thanks for the correction, I've been using PHP for over a decade and have always been under the impression that it's roots were in Perl. The PHP history page clearly supports that it was originally C as well. I'll edit my answer once I find a moment. – Ignacioignacius

I'll remove the bit about Perl since it doesn't mix well with official documentation, but this is still a confusing area in PHP's early development. – Ignacioignacius

PHP 7 seems very promising as a platform for real-time applications? What improvement/change in PHP7 for real-time applications? – Noseband

@I'll-Be-Back - fixed memory management / garbage collection, JIT compilation, etc. – Ignacioignacius

though also svn.php.net/viewvc/phpdoc/en/trunk/appendices/… && web.archive.org/web/20090426061624/http://us3.php.net/… – Trotskyism

Answer 3

Polling

Basically, polling is a technique of requesting the information from the server after regular intervals. This connection happens by following HTTP protocol. There are two types of polling:

1. Short Polling
2. Long Polling

Short Polling

In short polling, the client requests information from the server. The server processes the request. If data is available for the request, server responds to the request with the required information. However, if the server has no data available for the client, server returns an empty response. In both the situation, the connection will be closed after returning the response. Clients keep issuing new requests even after server sends the empty responses. This mechanism increases the network cost on the server.

Long polling

In long polling, the clients can request information from the server with the expectation that the server may not respond immediately. When the server receives the request, if it has no fresh data for the client, rather than returning an empty response, the server keeps the request open and waits for data to arrive. When the server receives new data, it delivers the response to the client right away, completing the open request. The client can then send another request for new updates after getting the answer from the server. Long polling reduces costs by reducing the number of empty responses.

WebSocket

WebSocket is a protocol that provides two-way(bi-directional) communication channels over a single TCP connection. Websocket facilitates a persistent connection between a client and a server, allowing both parties to begin transferring data at any moment. The WebSocket handshake is the procedure through which the client creates a WebSocket connection. If the operation is successful, the server and client can send and receive data at any time. Mostly used in real-time web applications such as WhatsApp, Uber.

Server-sent event (SSE)

Unlike WebSockets, we can not issue requests from a client to a server using SSE since it's a one-way connection. When we require "near real-time" transmission from the server to the client, or if the server generates data in a loop, SSE is the ideal choice.

Comet

Comet is a web application design paradigm that describes a continuous, two-way interaction between a server and a web browser using native HTTP methods. Comet is an umbrella term. Ajax Push, HTTP Streaming, and HTTP Server Push are some of the HTTP mechanisms that may be used to provide this event-driven interaction.

Venenose

Answer 4

You can easily use Node.JS in your web app only for real-time communication. Node.JS is really powerful when it's about WebSockets. Therefore "PHP Notifications via Node.js" would be a great concept.

See this example: Creating a Real-Time Chat App with PHP and Node.js

Debark