What technology does Google Drive use to get real-time updates?

Question

What technology does Google Drive use to do real-time?

When I type in a Google Drive document that is being accessed by multiple users, the Chrome Developer Tools Network tab shows that there are no WebSockets.

I see that the two most frequent types of AJAX call have either "bind?" or "save?" in the URL. "save?" POST requests are made every time I type, which makes sense- normal AJAX for sending updates to the server.

When another user types, the most recent "bind?" GET call stays open, and the amount of data transferred over that connection increases. Periodically, "bind?"s are closed and new ones open up, and the logic seems to be some function of duration and data size.

This isn't long-polling, since when the server sends updates it doesn't complete the response.

This doesn't seem to be server-sent events, since the content-type is "text/plain" instead of "text/stream".

Is there a name for what Google is doing? If so, how can I try implementing this?

Snowclad

Does same technology used in Google Calendar as well? – Igloo

Answer 1

Is there a name for Google's solution for real-time updates in Drive (such as "long polling" or "sockets")?

It didn't have a name until now. I'll call it "no-polling," to contrast with polling and long-polling.

With polling, the client periodically sends queries for new data.

With long-polling, the client queries for data, and the server holds onto the request, ending the response with the updates when there are updates.

No-polling (what Google Drive does) takes advantage of how the browser can read data from the body of a request before the request is complete. So as collaborators do more typing and edits, the server appends more data to the current request. If certain limits are met (length of the content or duration of the request), the request completes, and the client initiates a new request with the server.

How can I try implementing this?

For the client to send updates to the server: this can be done with normal POSTs.

For the client to subscribe to updates from the server:

• The client sends a GET for an update stream, then starts reading the body of the response before the response is complete.

  XHR objects can emit progress events before the request is complete. The (partial) response is accessible using xhr.responseText. ~~There's no simple way to watch for progress with fetch yet (as of May 2016).~~ When using fetch one can watch for progress by consuming the res.body ReadableStream.

• The client should initiate a new request when the current request ends.

The server has to:

• Keep track of which clients are subscribed to which update streams.
• When a request comes in for a particular update stream, write data to the response, but don't complete the response until the amount of data gets large or a timeout is met.

No-polling seems superior to long-polling, in my opinion, though I haven't played with it much. Long-polling forces a trade-off between latency and message size (given a constant rate of updates), a trade-off no-polling doesn't have to make. Another disadvantage of long-polling is that it can lead to many HTTP requests, paying the overhead of HTTP each time.

No-polling's big advantage over WebSockets is that no-polling is supported by every browser, though WebSocket support is pretty good — IE10+.

Snowclad

I think Facebook also does something similar. Try engaging in a conversation with some: it maintains a persistent HTTP connection as it grabs more data over time. – Dingus

@enlitement nice! I see that, too. The GETs to /pull? last up to around 50 seconds and have a series of JSON payloads and heartbeat messages. – Snowclad

Current fetch spec says that res.body is a ReadableStream, enabling you to watch for progress with fetch. – Toxoplasmosis

This is very interesting... I had never heard of this method before. – Puri

Is there any framework which already implement this? or any gist which implements this? @MaxHeiber – Anticosti

Not that I know of @MayankSingh, though I'd be curious to see one! – Snowclad

As far as I can see, it looks like they switched to "standard" long polling strategy, when the request is closed everytime data is sent. – Eltonelucidate

Answer 2

no-polling described here sounds like "server sent events - SSE" where client sends a http get request with "text/event-stream" as content type. Server holds on to the request and sends a response to the client whenever it needs to. It is persistent but differs from web-socket as web-socket it 2 way communication but SSE is one way(only server sends payload/message to client)

Balzer

A client would normally send a request, and SSE keeps client updated in real-time – Splint