Cross-browser implementation of "HTTP Streaming" (push) AJAX pattern

Question

Client request web page from server. Clent then requests for extra calculations to be done; server performs series of calculations and sends partial results as soon as they are available (text format, each line contains separate full item). Client updates web page (with JavaScript and DOM) using information provided by server.

This seems to fit HTTP Streaming (current version) pattern from Ajaxpatterns site.

The question is how to do it in cross-browser (browser agnostic) way, preferably without using JavaScript frameworks, or using some lightweight framework like jQuery.

The problem begins with generating XMLHttpRequest in cross-browser fashion, but I think the main item is that not all browsers implement correctly onreadystatechangefrom XMLHttpRequest; not all browsers call onreadystatechange event on each server flush (BTW. how to force server flush from within CGI script (in Perl)?). Example code on Ajaxpatterns deals with this by using timer; should I drop timer solution if I detect partial response from onreadystatechange?

---

Added 11-08-2009

Current solution:
I use the following function to create XMLHttpRequest object:

function createRequestObject() {
        var ro;
        if (window.XMLHttpRequest) {
                ro = new XMLHttpRequest();
        } else {
                ro = new ActiveXObject("Microsoft.XMLHTTP");
        }
        if (!ro)
                debug("Couldn't start XMLHttpRequest object");
        return ro;
}

If I were to use some (preferably light-weight) JavaScript framework like jQuery, I'd like to have fallback if user chooses not to install jQuery.

I use the following code to start AJAX; setInterval is used because some browsers call onreadystatechange only after server closes connection (which can take as long as tens of seconds), and not as soon as server flushes data (around every second or more often).

function startProcess(dataUrl) {
        http = createRequestObject();
        http.open('get', dataUrl);
        http.onreadystatechange = handleResponse;
        http.send(null);

        pollTimer = setInterval(handleResponse, 1000);
}

The handleResponse function is most complicated one, but the sketch of it looks like the following. Can it be done better? How it would be done using some lightweight JavaScript framework (like jQuery)?

function handleResponse() {
    if (http.readyState != 4 && http.readyState != 3)
        return;
    if (http.readyState == 3 && http.status != 200)
        return;
    if (http.readyState == 4 && http.status != 200) {
        clearInterval(pollTimer);
        inProgress = false;
    }
    // In konqueror http.responseText is sometimes null here...
    if (http.responseText === null)
        return;

    while (prevDataLength != http.responseText.length) {
        if (http.readyState == 4  && prevDataLength == http.responseText.length)
            break;
        prevDataLength = http.responseText.length;
        var response = http.responseText.substring(nextLine);
        var lines = response.split('\n');
        nextLine = nextLine + response.lastIndexOf('\n') + 1;
        if (response[response.length-1] != '\n')
            lines.pop();

        for (var i = 0; i < lines.length; i++) {
            // ...
        }
    }

    if (http.readyState == 4 && prevDataLength == http.responseText.length)
        clearInterval(pollTimer);

    inProgress = false;
}

Mencher

You should definitely add that code example as a reply and mark it as the correct one! – Epidermis

"if user chooses not to install jQuery" ? – Marivelmariya

Hi, just came across your solution, but I'm afraid that it still won't work with IE, since when you'll try to get the responseText while the requests still wasn't ended, then you'll get the following message: "The data necessary to complete this operation is not yet available". – Shortwinded

Answer 1

The solution you linked to is not AJAX at all, actually. They call it HTTP Streaming but it's essentially just long polling.

In the example they link to, you can see for yourself quite easily with firebug. Turn on the Net panel - there are no XHR entries, but it takes just a hair over 10 seconds to load the original page. That's because they're using PHP behind the scenes to delay the output of the HTML. This is the essence of long polling - the HTTP connection stays open, and the periodic HTML sent back is javascript commands.

You can opt to do the polling completely on the client side, though, with setTimeout() or setInterval()

A jQuery example

<script type="text/javascript">
  $(document).ready(function()
  {
    var ajaxInterval = setInterval( function()
    {
      $.getJSON(
        'some/servie/url.ext'
        , { sample: "data" }
        , function( response )
          {
            $('#output').append( response.whatever );          
          }
      );
    }, 10000 );  
  });
</script>

Aiguille

Not exaxtly what I want. The calculation on server generates output in plain text format. With XHR I can get this response directly in client (onreadystatechange on flush / timer) and edit web page according to partial data I get. – Wallow

What isn't what you want? The long polling? I'm not recommending either method - I'm just telling you what your options are. – Aiguille

In case you want to use long pooling (Comet), you should consider using Meteor server software, because Apache is not designed to to that sort of things. And there is also javascript library that handles almost everything for you, I just can't remember it's name, will post it later. – Catercornered

Answer 2

I would take a look at orbited

They use several comet transport implementation that they choose based on configuration and browser sniffing.

See http://orbited.org/svn/orbited/trunk/daemon/orbited/static/Orbited.js

and look for "Orbited.CometTransports"

Some of the different transports must be matched by the backend implementation, so have a look at the server side for orbited also.

Mojgan