XmlHttpRequest.responseText while loading (readyState==3) in Chrome

Question

I am trying to "streaming" (from server to client) in Javascript by ajax (by XmlHttpRequest (=xhr). I am using modified handleResponse function described in Cross-browser implementation of "HTTP Streaming" (push) AJAX pattern

function handleResponse() {
if (http.readyState != 4 && http.readyState != 3)
    return;
if (http.readyState == 3 && http.status != 200)
    return;
if (http.readyState == 4 && http.status != 200) {
    clearInterval(pollTimer);
    inProgress = false;
}
// In konqueror http.responseText is sometimes null here...
if (http.responseText === null)
    return;

while (prevDataLength != http.responseText.length) {
    if (http.readyState == 4  && prevDataLength == http.responseText.length)
        break;
    prevDataLength = http.responseText.length;
    var response = http.responseText.substring(nextLine);
    var lines = response.split('\n');
    nextLine = nextLine + response.lastIndexOf('\n') + 1;
    if (response[response.length-1] != '\n')
        lines.pop();

    for (var i = 0; i < lines.length; i++) {
        // ...
    }
}

if (http.readyState == 4 && prevDataLength == http.responseText.length)
    clearInterval(pollTimer);

inProgress = false;
}

With php script, which flushes me data (without ajax it really flushes data to browser while progressing)

I have no problem in Firefox, but Google Chrome and IE give me an empty responseText while xhr.readyState equals to 3. I found that problem described in the Internet, but it didn't give me any solution.

Do you know, how to pass by this implementation problem in Chrome? (w3c says, that responseText can't be NULL in readyState==3 - Chrome implemented this rule, but gives only empty string)

And if you don't know, do you know any working solution in some products? (opensource frameworks, librararies etc.)

Thanks a lot for your ideas.

Edit: The workaround is in creating iframe, call the script to iframe and flush data here and grab data by javascript from iframe. But this is not ajax solution. I really would like to see pure ajax solution.

Conoid

Are you sure you're setting a "Content-Type" of "text/plain" or "application/x-javascript" when you start your response from the server? Apparently the Webkit browsers insist upon that, or they might. – Bindweed

I set content-type to "application/x-www-form-urlencoded". I tried - as you said - application/x-javascript, but this is not working at all. – Conoid

Answer 1

Chrome has a bug where it will only populate xhr.responseText after a certain number of bytes has been received. There are 2 ways to get around this,

Set the content type of the return to "application/octet-stream"

or

Send a prelude of about 2kb to prep the handler.

Either of these methods should make chrome populate the responseText field when readyState == 3.

IE7/8 on the other hand can't do it, you need to resort to long polling or use the cross domain trick with XDomainRequest in IE8, a la MS

Philanthropy

Set Content-type not working, 2k is too big, any other options ? – Jenninejennings

content type of "application/octet-stream" or "application/x-javascript" is not working for Chrome, 2k works. – Advance

I know this is a 10 year old answer (almost to the day), but this is still accurate in Chrome 87.0.4280.141. If you send a little bit of dummy payload, Chrome's onreadystatechange will fire with HEADERS_RECEIVED (2) and LOADING (3). Thank you so much! I was searching like crazy. – Svelte

Answer 2

To expand on Andrew's answer, this is the cross-browser solution I came up with.

Works correctly in 99% of browsers, namely IE ≥ 8, Chrome, Firefox, and Safari, sending incremental events as soon as data is received by the browser (but see the notes below.)

if (/MSIE [8-9]/.test(navigator.appVersion)) {
    var get = new XDomainRequest()
    get.onprogress = handleData
    get.onload = handleData
} else {
    var get = new XMLHttpRequest()
    get.onreadystatechange = handleData
}
get.open('get', '/example/url')
get.send()

function handleData() {
    if (get.readyState != null && (get.readyState < 3 || get.status != 200)) {
        return
    }
    // process incremental data found in get.responseText
}

IE 8–9 will start populating responseText after 2kB of data, so if that's not ok, you should send an initial padding of 2kB.

Chrome needs either that, or Content-Type: application/octet-stream.

Agan

Answer 3

Have you considered using WebSockets or server-sent events?

Most major browsers now support the WebSocket protocol, though if your site needs to work in IE 9 or older, or in Android Browser 4.3 or older, you would have to keep the code that uses XMLHttpRequest as a fallback.

Most of these browsers also support a feature called server-sent events, which unlike WebSockets, can be implemented on the server using a traditional HTTP daemon and CGI/PHP script, though only provides one-way communication.

See also: WebSockets vs. Server-Sent events/EventSource

Incubate

Sweet future :) But there is the last one browser... >IE9 Websockets will be nice, and for my purpose would be the most amazing Node.js :) – Conoid

It think that Mr. Moravec would have problems with his FF3 too, if he choose web sockets now. According to the Wikipedia article on the topic support is in FF4, which is in beta right now - so it will be a while before everybody has it. I have not come around to test it myself yet, I'm a bit lazy right now and they are not distributing debs yet - no just click & install w/ other words. – Pyrogen

Answer 4

Well, unfortunately every part of XmlHttpRequest (or any web standards) isn't fully implemented in all browsers. But you have several other options for HTTP Streaming:
Wikipedia: Push technology
Wikipedia: Comet (programming) Wikipedia: Web Sockets (experimental, low browser support)

I saw in your comment that you would like it to be pure AJAX, but I like to suggest possible alternate ways to solutions. You could use a JavaApplet where possible or a Flash Object. For the latter you won't need a flashy and expensive IDE, you can use Haxe to create Flash/SWF files and you will feel pretty comfortable with it as you know JavaScript.

Here is a Flash/Neko chat example that probably can be adopted to other platforms and usages as well.

I wish you best of good luck.

Pyrogen

Answer 5

try using the responseStream/responseBody property when in IE. I was thinking of doing a similar thing once and ran into the same problem. Unfortunately neither are w3c specs

http://www.w3.org/TR/XMLHttpRequest/#the-responsetext-attribute

Guinevere

I'd try it, but IE end up with error (no such method or result is null) – Conoid

Answer 6

As i understand it, making partial text available on readyState 3 is a non-standard firefox only behaviour that is simply not possible to directly emulate in other browsers, what you might want to do instead is make multiple sequential requests for small chunks of the data rather than one 'streaming' request

Retainer

Answer 7

This worked for me for Chrome, but not IE:

[test.php]:

<?php
Header('Content-type: text/plain');
while (1) {
    echo str_pad('test: '.mt_rand(1000,9999), 2048, ' ');
    flush();
    sleep(1);
}

[test.html]:

<!DOCTYPE html>
<html lang="en">
    <head>
        <title>Stream test</title>
        <meta charset="UTF-8" />
        <script type="text/javascript">
            function xmlHttpRequest() {
                return (function (x,y,i) {
                    if (x) return new x();
                    for (i=0; i<y.length; y++) try { 
                    return new ActiveXObject(y[i]);
                    } catch (e) {}
                })(
                    window.XMLHttpRequest, 
                    ['Msxml2.XMLHTTP','Microsoft.XMLHTTP']
                );
            };
            function stream(url) {
                // Declare the variables we'll be using
                var xmlHttp = xmlHttpRequest();
                xmlHttp.open("GET", url, true);
                var len = 0;
                xmlHttp.onreadystatechange = function() {
                if (xmlHttp.status == 200 && xmlHttp.readyState >=3) {
                    var text = xmlHttp.responseText;
                    text = text.substr(len, text.length-len);
                    len = xmlHttp.responseText.length;
                    console.log(text);
                }
                }
                xmlHttp.send(null);
            }           
            stream('/test.php');
        </script>
    </head>
    <body>
    </body>
</html>

YMMV.

Parol

Answer 8

As Jaroslav Moravec said, if you set the content-type in the header of the stream to application/x-javascript it works in Safari, Chrome and Firefox.

I have not tested IE.

Unchristian

Answer 9

Setting the content type of the return to "application/octet-stream" as suggested by Andrew was a great solution. Plus you should use XDomainRequest on IE.

To read the data as they come, you should just use an infinite loop (that stops when readystate = 4 or XDomainRequest.onLoad has been called) with a timeout.

Here's how I would do it:

var i = 0;
var returnValue = function() {
    if (!finished) {
        setTimeout(returnValue, 100);
    }
    var resp = ajax.responseText;
    var newI = resp.lastIndexOf(";") + 1;
    if (newI > i) {
        var lines = resp.substring(i, newI).split(";");
        for (var x = 0; x < lines.length; x++) {
            eval(lines[x]);
        }
        i = newI;
    }
}

Note: some says using eval is risky, I claim that's not where any risk really come from.

Hesione

Answer 10

The URLyou are submitting the request to - is it part of your domain?

It coulld be because of the same origin policy.

See this question and possible ways to circumvent it (and this article).

Cowbind

well, if that was the case, won't he be facing a problem w/ FF too? – Upstairs

@anirvan, yes. You are right. But with the code I don't see any other possibilities (with my knowledge). – Cowbind

Answer 11

once i had this problem using safari (never tested with chrome, maybe there was the same problem (chrome/safari both use the same rendering-engine (as far as i know) - don't know about the js-parts)). i never found a solution to work around that, but because of it was a small app in a company-wide intranet it wasn't a big problem to not support safari (ff was the default-browser anyway, and ff works).

Olvan

It's so sad for me... I implemented "graceful degradation", but I really want to know some workaround etc. I guess using flash as a server/client transport layer... The clean solution without flash would be amazing... – Conoid

Apple Safari and Google Chrome only uses the same rendering engine, Webkit, But I think that is so far the similarities go. Google Chrome uses its own brand new JavaScript-engine named V8. – Pyrogen