Cross-browser implementation of "HTTP Streaming" (push) AJAX pattern

Question

Client request web page from server. Clent then requests for extra calculations to be done; server performs series of calculations and sends partial results as soon as they are available (text format, each line contains separate full item). Client updates web page (with JavaScript and DOM) using information provided by server.

This seems to fit HTTP Streaming (current version) pattern from Ajaxpatterns site.

The question is how to do it in cross-browser (browser agnostic) way, preferably without using JavaScript frameworks, or using some lightweight framework like jQuery.

The problem begins with generating XMLHttpRequest in cross-browser fashion, but I think the main item is that not all browsers implement correctly onreadystatechangefrom XMLHttpRequest; not all browsers call onreadystatechange event on each server flush (BTW. how to force server flush from within CGI script (in Perl)?). Example code on Ajaxpatterns deals with this by using timer; should I drop timer solution if I detect partial response from onreadystatechange?

---

Added 11-08-2009

Current solution:
I use the following function to create XMLHttpRequest object:

function createRequestObject() {
        var ro;
        if (window.XMLHttpRequest) {
                ro = new XMLHttpRequest();
        } else {
                ro = new ActiveXObject("Microsoft.XMLHTTP");
        }
        if (!ro)
                debug("Couldn't start XMLHttpRequest object");
        return ro;
}

If I were to use some (preferably light-weight) JavaScript framework like jQuery, I'd like to have fallback if user chooses not to install jQuery.

I use the following code to start AJAX; setInterval is used because some browsers call onreadystatechange only after server closes connection (which can take as long as tens of seconds), and not as soon as server flushes data (around every second or more often).

function startProcess(dataUrl) {
        http = createRequestObject();
        http.open('get', dataUrl);
        http.onreadystatechange = handleResponse;
        http.send(null);

        pollTimer = setInterval(handleResponse, 1000);
}

The handleResponse function is most complicated one, but the sketch of it looks like the following. Can it be done better? How it would be done using some lightweight JavaScript framework (like jQuery)?

function handleResponse() {
    if (http.readyState != 4 && http.readyState != 3)
        return;
    if (http.readyState == 3 && http.status != 200)
        return;
    if (http.readyState == 4 && http.status != 200) {
        clearInterval(pollTimer);
        inProgress = false;
    }
    // In konqueror http.responseText is sometimes null here...
    if (http.responseText === null)
        return;

    while (prevDataLength != http.responseText.length) {
        if (http.readyState == 4  && prevDataLength == http.responseText.length)
            break;
        prevDataLength = http.responseText.length;
        var response = http.responseText.substring(nextLine);
        var lines = response.split('\n');
        nextLine = nextLine + response.lastIndexOf('\n') + 1;
        if (response[response.length-1] != '\n')
            lines.pop();

        for (var i = 0; i < lines.length; i++) {
            // ...
        }
    }

    if (http.readyState == 4 && prevDataLength == http.responseText.length)
        clearInterval(pollTimer);

    inProgress = false;
}

Answer 1

The solution you linked to is not AJAX at all, actually. They call it HTTP Streaming but it's essentially just long polling.

In the example they link to, you can see for yourself quite easily with firebug. Turn on the Net panel - there are no XHR entries, but it takes just a hair over 10 seconds to load the original page. That's because they're using PHP behind the scenes to delay the output of the HTML. This is the essence of long polling - the HTTP connection stays open, and the periodic HTML sent back is javascript commands.

You can opt to do the polling completely on the client side, though, with setTimeout() or setInterval()

A jQuery example

<script type="text/javascript">
  $(document).ready(function()
  {
    var ajaxInterval = setInterval( function()
    {
      $.getJSON(
        'some/servie/url.ext'
        , { sample: "data" }
        , function( response )
          {
            $('#output').append( response.whatever );          
          }
      );
    }, 10000 );  
  });
</script>

Answer 2

I would take a look at orbited

They use several comet transport implementation that they choose based on configuration and browser sniffing.

See http://orbited.org/svn/orbited/trunk/daemon/orbited/static/Orbited.js

and look for "Orbited.CometTransports"

Some of the different transports must be matched by the backend implementation, so have a look at the server side for orbited also.