Looking for recipe: Multiple async Ajax requests in JavaScript

Question

I am about to spend a lot of time solving a problem, and I wonder if there is an existing recipe for this. It's a browser-based application (JavaScript and Dojo Toolkit) that is a client of a RESTful Web Service.

It uses Comet to automatically update the display. There is a callback function that processes every message received. [Boring details of Comet: as a sort of background operation, an HTTP request is made to the server. This request blocks on the server until it has a message for us. When the client finally receives the response, it calls the callback function and then makes the next HTTP request. HTTP allows up to two simultaneous requests, so this "background" request does not block the "foreground" requests that happen when the user does things.]

There is an adapter layer below the UI layer. The UI layer thinks it's getting messages pushed to it. The adapter layer is doing the Comet requests and transforming the response from what the server sends to what the UI layer expects.

var ourEventFilter = dojo.hitch(this, function(evt) {
  if (evt["obj"]) {
        evt.obj = this.transform(evt.obj);
  }
  callUIEventHandler(evt);
}

[dojo.hitch() is syntactic sugar to make a closure, binding a function's this.]

The obj may look something like this:

{
   "resources": [
      {"name":"Me", "type":"vm", "link":"http://server/item/ABC"},
      {"name":"You", "type":"real", "link":"http://server/item/123"}],
   "subObjs": [
      "resources":[{"name":"Him", "type":"vm", "link":"http://server/item/DEF"} 
   ]
}

The transform function turns it into this:

{
   "resources": [
      {"name":"You", "type":"real", "link":"http://server/item/123"},
  ],
   "vms": [
      {"name":"Me", "type":"vm", "link":"http://server/item/ABC"}],
   "subObjs:" [
      "resources":[],
      "vms": [{"name":"Him","type":"vm", "link":"http://server/item/DEF"}]
   ]
}

We find those "resources" that are of type "vm" and move them to a separate array. So far so good. The transform function is simple enough. It's recursive because subOjbs can contain subObjs.

But now we need more information on the vms. We need to make Ajax calls to the server to get this information:

{
   "resources": [
      {"name":"You", "type":"real", "link":"http://server/item/123"}],
   "vms": [
      {"name":"Me", "type":"vm", "link":"http://server/item/ABC", "moreInfo":"X"}],
   "subObjs:" [
      "resources":[],
      "vms": [{"name":"Him","type":"vm", "link":"http://server/item/DEF", 
                "moreInfo":"Y"}]
   ]
}

The transform function looks something like this:

transform: function(obj) {
      var vms=[];
      var newResources = [];
      // Recurse on subObjs
      if (obj.subObjs) {
         for (var kx = 0; kx < obj.subObjs.length; kx++) {
            ojb.subObjs[kx] = this.transform(obj.subObjs[kx]);
      }
      // Move vms out of resources into vms
      if (obj.resources) {
         for (var jx = 0; jx < obj.resources.length; jx++) {
            if (obj.resources[jx].type == "vm") {
               var thisVM = obj.resources[jx];
               // Note:  more info needed here.
               //thisVM = this.getMoreInfo(thisVM);
               vms.push(thisVM);
            } else { 
               newResources.push(obj.resources[jx];
            }
         }
         obj.vms = vms;
         obj.resources = newResources;
      }
      return obj;
}

And now we have a problem. How do I wrote getMoreInfo()?

I could make synchronous calls at this point:

getMoreInfo: function(vm) {
   vmObj = callServerSynchronouslyToGET(vm.link);
   vm.moreInfo = vmObj ? vmObj.moreInfo : null;
}

But synchronous calls are never a good idea in Ajax (it would be Sjax).

I don't think it's possible to write getMoreInfo() as such to make asynchronous calls. I have to go back up through several layers of the onion and rewrite everything from a certain point downward, I hope without rewriting anything above the Comet-callback layer.

I know a recipe that transforms a recursive function into a non-recursive function. Is there a recipe that transforms an onion with a loop with synchronous GET at the center into a chain of asynchronous GETs?

Answer 1

There is a cross-browser extension to the JavaScript language called StratifiedJS.

It is designed to solve precisely the problem you are mentioning: It allows you to program in a synchronous style with everything being executed asynchronously under the hood.

The JS library that enables StratifiedJS on the browser is called "Oni Apollo". See http://onilabs.com/apollo for more details.

In your particular case, you could convert your whole asynchronous onion into synchronous code using StratifiedJS or you could keep your existing logic by just sticking it into a "text/sjs" script element:

<script src="http://code.onilabs.com/latest/oni-apollo.js"&gt;&lt;/script&gt;

<script type="text/sjs">

  // your existing code here

  getMoreInfo: function(vm) {
    var vmObj = require('http').get(vm.link);
    vm.moreInfo = vmObj ? vmObj.moreInfo : null;
  }
</script>

Here, require('http').get() performs an asynchronous XHR under the hood (for details, see the apollo api docs at the above link).