Queue ajax requests using jQuery.queue()

Question

I am using jQuery.queue() for the first time and haven't quite grasped it. Could someone please point out what I'm doing wrong?

Looking in firebug I am still seeing my POST requests firing at the same time - so I'm wondering if I'm calling dequeue() in the wrong place.

Also - how can I get the queue length?

The reason I need to queue these requests is that it gets fired on click of a button. And its possible for the user to click multiple buttons in quick succession.

Tried to strip out the basic structure of my code:

$("a.button").click(function(){
   $(this).doAjax(params);
});

// method
doAjax:function(params){ 

   $(document).queue("myQueueName", function(){
     $.ajax({
       type: 'POST',
       url: 'whatever.html',
       params: params,
       success: function(data){
         doStuff;

         $(document).dequeue("myQueueName");
       }
     });
   });

}

Electronics

This answer, also on StackOverflow, gives a great set of examples, including its use with ajax calls. – Whiny

I did actually try this approach - but i couldn't get this to work. I see it doesn't use a dequeue() and wondered if this may have been my problem? – Electronics

I guess next() is doing the same thing as dequeue? – Electronics

Yep, next() does the same thing. It's passed in by the queue logic when your function is called. – Whiny

Answer 1

You problem here is, that .ajax() fires an asyncronous running Ajax request. That means, .ajax() returns immediately, non-blocking. So your queue the functions but they will fire almost at the same time like you described.

I don't think the .queue() is a good place to have ajax requests in, it's more intended for the use of fx methods. You need a simple manager.

var ajaxManager = (function() {
     var requests = [];

     return {
        addReq:  function(opt) {
            requests.push(opt);
        },
        removeReq:  function(opt) {
            if( $.inArray(opt, requests) > -1 )
                requests.splice($.inArray(opt, requests), 1);
        },
        run: function() {
            var self = this,
                oriSuc;

            if( requests.length ) {
                oriSuc = requests[0].complete;

                requests[0].complete = function() {
                     if( typeof(oriSuc) === 'function' ) oriSuc();
                     requests.shift();
                     self.run.apply(self, []);
                };   

                $.ajax(requests[0]);
            } else {
              self.tid = setTimeout(function() {
                 self.run.apply(self, []);
              }, 1000);
            }
        },
        stop:  function() {
            requests = [];
            clearTimeout(this.tid);
        }
     };
}());

This is far away from being perfect, I just want to demonstrate the way to go. The above example could be used in a way like

$(function() {
    ajaxManager.run(); 

    $("a.button").click(function(){
       ajaxManager.addReq({
           type: 'POST',
           url: 'whatever.html',
           data: params,
           success: function(data){
              // do stuff
           }
       });
    });
});

Rocketry

ha, just realized you've named your function the same as the plugin i'm suggesting. – Yeomanry

@nathan: thanks, oh indeed - didn't know about your post, I was 4 minutes ahead :) – Rocketry

Okay - i understand why they were all firing at once thanks. I'm going to look into your suggestion of the handler. When would you suggest just using an existing plugin as @nathan suggested, or writing a handler like this? – Electronics

@MBax: I don't know the plugin, but I always prefer to do things on my own, for flexibility, knowledge and coolness :-) – Rocketry

@jAndy: :) cool indeed I am getting somewhere with this - but could you please explain what the setTimeout is doing? If the ajax request takes longer than this 1000 - could it cause probs?? Thanks – Electronics

@MBax: it just keeps the whole thing alive. If currently there are no queued requests, it just idle's and calls itself every second. To shutdown the whole thing call ajaxManager.stop(); – Rocketry

Just wanted to add a slight correction to the code above, for those trying to actually use the above code and it's not working for them. "params: params," should actually be "data: params," inside the actual request. I know this was originally in @MBax's code, but figured most people would look for the solution here. – Oysterman

jAndy ... if I attempt more than 2 simultaneous ajax requests, what happens to those additional requests normally? ... are they ignored/dropped? or are they automatically que'd by the browser? ... IF so, do all browsers behave the same?? ... and is there a standard window-bound object that manages that que? – Selfpossessed

many thanks for this large sample of code which works directly :) not perfect as he doesn't deal well with errors but yet, a long time saved thanks to you – Laynelayney

Just want to say. This is a fantastic fix. Thanks for sharing! Saved me a lot of time trying to create this very structure by myself – Lw

When do you have to use removeReq though? – Eddy

This is a great code, but I have some doubts; The ajax manager seems to resolved my problem. I have two types of request: 1) at the change of a select; 2) at the change of a input, where I have a forEach that call a lot of add(). If I use the stop() after the add(), the request is not send, otherwise if I don't use stop() or use before the add() the timer still there forever. How can I stop after all requests? – Amass

Well, I've just remove the else of if(requests.length) and I also call addReq() before run(). So it seems that worked. Thanks. – Amass

@jAndy, can you help me with this similar problem? #35549075 – Lewan

This works great! I was curious though, what keeps this from "being perfect" in your opinion? would like to understand any shortfalls you think are present with this code. – Lattonia

Answer 2

I needed to do a similar thing so thought I'd post my solution here.

Basically what I've got is a page which lists projects on shelves which all have distinctive criteria. I wanted to load the shelves one by one rather than altogether to get some content to the user quicker which they could look at whilst the rest loads.

Basically I stored the ID of each shelf in a JS array which I use when calling them from PHP.

I then created a recursive function which will pop the first index out of the array each time its called and request the shelf for the popped id. Once I have the response from the $.get() or $.post() whichever I prefer to use I then call the recursive function from within the callback.

Here's an elaboration in code:

// array of shelf IDs
var shelves = new Array(1,2,3,4);

// the recursive function
function getShelfRecursive() {

    // terminate if array exhausted
    if (shelves.length === 0)
        return;

    // pop top value
    var id = shelves[0];
    shelves.shift();

    // ajax request
    $.get('/get/shelf/' + id, function(){
         // call completed - so start next request
         getShelfRecursive();
    });
}

// fires off the first call
getShelfRecursive();

Headlight

To pop the top value, you could just have used var id = shelves.pop();. Then you wouldn't need to follow with an explicit .shift(); – Oratorian

Or if you wanted the the .shift() end, you could have used var id = shelves.shift(); – Oratorian

Answer 3

I use this very simple code to keep ajax calls from "overtaking" each other.

var dopostqueue = $({});
function doPost(string, callback)
{
    dopostqueue.queue(function()
    {
        $.ajax(
        {   
            type: 'POST',
            url: 'thephpfile.php',
            datatype: 'json',
            data: string,
            success:function(result) 
            {
                dopostqueue.dequeue();
                callback(JSON.parse(result));
            }
        })
    });
}

If you don't want the queue to handle itself, you can just remove the dequeue from the function and call it from another function. As to getting the queue length, for this example it would be:

dopostqueue.queue().length

Intact

Answer 4

you could extend jQuery:

(function($) {
  // Empty object, we are going to use this as our Queue
  var ajaxQueue = $({});

  $.ajaxQueue = function(ajaxOpts) {
    // hold the original complete function
    var oldComplete = ajaxOpts.complete;

    // queue our ajax request
    ajaxQueue.queue(function(next) {    

      // create a complete callback to fire the next event in the queue
      ajaxOpts.complete = function() {
        // fire the original complete if it was there
        if (oldComplete) oldComplete.apply(this, arguments);    
        next(); // run the next query in the queue
      };

      // run the query
      $.ajax(ajaxOpts);
    });
  };

})(jQuery);

then use it like:

$.ajaxQueue({
    url: 'doThisFirst.php',
    async: true,
    success: function (data) {
        //success handler
    },
    error: function (jqXHR,textStatus,errorThrown) {
        //error Handler
    }       
});
$.ajaxQueue({
    url: 'doThisSecond.php',
    async: true,
    success: function (data) {
        //success handler
    },
    error: function (jqXHR,textStatus,errorThrown) {
        //error Handler
    }       
});

of course you can use any of the other $.ajax options like type, data, contentType, DataType since we are extending $.ajax

Arteriosclerosis

Rad solution. To test the queuing I added a 1 second sleep in my PHP code and queued as many AJAX requests as I could. Each one finished as expected. Before it would trip up and miss a couple saves. Very cool. – Inhibitory

perfect! is it possible to know when the queueis empty or is running? – Evansville

Answer 5

I found the above solutions kind of complicated, plus I needed to alter the request just before sending (to update a fresh data token).

So I put this one together. Source: https://gist.github.com/2470554

/* 

Allows for ajax requests to be run synchronously in a queue

Usage::

var queue = new $.AjaxQueue();

queue.add({
  url: 'url',
  complete: function() {
    console.log('ajax completed');
  },
  _run: function(req) {
    //special pre-processor to alter the request just before it is finally executed in the queue
    req.url = 'changed_url'
  }
});

*/

$.AjaxQueue = function() {
  this.reqs = [];
  this.requesting = false;
};
$.AjaxQueue.prototype = {
  add: function(req) {
    this.reqs.push(req);
    this.next();
  },
  next: function() {
    if (this.reqs.length == 0)
      return;

    if (this.requesting == true)
      return;

    var req = this.reqs.splice(0, 1)[0];
    var complete = req.complete;
    var self = this;
    if (req._run)
      req._run(req);
    req.complete = function() {
      if (complete)
        complete.apply(this, arguments);
      self.requesting = false;
      self.next();
    }

    this.requesting = true;
    $.ajax(req);
  }
};

Decapitate

Please put the code (or at least relevant part of it) here as well. For all we know, github might be down tomorrow to never come back and this post will become meaningless. – Tocology

Wow. Really? "For all we know, github might be down tomorrow" – Unlimber

I assume Dec/2012 were bad days for Github? – Dowitcher

Answer 6

I needed to do this for an unknown number of ajax calls. The answer was to push each into an array and then use:

$.when.apply($, arrayOfDeferreds).done(function () {
    alert("All done");
});

Dalmatia

don't the ajax calls fire immediately when they're pushed onto the array. For example, var arr = []; arr.push($.get(...)); would fire the GET call immediately? – Decoct

could you make a more complete example? – Kosse

Answer 7

Another version of jAndy's answer, without timer.

var ajaxManager = {
    requests: [],
    addReq: function(opt) {
        this.requests.push(opt);

        if (this.requests.length == 1) {
            this.run();
        }
    },
    removeReq: function(opt) {
        if($.inArray(opt, requests) > -1)
            this.requests.splice($.inArray(opt, requests), 1);
    },
    run: function() {
        // original complete callback
        oricomplete = this.requests[0].complete;

        // override complete callback
        var ajxmgr = this;
        ajxmgr.requests[0].complete = function() {
             if (typeof oricomplete === 'function')
                oricomplete();

             ajxmgr.requests.shift();
             if (ajxmgr.requests.length > 0) {
                ajxmgr.run();
             }
        };

        $.ajax(this.requests[0]);
    },
    stop: function() {
        this.requests = [];
    },
}

To Use:

$(function() {
    $("a.button").click(function(){
       ajaxManager.addReq({
           type: 'POST',
           url: 'whatever.html',
           data: params,
           success: function(data){
              // do stuff
           }
       });
    });
});

Overly

How to queue .getJSON instead of ajax? – Lewan

Glbert, can you help me with this: #35541356 ? – Lewan

Answer 8

The learn.jquery.com website have a good example too:

// jQuery on an empty object, we are going to use this as our queue
var ajaxQueue = $({});

$.ajaxQueue = function(ajaxOpts) {
  // Hold the original complete function
  var oldComplete = ajaxOpts.complete;

  // Queue our ajax request
  ajaxQueue.queue(function(next) {
    // Create a complete callback to invoke the next event in the queue
    ajaxOpts.complete = function() {
      // Invoke the original complete if it was there
      if (oldComplete) {
        oldComplete.apply(this, arguments);
      }

      // Run the next query in the queue
      next();
    };

    // Run the query
    $.ajax(ajaxOpts);
  });
};

// Get each item we want to copy
$("#items li").each(function(idx) {
  // Queue up an ajax request
  $.ajaxQueue({
    url: "/ajax_html_echo/",
    data: {
      html: "[" + idx + "] " + $(this).html()
    },
    type: "POST",
    success: function(data) {
      // Write to #output
      $("#output").append($("<li>", {
        html: data
      }));
    }
  });
});

Rinse

Answer 9

I also had to do this within a solution i had and I found I could do it this way:

//A variable for making sure to wait for multiple clicks before emptying.
var waitingTimeout; 

$("a.button").click(function(){
   $(this).doAjax(params);
   clearTimeout(waitingTimeout);
   waitingTimeout = setTimeout(function(){noMoreClicks();},1000);
});

// method
doAjax:function(params){ 

   $(document).queue("myQueueName", function(next){
     $.ajax({
       type: 'POST',
       url: 'whatever.html',
       data: params,
       contentType: "application/json; charset=utf-8",
       dataType: "json",
       success: function(data){
         doStuff;
         next();
       },
       failure: function(data){
         next();
       },
       error: function(data){
         next();
       }
     });
   });

}

function noMoreClicks(){
    $(document).dequeue("myQueueName");
}

by using the next() callback that is passed in the queue function you can dequeue the next operation. So by putting the next in the handlers for the ajax, you effectively make the ajax calls asynchronous to the browser and the render or paint thread of the browser, but make them synchronous or serialized to each other.

Here is a very basic example. In the example fiddle. Click the button once and wait a second. You will see that the time out triggers and the single operation happens. Next click the button as fast as you can (or faster than one second) and you will see that all the times you click the button, the operations are queued and then only after waiting a second do they hit the page and fade in one after the other.

The beauty of this is that if the queue is already emptying, any operations you add to it while it is emptying are placed on the end and then just processed when the time comes.

Leghorn

Answer 10

Here is my solution, which I use to produce a queue of requests for some Browsergame. If anything happens I stop this queue and finish the work with some special last request or cleanup.

var get_array = ["first", "second", "third"];

var worker = $("<div />"); // to line up requests in queue
$.queuedAjax = function(args){  // add up requests for me       
    worker.queue(
        function(next){
            $.ajax(args).always(next);            
        }
    );
  };

$.queuedSomething = function(){ // add up something special for me
    worker.queue(
        function(next){
            //worker.clearQueue();
            //worker = $("<div />"); //cleanup for next .each
            //maybe another .each           
        }
    );
  };

$.each( get_array , function( key , value ) {
  $.queuedAjax({
    type: 'GET',
    url: '/some.php?get='+value,
    dataType: 'text',
    success: function(sourcecode){

        if (sourcecode.match(/stop your requests, idiot!/)) {   
            worker.clearQueue().queue($.queuedSomething);
            alert(' the server told me to stop. i stopped all but not the last ´$.queuedSomething()´ ');
        }

    }
  });           
}); 
$.queuedSomething();

Boson

Answer 11

just another example of a multi threaded queue runner i wrote for nodejs. You could adapt it to jquery or angular. Promises are slightly different in each API. I've used this pattern for things like extracting all items from large lists in SharePoint by creating multiple queries to fetch all data and allowing 6 at a time, to avoid server-imposed throttling limits.

/*
    Job Queue Runner (works with nodejs promises): Add functions that return a promise, set the number of allowed simultaneous threads, and then run
    (*) May need adaptation if used with jquery or angular promises

    Usage:
        var sourcesQueue = new QueueRunner('SourcesQueue');
        sourcesQueue.maxThreads = 1;
        childSources.forEach(function(source) {
            sourcesQueue.addJob(function() { 
                // Job function - perform work on source
            });
        }
        sourcesQueue.run().then(function(){
            // Queue complete...
        });
*/
var QueueRunner = (function () {
    function QueueRunner(id) {
        this.maxThreads = 1; // Number of allowed simultaneous threads
        this.jobQueue = [];
        this.threadCount = 0;
        this.jobQueueConsumer = null;
        this.jobsStarted = 0;
        if(typeof(id) !== 'undefined') {
            this.id = id;
        }
        else {
            this.id = 'QueueRunner';
        }
    }    
    QueueRunner.prototype.run = function () {
        var instance = this;        
        return new Promise(function(resolve, reject) {
            instance.jobQueueConsumer = setInterval(function() {
                if(instance.threadCount < instance.maxThreads && instance.jobQueue.length > 0) {
                    instance.threadCount++;
                    instance.jobsStarted++;
                    // Remove the next job from the queue (index zero) and run it
                    var job = instance.jobQueue.splice(0, 1)[0];
                    logger.info(instance.id + ': Start job ' + instance.jobsStarted + ' of ' + (instance.jobQueue.length + instance.jobsStarted));
                    job().then(function(){
                        instance.threadCount--;
                    }, function(){
                        instance.threadCount--;
                    });
                }
                if(instance.threadCount < 1 && instance.jobQueue.length < 1) {
                    clearInterval(instance.jobQueueConsumer);
                    logger.info(instance.id + ': All jobs done.');
                    resolve();
                }
            }, 20);
        });     
    };
    QueueRunner.prototype.addJob = function (func) {
        this.jobQueue.push(func);
    };
    return QueueRunner;
}());

Detached

Answer 12

Using a framework which provides observable support such as knockout.js you can implement an observing queue which when pushed onto will enqueue the call and a shift will process the process.

A knockout implementation would look like the following:

var ajaxQueueMax = 5;
self.ajaxQueue = ko.observableArray();
self.ajaxQueueRunning = ko.observable(0);

ko.computed(function () {
  if (self.ajaxQueue().length > 0 && self.ajaxQueueRunning() < ajaxQueueMax) {
    var next = self.ajaxQueue.shift();
    self.ajaxQueueRunning(self.ajaxQueueRunning() + 1);
    $.ajax(next).always(function () {
      self.ajaxQueueRunning(self.ajaxQueueRunning() - 1);
    });
  }
});

Observe that we take advantage of the observables telling us when we should send off another ajax request. This method can be applied in a more generalised form.

As an example, imagine you had a knockout mapping that retrieved lots of entries but you needed to call another service per item to enrich them, say set a value.

self.widgets = ko.observableArray();

ko.computed(function () {
  var mapping = {
    create: function (options) {
      var res = ko.mapping.fromJS(options.data);
      res.count = ko.observable();

      // widget enrichment.
      self.ajaxQueue.push({
        dataType: "json",
        url: "/api/widgets/" + options.data.id + "/clicks",
        success: function (data) {
          res.count(data);
        }
      });
      return res;
    }
  };

  // Initial request for widgets
  $.getJSON("/api/widgets", function (data) {
    ko.mapping.fromJS(data, mapping, self.widgets);
  });
});

Termination