How to wait for the last promise in a dynamic list of promises?

Question

I have a function F that starts an asynchronous process X. The function returns a promise that is resolved when X ends (which I learn by means of a promise returned by X).

While the (w.l.o.g.) first instance of X, X1, is running, there may be more calls to F. Each of these will spawn a new instance of X, e.g. X2, X3, and so on.

Now, here's the difficulty: When X2 is created, depending on the state of X1, X1 should conclude or be aborted. X2 should start working only once X1 is not active any more. In any case, the unresolved promises returned from all previous calls to F should be resolved only once X2 has concluded, as well - or, any later instance of X, if F gets called again while X2 is running.

So far, the first call to F invokes $q.defer() to created a deferred whose promise is returned by all calls to F until the last X has concluded. (Then, the deferred should be resolved and the field holding it should be reset to null, waiting for the next cluster of calls to F.)

Now, my issue is waiting until all instances of X have finished. I know that I could use $q.all if I had the full list of X instances beforehand, but as I have to consider later calls to F, this is not a solution here. Ideally, I should probably then-chain something to the promise returned by X to resolve the deferred, and "unchain" that function as soon as I chain it to a later instance of X.

I imagine that something like this:

var currentDeferred = null;

function F() {
    if (!currentDeferred) {
        currentDeferred = $q.defer();
    }

    // if previous X then "unchain" its promise handler
    X().then(function () {
        var curDef = currentDeferred;
        currentDeferred = null;
        curDef.resolve();
    });

    return currentDeferred.promise;
}

However, I don't know how to perform that "unchaining", if that is even the right solution.

How do I go about this? Am I missing some common pattern or even built-in feature of promises, or am I on the wrong track altogether?

---

To add a little context: F is called to load data (asynchronously) and updating some visual output. F returns a promise that should only be resolved once the visual output is updated to a stable state again (i.e. with no more updates pending).

Deadeye

@T.J.Crowder: I just added a sample of how I imagine this to work. – Deadeye

@t.niese: I just did. I think from the code sample, it becomes apparent that it is not as easy as chaining the new promise to the last one, because once the previous promise has run, my reference to currentDeferred is gone, and currentDeferred has been resolved (even though it should not if another instance of X is about to follow). – Deadeye

@T.J.Crowder: No, I'm not saying X1 should wait for X2. I'm saying the promise returned from the call to F that spawned X1 should wait for X2. I have added the word returned to clarify that in the second sentence you cite, I am just referringt to the promises returned from calls to F, not to the "internal" promises obtained from X. Thanks for the hint. – Deadeye

Sounds a bit like an XY Problem. The [...] In any case, the unresolved promises returned from all previous calls to F should be resolved only once X2 has concluded, as well - or, any later instance of X, if F gets called again while X2 is running.[...] is fishy, because it could lead to infinite pile up. – Narrow

"I should probably then-chain something to the promise, and later "unchain" that function" - yes, that's how it ideally would work, but unfortunately native promises do not support cancellation as "un-chaining" callbacks. You can however try my promise library Creed that does support this usage, and here's an example that does exactly what you want. – Anzovin

@T.J.Crowder: "It's also impossible, in the general case; X1's processing may finish before the call to F that produces X2." - that's why I wrote "until the last X has concluded. (Then, the deferred should be resolved and the field holding it should be reset to null, waiting for the next cluster of calls to F.)". If X1 finishes before F is called again, the promise returned by X1's F call is resolved, and the system goes inactive until the next call to F. – Deadeye

"Sounds a bit like an XY Problem." - I have added a brief paragraph on the concrete context. "it could lead to infinite pile up." - I'm not quite seeing that issue. A data update does not automatically cause another data update; only user interaction does. – Deadeye

If I understand correctly, you have a variant of the "Chinese plate spinner" problem. In this variant, (1) plates, once started, may not be revived but new plates may be added to keep the act alive, (2) the act is over when all plates have fallen. – Trotskyite

@Roamer-1888: This sounds like a fitting description, but is that the "official" name of said logical/concurrency problem? I can't find anything online about it. – Deadeye

"Chinese plate spinner" was an old coding exercise given to students years ago, years before promises, or even the web, were ever conceived. The other, more popular one, from the same era was the "Dining philosophers problem". I recall making a real hash of both of them. – Trotskyite

Hava a look at getting-the-latest-data-from-a-promise-returning-service-called-repeatedly. In my answer i show a simple implementation of cancellable promises as an extention to the native promises. You may not need a library just for this job. – Recapture

If your X1 is a special case - an "umbrella" for X2, X3 et seq - then you may be better off with a constructor that you call initially (with or without new) to obtain an instance, eg var foo = new Foo(). Then, from the instance obtain your umbrella, eg var X1 = foo.promise(). Then call some other method to obtain further promises, eg var X2 = foo.add(). In addition to the two methods, Foo() will consist primarily of the solution offered by T J Crowder. – Trotskyite

@Roamer-1888: No, none of the instances of X is inherently a special case. Arguably, only the last one in a cluster of calls to F happens to be a bit special, in that its conclusion leads to the resolution of the promise returned by F. But basically, each X is exactly the same process. – Deadeye

Ah, Ok, in that case I have misread the question; or rather X1 being a special case is one interpretation that I can put on it. – Trotskyite

@Roamer-1888: Thanks for pointing it out. I have added "w.l.o.g." in order to possibly reduce the confusion for readers. – Deadeye

Answer 1

F is called to load data (asynchronously) and updating some visual output. F returns a promise that should only be resolved once the visual output is updated to a stable state again (i.e. with no more updates pending).

Since all callers of F will receive a promise they need to consume, but you only want to update the UI when all stacked calls have completed, the simplest thing is to have each promise resolve (or reject) with a value telling the caller not to update the UI if there's another "get more data" call pending; that way, only the caller whose promise resolves last will update the UI. You can do that by keeping track of outstanding calls:

let accumulator = [];
let outstanding = 0;
function F(val) {
  ++outstanding;
  return getData(val)
    .then(data => {
      accumulator.push(data);
      return --outstanding == 0 ? accumulator.slice() : null;
    })
    .catch(error => {
      --outstanding;
      throw error;
    });
}

// Fake data load
function getData(val) {
  return new Promise(resolve => {
    setTimeout(resolve, Math.random() * 500, "data for " + val);
  });
}

let accumulator = [];
let outstanding = 0;
function F(val) {
  ++outstanding;
  return getData(val)
    .then(data => {
      accumulator.push(data);
      return --outstanding == 0 ? accumulator.slice() : null;
    })
    .catch(error => {
      --outstanding;
      throw error;
    });
}

// Resolution and rejection handlers for our test calls below
const resolved = data => {
  console.log("chain done:", data ? ("update: " + data.join(", ")) : "don't update");
};
const rejected = error => { // This never gets called, we don't reject
  console.error(error);
};

// A single call:
F("a").then(resolved).catch(rejected);

setTimeout(() => {
  // One subsequent call
  console.log("----");
  F("b1").then(resolved).catch(rejected);
  F("b2").then(resolved).catch(rejected);
}, 600);

setTimeout(() => {
  // Two subsequent calls
  console.log("----");
  F("c1").then(resolved).catch(rejected);
  F("c2").then(resolved).catch(rejected);
  F("c3").then(resolved).catch(rejected);
}, 1200);

.as-console-wrapper {
  max-height: 100% !important;
}

(That's with native promises; adjust as necessary for $q.)

To me, "don't update" is different from "failed," so I used a flag value (null) rather than a rejection to signal it. But of course, you can use rejection with a flag value as well, it's up to you. (And that would have the benefit of putting the conditional logic ["Is this a real error or just a "don't update"?] in your catch handler rather than your then [is this real data or not?]... Hmmm, I might go the other way now I think of it. But that's trivial change.)

Obviously accumulator in the above is just a crude placeholder for your real data structures (and it makes no attempt to keep the data received in the order it was requested).

I'm having the promise resolve with a copy of the data in the above (accumulator.slice()) but that may not be necessary in your case.

Calamint

This looks promising (no pun intended). Indeed, I think I can resolve my deferred only if outstanding is 0 rather than signal anything back to earlier callers of F at all, while using this technique to keep track of whether there are any pending invocations of F. – Deadeye

@O.R.Mapper: Glad that helpes. I believe not resolving the deferred at all would be violating the contract of a promise, which is that it will resolve or reject. Having said that, I don't see it in the Promises/A+ spec. That spec is intentionally minimal, though, focussing on an interoperable then method. I also wonder about memory implications, though that may be paranoia on my part (you'll be able to tell easily enough with memory profiling). – Calamint

Oh, I will resolve the promise returned by the F call that spawned X1, but not in my promise handler at the end of X1 (but only in my promise handler at the end of the last concurrently launched X). Remember that each call of F returns the same promise until there are no more pending calls and the promise gets resolved once. – Deadeye