Throttle JavaScript function calls, but with queuing (don't discard calls)

Question

How can a function rate-limit its calls? The calls should not be discarded if too frequent, but rather be queued up and spaced out in time, X milliseconds apart. I've looked at throttle and debounce, but they discard calls instead of queuing them up to be run in the future.

Any better solution than a queue with a process() method set on an X millisecond interval? Are there such standard implementations in JS frameworks? I've looked at underscore.js so far - nothing.

Stigma

Whats wrong with the interval timer method? – Cineaste

@Petah: nothing in principle, but I don't want to reinvent the wheel. – Stigma

Its hardly reinventing the wheel, should be < 20 LOC. – Cineaste

I don't like the word "interval" here, especially if you're doing things that have arbitrary processing requirements. Use the word timeout and your phraseology will match how you should be writing the code (with setTimeout) – Bourke

@PaulS.: the use I had in mind for setInterval was to process the queue, like in this example. – Stigma

@DanDascalescu With setInterval, when time to process exceeds the delay, you end up in an environment where the second tries to happen before the first finishes, which makes both slower, so the third tries to happen before the second finishes, and maybe the first. Ultimately, the n+1th starts to happen before the nth and n-1th have finished, and then all hell breaks loose – Bourke

Answer 1

While the snippets offered by others do work (I've built a library based on them), for those who want to use well-supported modules, here are the top choices:

• the most popular rate limiter is limiter
• function-rate-limit has a simple API that works, and good usage stats on npmjs
• valvelet, a newer module, claims to do an even better job by supporting promises, but hasn't gained popularity yet

Stigma

Existing answers and libraries mentioned didn't satisfy my need so I wrote yet another one: debounce-queue. Its unique feature being that it gives an array of all previous items instead of just the last/first one. – Charry

Answer 2

Should be rather simple without a library:

var stack = [], 
    timer = null;

function process() {
    var item = stack.shift();
    // process
    if (stack.length === 0) {
        clearInterval(timer);
        timer = null;
    }
}

function queue(item) {
    stack.push(item);
    if (timer === null) {
        timer = setInterval(process, 500);
    }
}

http://jsfiddle.net/6TPed/4/

Cineaste

Thanks for this clear answer. I've combined it with @PaulS.'s answer to return a limited function, and I've made a change to start the queue right away (because setInterval delays first). Here's the fiddle. What do you think? – Stigma

Answer 3

Here is an example which carries forward this (or lets you set a custom one)

function RateLimit(fn, delay, context) {
    var canInvoke = true,
        queue = [],
        timeout,
        limited = function () {
            queue.push({
                context: context || this,
                arguments: Array.prototype.slice.call(arguments)
            });
            if (canInvoke) {
                canInvoke = false;
                timeEnd();
            }
        };
    function run(context, args) {
        fn.apply(context, args);
    }
    function timeEnd() {
        var e;
        if (queue.length) {
            e = queue.splice(0, 1)[0];
            run(e.context, e.arguments);
            timeout = window.setTimeout(timeEnd, delay);
        } else
            canInvoke = true;
    }
    limited.reset = function () {
        window.clearTimeout(timeout);
        queue = [];
        canInvoke = true;
    };
    return limited;
}

Now

function foo(x) {
    console.log('hello world', x);
}
var bar = RateLimit(foo, 1e3);
bar(1); // logs: hello world 1
bar(2);
bar(3);
// undefined, bar is void
// ..
// logged: hello world 2
// ..
// logged: hello world 3

Bourke

I've combined this answer with @Petah's solution of using setInterval. Here's the fiddle. What do you think? – Stigma

Answer 4

I needed a TypeScript version of this, so I took @Dan Dascelescu's fiddle and added types to it.

Please leave a comment if you can improve the typings 🙏

function rateLimit<T>(
  fn: (...args: Array<any>) => void,
  delay: number,
  context?: T
) {
  const queue: Array<{ context: T; arguments: Array<any> }> = []
  let timer: NodeJS.Timeout | null = null

  function processQueue() {
    const item = queue.shift()

    if (item) {
      fn.apply<T, Array<any>, void>(item.context, item.arguments)
    }

    if (queue.length === 0 && timer) {
      clearInterval(timer)
      timer = null
    }
  }

  return function limited(this: T, ...args: Array<any>) {
    queue.push({
      context: context || this,
      arguments: [...args],
    })

    if (!timer) {
      processQueue() // start immediately on the first invocation
      timer = setInterval(processQueue, delay)
    }
  }
}

Aixlachapelle

Answer 5

I think this could get simpler and more reusable with decorators:

/**
 * Sleeps async for a given amount of time.
 * @param milisec
 * @returns
 */
function asyncDelay(milisec: number): Promise<void> {
  return new Promise<void>((resolve) => {
    setTimeout(() => { resolve(); }, milisec);
  });
}

/**
 * Generates a random int within the max and min range.
 * Maximum is exclusive and minimum is inclusive.
 * @param min
 * @param max
 */
export const getRandomInt = (
  min: number,
  max: number,
): number => (
  Math.floor(
    Math.random() * (
      Math.floor(max) - Math.ceil(min)
    ) + Math.ceil(min),
  )
);

/**
 * Will throttle a method by a fixed ms time, if number is passed.
 * if tuple, will throttle by a random time (ms) between each.
 * @param milliseconds
 * @returns
 */
function throttle(milliseconds: number | [number, number]): any {
  let lastCall = 0;
  return function (target: any, propertyKey: string, descriptor: PropertyDescriptor) {
    const originalMethod = descriptor.value;
    descriptor.value = async function (...args: any[]) {
      const now = Date.now();
      if (!lastCall) {
        lastCall = now;
        return originalMethod.apply(this, args);
      }
      const ms = Array.isArray(milliseconds)
        ? getRandomInt(milliseconds[0], milliseconds[1])
        : Math.round(milliseconds);

      const diff = now - lastCall;
      if (diff < ms) {
        await asyncDelay(ms - diff);
      }
      lastCall = Date.now();
      return originalMethod.apply(this, args);
    };

    return descriptor;
  };
}

The you can just do:

class MyClass {
  @throttle(1000)
  async fixedRateLimitedMethod() {
    await doSomething()
  }

  @throttle([1000, 5000])
  async randomRateLimitedMethod() {
    await doSomething()
  }
}

TSPlayground

Quintillion