Limit method calls per second(s) (refuse when limit reached)

Question

I have a method that does some IO, and I'd like to limit the calls (per second) to this method to avoid the backend to get bursts of concurrent requests it can't handle.

If the requirement came without the "per second" I could just use a stack (basically just a counter) and offer() when starting a request, and poll() when done. With the "per second" requirement I'd somehow need to clean slots on the stack that are older than a given lapse of time.

How do I do that correctly? The structure should be thread-safe, obviously.

Thank you for your time!

Dawkins

Have a look at this question (#668008). It's equally applicable to this problem. – Chu

I did some unit testing on that, and it looks like it works flawlessly (until now). Post this as an answer please. – Dawkins

Did you consider using Google Guava RateLimiter? See my post [here][1]. [1]: https://mcmap.net/q/143845/-throttling-method-calls-to-m-requests-in-n-seconds – Immerse

@schnatterer: the RateLimiter blocks when the threshhold has been reached. I need it to tell me thati t has been reached, not block. – Dawkins

Answer 1

Have a look at this question. The answer is equally applicable to this problem, even if the question is phrased rather differently.

I confess, I find it slightly mysterious, but it works beautifully.

Chu

Answer 2

You could use a queue but with some modifications. Whenever you want to add to the queue (the method gets called) you check how many elements are in it and when they were added. All elements that were inserted more than one second ago can be removed. Compare the number of remaining elements with the rate per second and you have a decision whether to reject the method execution or not. It might get a little bit more complicated if you intend to have the method block instead of reject.

There are likely to be many possible implementations and you have to check whether they are feasible for your problem. You could also take a look at implementations of Token buckets which is the general concept of your problem.

Unzip

Your first solution works, but I'd prefer to avoid the overhead of removing let's say 1000 entries after some inactivity. It's however better than my "queue where every element has a timeout after which it removes itself from the collection" I had in mind (it's horrible, but simple (I think ;)). – Dawkins

Timing could be an issue in both your and my version. I think Java is not really suited that well for this kind of requirement. Prepare to do some measuring. ;) – Unzip

Answer 3

It sounds like you need to decouple the IO work from the requesting thread and offload it instead to a pool of threads with a given size. That way you can explicitly control the number of threads doing IO at the same time.

ExecutorService pool = Executors.newFixedThreadPool(10);

public void myMethod() {

  pool.submit(new Runnable() {
    public void run() {
      //Do IO work here
    }
  });

}

In this example there will never be more than 10 threads doing IO work. Its not limiting the number of requests per second but controlling the concurrency is probably a better way of throttling.

Argyle