Throttling a Rest API in Java

Question

I wanted to add a way to throttle the number of requests coming on each API from a certain client. So, I wanted to basically limit the number of requests per API per client.

I am using DropWizard as framework. Can somebody recommend the ways to achieve this? I need something that will work for Distributed system.

Abernethy

If yo'er looking to rate limit instead of hard limit don't overlook: docs.guava-libraries.googlecode.com/git/javadoc/com/google/… – Diedrediefenbaker

This is good. But I am looking for a distributed solution i.e. which will work on vip instead of machine directly. Please suggest if it is possible in some way. – Abernethy

Answer 1

A simplistic approach would be to use a Filter and wrap it around all your API calls in web.xml. Assuming your clients send an API keys identifying them in a HTTP header, you could implement a filter like this:

public class MyThrottlingFilter extends Filter {

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {

        HttpServletRequest httpreq = (HttpServletRequest) req;
        String apiKey = httpreq.getHeader("API_KEY")

        if (invocationLimitNotReached(apiKey))
            chain.doFilter(req, res);
        else
            throw ...
    }
}

and then register it like this:

<filter>
    <filter-name>MyThrottlingFilter</filter-name>
    <filter-class>com.my.throttler.MyThrottlingFilter</filter-class>
</filter>
<filter-mapping>
    <filter-name>MyThrottlingFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

Of course, identifying your clients may be more difficult than this, if you use some other authentication methods, but the general idea should be the same.

Uncut

This is good. But I am looking for a distributed solution i.e. which will work on vip instead of machine directly. Please suggest if it is possible in some way. – Abernethy

Then I'd go with a load balancer. Try settung up apache with mod_proxy_balancer or HAProxy as R4J suggested. But this has nothing to do with the question you've originally asked, and is completely unrelated to DropWizard and Java. – Uncut

Answer 2

Do you want to have such logic enclosed in your application? Maybe some external Load Balancer would be a better choice?

You could try out HAProxy and have entire throtlling logic outside of your application.

A big advantage of such approach is the fact you would not have to rebuild and redeploy application whenever throtlling requirements change. Also, HAProxy will take much smaller amount of time to restart than a typical Java application.

Sediment

Answer 3

I think an interceptor like HandlerInterceptor would solve the purpose.

Nadaha

This is good. But I am looking for a distributed solution i.e. which will work on vip instead of machine directly. Please suggest if it is possible in some way. – Abernethy

Answer 4

If you absolutely must have it in Dropwizard, then I would do as npe suggests. The change that is needed is to share the "rate" via an external process, e.g. redis.

So in npe's example, invocationLimitNotReached would check a redis host to figure out what the current rate is (perhaps it stores a list of all current requests), and if adding the current request would go over that threshold.

If adding the current request wouldn't exceed the allowed rate, then you add the request to the list, and when the request is finished, you remove the request from the redis list. The redis list entries can have a TTL, so if a dropwizard instance that is servicing 20 requests suddenly disappears, after the TTL, they're removed from the list of "currently executing requests".

Danieledaniell

Answer 5

Building on the answer given by npe, you can "achieve distributed"-ness by storing the per user API hit counts in a central store like Redis which the invocationLimitNotReached(apiKey) method can then use to determine rate limit breaches.

The hard part, of course, is to determine how to "expire" the hit counts falling outside your limiting window.

Chadwick