API Throttling Best Practices

Question

I have a SOAP api that I would like to throttle access to on a User basis after "x" many calls have been received in "y" amount of time.

After searching around, the #1 consideration (obviously) is to consider your parameters for when to throttle users. However, I don't see much in the way of best practices/examples for implementing such a solution. I did see the Leaky Bucket Method which makes sense. I have to believe there are more ideas out there though.

Any other takers on how you go about implementing your throttling solution? Questions include:

• Do any frameworks provide capabilities (e.g. Spring, etc.) for throttling in web apis?
• Seems to me you would need to store access information per user. How do you minimize the database overhead for doing this EVERY call?
• Do you even NEED to access a datastore to implement this?

Osteitis

Just a note that leaky bucket is more targeted to traffic (uncountable, flowing content), where for request/call rate shaping you would use Token bucket Algorithm. There's just a slight difference - the mechanism is the same but instead of pouring water into the bucket, you add countable tokens, which fits better to requests. – Eightfold

Answer 1

For what its worth, I've sort of answered this question after working on some other production projects.

1. Home brew: Using Spring AOP to pointcut around the method calls prior to executing API method code is one home-brew way if you have your own algorithm to implement. This ends up being pretty elegant and flexible as you can capture a lot of metadata prior to deciding what to do with the request.
2. API Management Service: If you're talking about a production system and you have the budget, probably the best way to go is to delegate this to an API Management layer like Apigee or Mashery.

Advantage is that it separates the concerns so its easier to change and allows you to focus just on your API. This is especially helpful if business stakeholders are involved and you need a good UI and dictionary of terms.

Disadvantage, of course is the cost and the vendor lock in.

Hope this helps someone!

Osteitis