Queuing Guzzle Requests With Limits

Question

I'm working on a Laravel application, using Guzzle 6. A lot of functionality relies on an API, of which I've created a wrapper for.

My wrapper's a single class, that creates the Guzzle client in the __construct(), and has a variety of public functions which return responses from Guzzle requests.

The API I'm using has a limit of 40 requests every 10 seconds. I am caching things, so it would be very rare to hit this limit, but I'd like to know that my application wouldn't just die if it did!

Some notes about my app:

• API calls are only made if the same call hasn't been made in the past 6 hours. If it has, the call is never made and the response is served directly from my redis cache.
• In most cases, API calls are made via user actions. The application would never get close to hitting these limits itself.
• In most cases, I already have the required data to show the requested pages to users. An API call may be done in the background to see if anything needs to be updated on my end, but if I already have the data, and the API request failed, this wouldn't render the page useless.
• The app is live, https://likethis.tv if you'd like to look. I'm using TMDb API.

So, my question is, how should I make sure I do not hit this limit? A few ideas of mine are the following:

• Use the Laravel queuing system to place the Guzzle requests into a queue, and only process them if we still have requests left. If not, wait until the 10 second cooldown has passed...
• Use a HandlerStack for Guzzle directly. Not sure if this is possible, but I've used the HandlerStack for caching responses before.

I'm trying to not to provoke too opinionated responses, but I'm sure there's probably a better and/or easier way than the above, or if they are good ideas, any pointers or recommendations would be great.

Thanks in advance.

Arietta

Added a bounty on this one. I also would like to know how to effectively throttle my own call to an API and I was thinking of using queues also, but can't figure the proper way to do it. – Forzando

should the response of the api passed into a db or should it be displayed to the client? That second case looks to be the difficult one. – Nutgall

Is order of API calls relevant in some way? – Feeble

The response is used to update/create records in the database. The order isn't important. At the moment my wrapper is only handling 404s, so I think I'll redo that bit, and hand the error response to whatever just made the call so it can decide what to do. For example, I don't need to throw an error if I have a record in my DB when only API request errored. – Arietta

Answer 1

1. Wrap your API calls with Jobs and push them to separate queue:

   ApiJob::dispatch()->onQueue('api');
   

2. Use mxl/laravel-queue-rate-limit package (I'm the author) to rate limit api queue. Add this to config/queue.php:

   'rateLimit' => [
       'api' => [
           'allows' => 40,
           'every' => 10
       ]
   ]
   

3. Run queue worker:

   $ php artisan queue:work --queue api
   

See also this answer.

Mueller

Answer 2

There's not enough information to really dig deep into this, but to get you started, good APIs typically return a 429 response code when you're exceeding their throttled limit.

You could use $res->getStatusCode() from guzzle to check for this and flash a message back to the user if they're making too many requests too quickly.

Can you give some more information about what your app is doing? Are you making requests in a foreach loop? Is the view dependent on data from this API?

Cryotherapy

The API does return useful error codes, so this is a possibility. I'll probably do what I said in the comment above on my question, not handle the error responses in my wrapper, but pass them to my controller as in some cases, the API not returning what I need isn't the end of the world. – Arietta

Answer 3

I am also working on same issue, i preferred a callback based architecture where my Client class controls the flow of requests. Currently I am doing sleep and check algorithm. I works for me as i have 3 seconds of cool down time.

I use Cache to hold the count of fired requests.

while(($count = Cache::get($this->getCacheKey(),0)) >= 40){ // Max request
    sleep(1);
}
Cache::set($this->getCacheKey(), ++$count);
// fire request

function getCacheKey(){
    return floor(time()/10); // Cool down time
}

Queueing seems to be a better options, and I will eventually move to that. There are few things you need to keep in mind before putting queue in between.

1. Callback based Architecture, because you may need to save a serialise state of code in queue. Callback based design will give control all control to Client Class. You will not have to worry about throttling in you code.
2. Serialization could be tricky, try __sleep and __wakeup.
3. You may also want to prioritise few call, you can allocate a quota from clients for such calls.

Constantin

Answer 4

Personally I think Guzzle should not handle this case, but if you want Guzzle handle it I would write a Middleware which checks the response and if it returns a rate limit error (eg. status code 429). Then you can either emit a custom error or wait until the rate limit is over and try again. However this could possibly end up in long response time (since you wait for the rate limit).

I don't think the Laravel queue would be any better since that would make the response asynchronously available and you would have to poll your database or your cache, wherever you store the results. (Of course it can work if you don't need the results to be immediately available)

If this third party service is directly connected to a user facing interface, I would probably apply the same rate limit (in your application code) and return an error message to the user instead of waiting and autoresolving the issue.

Jemima