OpenAi api 429 rate limit error without reaching rate limit

Question

On occasion I'm getting a rate limit error without being over my rate limit. I'm using the text completions endpoint on the paid api which has a rate limit of 3,000 requests per minute. I am using at most 3-4 requests per minute.

Sometimes I will get the following error from the api:

• Status Code: 429 (Too Many Requests)
• Open Ai error type: server_error
• Open Ai error message: That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists.

Open ai documentation states that a 429 error indicates that you have exceeded your rate limit which clearly I have not. https://help.openai.com/en/articles/6891829-error-code-429-rate-limit-reached-for-requests

The weird thing is the open ai error message is not stating that. It is giving the response I usually get from a 503 error (service unavailable).

I'd love to hear some thoughts on this, any theories, or if anyone else has been experiencing this.

Williamswilliamsburg

Possibly related: OpenAI API giving error: 429 Too Many Requests – Coquette

Answer 1

You have to add credit balance to your account even if you want to use the API in free tier. Minimum credit you can deposit is $5. If you don't add the credits, you will get 429 rate limit exception.

To add credit, go to the billing page and then click on Add to credit balance.

Swaine

Answer 2

I have seen a few message on the OpenAI community forum with similar messages. I suggest checking out the error code guide we have with suggestion to mitigate these errors, in general though, it's possible the model itself was down and has nothing to do with your rate limit: https://platform.openai.com/docs/guides/error-codes

Overdye

I appreciate the reply, but I also feel a bit gaslit when I am following all described practices, am nowhere near the published rate limit on a paid account, and the API returns 429 indicating I need to change my calling behavior. I think OpenAI doesn't have the capacity on their side and should return an appropriate error. – Lampe

Answer 3

This error indicates the OpenAI servers have too many requests from all users and their servers have reached their capacity to service your request. It's pretty common at the moment.

Hopefully they will upgrade their servers soon. Not really sure why it is a big problem since they run on Azure and should be able to scale based on ramped up demand. Maybe they are just trying to minimise costs.

Fruitage

"Not really sure why it is a big problem since they run on Azure and should be able to scale based on ramped up demand." - the problem is GPUs. – Quinze

Answer 4

The solution for me was to add credit to the OpenAI API account, then wait a few minutes and it worked. More detail here, but tl;dr

1. Go to the API settings here: https://platform.openai.com/settings/organization/billing/overview
2. Click on 'Billing'
3. Click on 'Add to credit balance'.
4. Try again in a few minutes

Tito