Our application uploads hundreds of videos per day using the YouTube API, but now getting quota exceeded errors

Question

My application allows users to "link" their YouTube accounts to our system and then we allow them to upload videos to their channels both automatically in some cases, and on an "upload" action on their part. We have hundreds of videos being uploaded because we have thousands of users that use our system.

Today I saw hundreds of errors in my application when our batch automatic upload job was running. The errors were for:

quotaExceeded, video upload limit reached.

My API quotas are very high (50,000,000 per day, 3,000,000 per 100 seconds, 300,000 per 100 seconds per user), so if a video is 1600 points, I have enough limit to upload 30,000+ per day, 1,800 videos per 100 seconds, and 180+ per 100 seconds per user.

I have seen other questions out there hinting at some newly enforced limit by Google to just 50 videos and then 1 video every 15 minutes? This is a very low limit for my application which has such a large quota! To be clear, at most I have seen my application handle 1,000 videos in a single day (1,600,000 points of my total 50,000,000 quota).

Is there any way I can get this artificial limit of 50 videos and then 1 per 15 minutes removed? This is a major block to my users' functionality for a very popular web app. I could understand if it were 50 videos per user and then 1 every 15 minutes, but I highly doubt my errors this morning were from one user trying to upload 300+ videos at a time. My system only uploads their "newest" videos each day, which most people only have 1-10 videos at max. Hundreds would be a very rare edge case.

Contemplative

Might be relevant? I suspect you'll have to log a case with tech support, unless there is some general knowledge in the YT developer community about the (new?) quota rules. – Faceless

@halfer, how do I log a case with Google support on this issue? Their website for the "support" section of the API is what brought me here! – Contemplative

I don't know I'm afraid, I am not a Google user. This is probably also relevant. There is an answer below though, did that help? – Faceless

@halfer, the answer below did not help. I wish I could ping that Richard guy from the other question. I can't comment on other posts yet, though... I at least need clarification on whether or not this issue is on a USER only basis, as in, per channel 50 videos per day; or a global API user basis, as in, 50 uploads to even 50 different users/channels still causes the limit to be reached? – Contemplative

I'll ping him for you. – Faceless

That user has added an answer though, which is not particularly conclusive - I am not sure if any more detail could be added? – Faceless

Our Google liaison has now escalated our quota issue to a 'Product Expert' but is not hopeful of a resolution. Even though they applied an 'increased upload allowance' to the channel, it is still stuck at a hard 50 limit with 1 every 15 mins from there. I'll let you know more when they get back. Still inconclusive I'm afraid – Breathing

Thank you @RichRoss. Could you clarify whether this 50 upload limit is a Per Channel basis and not a per API account basis? My application is authorizing as the end user to upload to thousands of different channels and I want to make sure the errors I saw were only affecting power users trying to upload more than 50 to their individual channel and not affecting all the other users. Thanks! – Contemplative

Yes it is definitely per channel. Your API quota for the Google project associated to your app will define the total usage across all of your users using different channels. This is the quota you have to fill out the application form to have increased from 1M 'credits'. Our problem is that our app is a single user uploading to 1 channel. In other news, the error message returned by YT has changed today so maybe they are up to something (HttpError 410 when requesting googleapis.com/upload/youtube/v3/… returned "Backend Error") – Breathing

Answer 1

You can actually confirm in your Developers Console the available quota for your application. By default, YouTube API have a quota allocation of 1 million units per day as mentioned in Quota usage. If you see that your usage reached your quota limit, you can request additional quota on the Quotas tab.

Note also that, all API requests, including invalid requests, incur a quota cost of at least one point. You may use the Quota Calculator to get an estimate of the quota cost for an API query.

On the other hand, to work efficiently with your quota and if you haven't done so, I suggest that you implement exponential backoff if you're encountering high error ratio. See this sample code which shows an exponential backoff strategy to resume a failed upload. Also, if applicable, subscribe to Push Notifications which is much more efficient than polling-based solutions.

Check the documentation for more information on how PubSubHubbub callback server receives Atom feed notifications when a channel does any of the following activities:

• uploads a video
• updates a video's title
• updates a video's description

Hope that helps!

Autolysin

I am awareof how to see my quota, as I've described in detail in my original question how much quota I have, as well as how much I've used on the busiest day of my application. I'm not worried about backoff strategies or any other options to handle uploads. I'm only concerned about the apparently new requirement of only 50 videos per day, then 1 per every 15 minutes for the rest of the day?! That's ludicrous for a very popular application such as mine. – Contemplative