Twitter Streaming API limits?

Question

I understand the Twitter REST API has strict request limits (few hundred times per 15 minutes), and that the streaming API is sometimes better for retrieving live data.

My question is, what exactly are the streaming API limits? Twitter references a percentage on their docs, but not a specific amount. Any insight is greatly appreciated.

What I'm trying to do:

• Simple page for me to view the latest tweet (& date / time it was posted) from ~1000 twitter users. It seems I would rapidly hit the limit using the REST API, so would the streaming API be required for this application?

Dulsea

Answer 1

You should be fine using the Streaming API, unless those ~1000 users combined are tweeting more than (very) roughly 60 tweets per second at any moment.

Using the Streaming API endpoint statuses/filter with the follow parameter, you are allowed up to 5000 users. There is no rate limit except when the stream returns more than about 1% of the all tweets being tweeted at that moment. (60 tweets per second is 1% of the average rate of tweets, which is always fluctuating, so don't rely on that number.)

If your stream does go above the 1% threshold, you can detect this. (See the LIMIT notice.) Then you would use the REST API to find missed tweets.

Allanallana

Upvote for the hint on limit to find out about amount of missed tweets due to the threshold. – Drafty

Ok cool this makes sense, thank you. So if I needed to monitor 5001 twitter users, I would hit a limit there as well? – Dulsea

hey @Jonas, do you have any source/documentation you can link for this? – Bridlewise

Hi @Bridlewise Twitter does not publish this. It's easy to find statistics by googling, nor is it difficult to measure this yourself. – Allanallana

@Jonas, I would like to know that if I am using Twitter Streaming API to know about tweets related to a keyword, then for how long it can last – Circumlocution

@Circumlocution Here is Twitter's documentation for streaming developer.twitter.com/en/docs/tweets/filter-realtime/…. Use the track parameter. I don't understand what you are referring to by "how long it can last." – Allanallana

@Allanallana sorry for not being clear, I mean that if I put my script on a cloud server like digital ocean, then without crashing for how many days it can run and store those keyword related tweets. – Circumlocution

@Circumlocution That depends on how you write your code. There are a variety of error conditions that you must code for. Read this to get an idea geduldig.github.io/TwitterAPI/faulttolerance.html – Allanallana

@Allanallana Thank you very much for your answer! But I am wondering if the 1% rule still holds for today. And if I use the streaming API to get the tweets posted in a predefined bounding box, could I get all the tweets posted in that bounding box if the 1% limit is not reached? – Seamanlike

@BrightChang The exact percentage that represents the "small random sample" is not published, but the percentage probably has not changed. Yes, you can use the bounding box to get all tweets provided the stream does not reach the limit. – Allanallana

Hey @Jonas, sorry for digging on an old thread. I understand that the streaming API would only hit the limit when returning about 60 tweets a sec - which also is a rough number. Can I assume that I'd be fine if I stream tweets from like 2-3 profiles, publishing frequency is way less than 60/sec and that too with filtering retweets? I'm pretty sure I'd be but just wanted to confirm :) appreciate any suggestion... – Turgite

Sounds okay to me. If I understand you... you will have 3 streams, each authenticated with different credentials. – Allanallana

appreciate the response. Is that possible to have one stream (filter) and get it to pull tweets from like 2 or 3 users? @Allanallana – Turgite

Use the follow parameter described here developer.twitter.com/en/docs/twitter-api/v1/tweets/… – Allanallana

Here is an interesting observation on dev portal, THIS is a snapshot of filter stream endpoint docs. It says to right, rate limit = 50 per 15 min. What does that mean? (btw I think i created a project which is on v2 endpoints). not sure if that makes a difference. @Allanallana – Turgite

Each stream connection is one request. You can find similar docs for v2 endpoints on developer.twitter.com. Start there. – Allanallana

ahhh so it means upto 50 streams in a 15 min window - that's pretty good. Also the link to start here didn't render but no worries. I appreciate the help :) – Turgite

That was not a link. A link is developer.twitter.com – Allanallana

Answer 2

Twitter simply will not allow multiple streams from one registered app/account. Doing so will result in the older one being closed.

Also too many connection tries are not allowed as well and will result in a user being blocked.

Reference docs: Public Streaming API (outdated)

Drafty

I think a single stream would work fine in this situation. – Allanallana

That's right, a single public stream is the way to go in this case. I was referring to the op asking for the restrictions of the streaming api. – Drafty

Thanks - good to know about the multiple connection tries. This means that I shouldn't connect to the streaming API feed every time I refresh / open my page that displays the tweets, correct? I should have a standalone file continuously running that is maintaining a connection to the streaming API? – Dulsea

Exactly, you need to to prevent being blocked. – Drafty

The Public Streaming API link redirects to /docs now. The last version that had useful content dates from Sep 2017. – Morpho

Thanks for the hint, my answer is from Jan 2016, and tbh I haven't worked on projects like that since then, so, sorry when this is outdated, but it's hard to maintain your answers over several years. In general I think possibilities regarding usage of these APIs have shifted. – Drafty