GetTwitter stops fetching data from Twitter after few hours

Question

I used GetTwitter processor in Apache Nifi to process live tweets using UserId. Initially it works fine but after few hours no tweets is retrieved from Twitter. May I know why I am getting this issue.

Note: Since I am using 25000 user id to fetch live tweets from them. As I referred some blogs it shows 5000 UserId is maximum limit for GetTwitter processor. I used 5 GetTwitter processor with same access credentias to divide the 25000 user ids to 5 parts.

Dithyrambic

Answer 1

It sounds like you are encountering a rate limiting issue. Twitter's documentation on it can be found at https://developer.twitter.com/en/docs/basics/rate-limiting

Ahead

Can confirm that if you are pulling from an API like twitter with Nifi, and it suddenly stops working...it is always the API squeezing you off. – Wilsonwilt