tweepy get tweets between two dates

Question

I have the following code in Python:

import tweepy

consumer_key = "..."
consumer_secret = "..."

access_token = "..."
access_token_secret = "..."

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

start_date = datetime.datetime(2018, 1, 19, 12, 00, 00)
end_date = datetime.datetime(2018, 1, 19, 13, 00, 00)

api = tweepy.API(auth)

for tweet in tweepy.Cursor(api.user_timeline, screen_name="@IBM", since=start_date, until=end_date).items():
    print("ID TWEET: " + str(tweet.id))

Is there a way to get tweets between start_date and end_date, by modifying the cursor with tweepy?

I have already tried to use the since= and until= parameters, but they have not worked.

Thank you in advance.

Atiana

until has a time limit: (...) the search index has a 7-day limit. In other words, no tweets will be found for a date older than one week. developer.twitter.com/en/docs/tweets/search/api-reference/… – Alcantara

may be this should help... https://stackoverflow.com/questions/26205102/making-very-specific-time-requests-to-the-second-on-twitter-api-using-python or this one https://gist.github.com/alexdeloy/fdb36ad251f70855d5d6 – Oxygenate

Answer 1

First of all the Twitter API does not allow to search by time. Trivially, what you can do is fetching tweets and looking at their timestamps afterwards in Python, but that is highly inefficient.

You can do that by the following code snippet.

consumerKey = "CONSUMER_KEY"
consumerSecret = "CONSUMER_SECRET"
accessToken = "ACCESS_TOKEN"
accessTokenSecret = "ACCESS_TOKEN_SECRET"

auth = tweepy.OAuthHandler(consumerKey, consumerSecret)
auth.set_access_token(accessToken, accessTokenSecret)

api = tweepy.API(auth)

username = sys.argv[1]
startDate = datetime.datetime(2011, 6, 1, 0, 0, 0)
endDate =   datetime.datetime(2012, 1, 1, 0, 0, 0)

tweets = []
tmpTweets = api.user_timeline(username)
for tweet in tmpTweets:
    if tweet.created_at < endDate and tweet.created_at > startDate:
        tweets.append(tweet)

while (tmpTweets[-1].created_at > startDate):
    tmpTweets = api.user_timeline(username, max_id = tmpTweets[-1].id)
    for tweet in tmpTweets:
        if tweet.created_at < endDate and tweet.created_at > startDate:
            tweets.append(tweet)

Although highly inefficient. It works, can helped me in creating my own bot.

Unperforated

Thank you all for your replies! – Atiana

This worked for me too. Note, that I had to import datetime which is not shown in the example code. – Maori

how to make this to search for a hashtag and get tweets between start_date and end_date. – Scenography

I think you can't. You can only download tweets from a given hashtag through search something like this: twapi.search(q=query, count=100, since_id=since_id, max_id=str(last_id - 1), tweet_mode='extended') for the past 10 days. The only workaround for that is you know the tweet_id which you can download it regardless of time-limit set by Twitter API. – Lecce

Answer 2

I've just used until (optional operator) and it seems to work pretty well. I used it like this:

tweets = tw.Cursor(api.search,
                   q=search_words,
                   lang="en",
                   since=date_since,
                   until=date_until,
                   result_type="recent"
                   ).items(2)

Tillion

since was removed as a search parameter for api.search in version 3.8.but until is still there : Returns tweets created before the given date.... Date should be formatted as YYYY- MM-DD – Mahican

Including just start date is giving me data for only the current system date. When I try to include end date also, I get the error, " local variable 'csvFile' referenced before assignment" – Polysepalous

Answer 3

Inspired by @papaya answer here, this works for me, for multiple hashtags query

startDate = utc.localize(startDate) 
endDate = utc.localize(endDate)   

tweets = []
tmpTweets = api.search_tweets('hashtags and filteration')

for tweet in tmpTweets:
    if tweet.created_at < endDate and tweet.created_at > startDate:
        tweets.append(tweet)

while (tmpTweets[-1].created_at > startDate):
    tmpTweets = api.search_tweets(new_search, max_id = tmpTweets[-1].id)
    for tweet in tmpTweets:
        if tweet.created_at < endDate and tweet.created_at > startDate:
            tweets.append(tweet)

Pawnshop