How can I get the position (lat, lon) of a foursquare checkin posted on twitter?

Question

1. I search with the twitter API all the public tweets with "4sq.com":

   http://search.twitter.com/search.json?q=4sq.com

so I get all the check-in on Foursquare that actually shared on Twitter.

{"created_at":"Sun, 08 Jan 2012 20:05:27 +0000","from_user":"fenavergara","from_user_id":50876086,"from_user_id_str":"50876086","from_user_name":"Fernanda Vergara Ch","geo":null,"id":156104246239051776,"id_str":"156104246239051776","iso_language_code":"lt","metadata":{"result_type":"recent"},"profile_image_url":"http://a2.twimg.com/profile_images/1708744785/image_normal.jpg","profile_image_url_https":"https://si0.twimg.com/profile_images/1708744785/image_normal.jpg","source":"<a href="http://foursquare.com" rel="nofollow">foursquare</a>","text":"I'm at Roca Roja (Antofagasta) http://t.co/DIKdYGwD","to_user":null,"to_user_id":null,"to_user_id_str":null,"to_user_name":null},{"created_at":"Sun, 08 Jan 2012 20:05:27 +0000","from_user":"Fabioland76","from_user_id":357326176,"from_user_id_str":"357326176","from_user_name":"Fabio Landini","geo":null,"id":156104246104821760,"id_str":"156104246104821760","iso_language_code":"no","metadata":{"result_type":"recent"},"profile_image_url":"http://a3.twimg.com/profile_images/1506958638/profiloJPG_normal.jpg","profile_image_url_https":"https://si0.twimg.com/profile_images/1506958638/profiloJPG_normal.jpg","source":"<a href="http://foursquare.com" rel="nofollow">foursquare</a>","text":"I'm at Burger King (Gallerie Cantoni, Legnano) http://t.co/rU1jIJ4A","to_user":null,"to_user_id":null,"to_user_id_str":null,"to_user_name":null}

With these tweets, Foursquare provides a link (like this one: http://t.co/DIKdYGwD) where I can see the position of the check-in on a map (if I display the link on a webpage).

How can I get the latitude and longitude programmaticly directly from the list of tweets?

Idea: Do I need to fetch the link on a webpage and analyse it to get the latitude and longitude?

EDIT: I don't want to use Twitter geocode

Scoot

Answer 1

You already have what you want, look at the "geo" section in sample of search output you posted. It's null because it is not filled (twitter user forget to enable twitter location in settings and all locations from foursquare are ignored by twitter).

You can filter twits by location, look at the Twitter Dev documentation.

UPD:

If you want to get location even in case when it is not stored to twit, you have to perform request to foursquare short link and parse venue information (you can steal location also). Look at the source of the sample 4sq.com page (small part):

...
options['lat'] = 42.98758759825106;
options['lng'] = -71.50742411613464;
options['fuzzy'] = false;
options['venue'] = {"id":"v4f08df964fc62d44946f058a","venue":{"id":"4f08df964fc62d44946f058a","name":"ABC News \u2013 WMUR Debate","contact":{"twitter":"MittRomney"},"location":{"address":"100 St Anselm Drive","lat":42.98758759825106,"lng":-71.50742411613464,"city":"Manchester","state":"NH"},"categories":[{"id":"4bf58dd8d48988d1af941735","name":"College Auditorium","pluralName":"College Auditoriums","shortName":"Auditorium","icon":{"prefix":"https:\/\/foursquare.com\/img\/categories\/education\/default_","sizes":[32,44,64,88,256],"name":".png"},"primary":true}],"verified":false,"stats":{"checkinsCount":7,"usersCount":7,"tipCount":1}}};
...

Unfortunately, Foursquare team can change the format of the page or limit requests from IP Address without any notes. That's why this solution is ugly.

UPD2

See answer from @MikeLewis, I've missed that Foursquare API supports requesting information by checking id.

Laser

I know..I don't want to geolocalize tweets but the check-in on Foursquare shared on Twitter. And Twitter geocode is not filled via Fousquare check-in. – Scoot

@Scoot You requests twits from Twitter server, if twit doesnt contain latitude and longitude you can get this information only from Foursquare server. From first look, you need to get venue ID from short URL and request venue information using Foursquare API – Laser

Thanks for your answer. If I understand well: I can parse the source code of the webpage to get the lat, lon. In this case, I can avoid the rate limit of 4sq. But Foursquare can change the code source format. Am I right? – Scoot

@Scoot Right, but I'm not sure if you can avoid rate limits of 4sq using this method (it is not documented and possibly can be changed in feature by foursquare team). – Laser

It's a public request from twitter so I can avoid that. Thanks for your help. – Scoot

If the geo information is not in Twitter, you'll need to request it form the foursquare API. You'll need to make authenticated requests, so you'll be maxed out at 500 requests/hour. – Fagoting

Answer 2

There's actually a much nicer solution here if you would like to use the foursquare API. One that doesn't depend upon a user having enabled Twitter's geo-tagging.

You noticed already that the link in the status update redirects to a foursquare page. Rather than screen scraping, you may notice there are two key pieces of information in the url. The checkin id, and a parameter called s (for signature) that lets you access that checkin via foursquare's API.

https://foursquare.com/dens/checkin/4f0b3676e4b0e8ed3a590e41?s=MWHM3-0BsLXIJprfbALvOM606ZE&ref=tw

For instance, in the above url, the checkin id is 4f0b3676e4b0e8ed3a590e41 while the signature is MWHM3-0BsLXIJprfbALvOM606ZE

Using the /checkins endpoint, you can get the full checkin details (including lat/long of the venue). Dox here: https://developer.foursquare.com/docs/checkins/checkins

Phytohormone