Python: using Google Finance to download index data

Question

I've been successful at downloading stock data from Google Finance, like so:

import pandas as pd
from pandas_datareader import data as web   
import datetime
start = datetime.datetime(2016,1,1)
end   = datetime.date.today()
apple = web.DataReader('aapl', 'google', start, end)

I thought I'd be able to use the same framework for index data. But this doesn't work:

spx = web.DataReader('INDEXSP', 'google', start, end)

RemoteDataError: Unable to read URL: http://www.google.com/finance/historical

Does Google not support this for indices?

Or do I need a different protocol?

Pig

I was also struggling with this. Eventually, I just went to look for a download of the data in csv format and then import it into python using pd.read_csv – Pelotas

Answer 1

Index data is available:

given you call has named an <instrument> that the Google API was not ready to map onto it's historical records, try to find the proper <instrument> name manually first.

S&P 500 INDEX ( INDEXCBOE:SPX )
v/s
S&P 500 ( INDEXSP:.INX )
...
DAX PERFORMANCE-INDEX ( INDEXDB:DAX )

---

Both work and serve data on Google Finance side:

Pontificals

That doesn't work either. web.DataReader('INDEXCBOE:SPX', 'google', start, end) still returns the same error. – Pig

In that case, the trouble is not with the Google Finance Backend processing, but rather with the .DataReader() implementation, as Google provides data, QED in the click-through link above. If the current .DataReader() implementation does some <instrument> name mangling, try as the second proposal above, using the "aapl" -> "(NASDAQ:AAPL)" analogy, where .DataReader() translated a ticker symbol into a fully-qualified instrument name. Test an <instrument> named as just ".INX" that ought be properly translated to INDEXSP:.INX. If not, .DataReader() implementation fails. – Pontificals

To be clear, neither INDEXCBOE:SPX nor INDEXSP:.INX work. – Pig

Ok, the trouble remains in the .DataReader() implementation. Q.E.D. Thanks for sharing your experience. – Pontificals

Not so sure about this @user3666197. Just confirming that the data is displayed doesn't mean it is available as csv. The path should be finance.google.com/finance/…, but that link is broken. You can confirm it works for the etf (SPY): finance.google.com/finance/historical?q=SPY&startdate=2017/… – Erse

Did not get your point with a broken link objection. Data was properly shown on screen by visitng URL: finance.google.com/finance/historical?q=INDEXCBOE:SPX – Pontificals

Sorry to be unclear--my point is that that link is not where DataReader pulls form. It pulls from the CSV export link, which is different from what you posted in your most recent comment. If you try the two CSV export links in my comment, you'll see that the index link is nonexistent, the ETF link works. – Erse

Also, for what it's worth, the pull request to change to correct date format has been rejected several times – Erse

So, as noted half a year ago -- the trouble remains in the .DataReader() implementation. Q.E.D. An unfulfilled promise from a .DataReader() is simply still an unfulfilled promise, irrespective of the implementation troubles, that caused that .DataReader() promise to load data let remain not fulfilled. Thanks for sharing your experience. – Pontificals

Agree to disagree. SPX data wouldn't be available whether or not that pull request was accepted – Erse

Answer 2

For DAX you can use 'NASDAQ:DAX' which downloads from google with the datareader. However, this ETF starts only from 2014-10-23.

Haemostat

Answer 3

This is an issue on Google's side. Compare the historical prices page for the S&P to that for Google and you'll see that the latter has a link to "download to spreadsheet" while the former does not. pandas-datareader simply goes off of this csv link.

So to your comment, I would not consider this a broken implementation within pandas-datareader, just one that won't work for cases where Google Finance doesn't provide that .csv.

Erse