Does Yahoo finance have data request upper limit. Is there an alternative or workaround?

Question

Hi there I have scanned this site and Google for any latest updates on downloading stock info from Yahoo Finance. I have not had much success. As I am testing my app, I came across an 404 server not found exception using the following C# code snippet:

string urlTemplate =
                 @"http://ichart.finance.yahoo.com/table.csv?s=[symbol]&a=" +
                   "[startMonth]&b=[startDay]&c=[startYear]&d=[endMonth]&e=" +
                      "[endDay]&f=[endYear]&g=d&ignore=.csv";
...
 WebClient wc = new WebClient();
            try
            {
                history = wc.DownloadString(urlTemplate);
            }

Do you know if Yahoo recently (or always) had some update on how much much you can request from Yahoo's Finance server? If so, does anyone know the upper limit or threshold? Is it over time or max number of daily requests? I thought about putting a random sleep request of up to 2 minutes to get by this. I don't think that would help. Is there any alternatives that would enable me to constantly make requests to Yahoo Finance? I thought Yahoo had some kind of subscription service you could use for this exact purpose. I cannot find anything about it. If none of this is a no go with Yahoo, does anyone have any recommendations of affordable alternative services or data feed services?

Santossantosdumont

How often the Yahoo page is updated? You can cache these values locally in a file or data base, storing the date time these data was get. Then, if it's bigger than x minutes, you get the page again and update your local values. – Diomedes

good answers here: #9347082 – Padnag

Answer 1

See the Yahoo Query Language Usage Information and Limits page. This is for all of the YQL APIs, not just the Finance API.

YQL Rate Limits:

What this means:

• Using the Public API (without authentication), you are limited to 2,000 requests per hour per IP (or up to a total of 48,000 requests a day).
• Using the Private API (with OAuth authentication using an API key), you are limited to 20,000 requests per hour per IP and you are limited to 100,000 requests per day per API Key.

As for implementation, if you wish to use the Yahoo Finance API, then you must adhere to their requirement. Build a system that only makes the number of requests they allow to prevent errors. You can do this with a database (in case there is more than one script/application client using it) and use threads/jobs (jobs=polling) to wait until it can make more requests. Get an API key to extend your limit. Get multiple API keys to extend it even more.

As for an alternative, there are too many to count. I've only used the Yahoo and Google APIs. Yahoo is my preferred option, but I haven't found a free alternative that is comparable. Try your luck and let me know if you find one!

Gimlet

Answer 2

They never said what the limit was. I used to update something like 6000 stocks daily, it worked. On some other pages the limit seems to be much higher, but on historic stock prices they do block at some point.

By the way, sometimes yahoo server returns undocumented HTTP 999 code which I interpret as "too many requests".

If all you want is end of day data then perhaps you can try this: http://www.eoddata.com/products/default.aspx (I've never tried it myself)

Of course there are paid subscriptions for more frequent data.

Glaring