pandas_datareader, ImportError: cannot import name 'urlencode'

Question

I am working fine with pandas_datareader, then today I installed below both yahoo finance from the below link trying to solve another issue.

No data fetched Web.DataReader Panda

pip install yfinance
pip install fix_yahoo_finance

After the above installtion, pandas_datareader cannot be used anymore. I googled it and I did add the below import, and pandas_datareader is still not working.

from urllib.parse import urlencode

Here is the error:

    from pandas_datareader import data
  File "C:\Users\yongn\Anaconda3\lib\site-packages\pandas_datareader\__init__.py", line 2, in <module>
    from .data import (
  File "C:\Users\yongn\Anaconda3\lib\site-packages\pandas_datareader\data.py", line 11, in <module>
    from pandas_datareader.av.forex import AVForexReader
  File "C:\Users\yongn\Anaconda3\lib\site-packages\pandas_datareader\av\__init__.py", line 6, in <module>
    from pandas_datareader.base import _BaseReader
  File "C:\Users\yongn\Anaconda3\lib\site-packages\pandas_datareader\base.py", line 7, in <module>
    from pandas.io.common import urlencode
ImportError: cannot import name 'urlencode'

Teresetereshkova

Answer 1

I encountered exactly the same error. I am using python anaconda 2020_07 version.

The solution is to use the latest pandas-datareader v0.9 from anaconda channel. If you use the pandas-datareader package from conda-forge which is using older version v0.8.1, you will encounter the error. This is the status as of 20Dec2020.

I ran the command below to install the latest pandas-datareader package.

conda install -c anaconda pandas-datareader

The error message disappeared and the problem has been fixed.

EDIT: If conda later downgrades pandas-datareader back to conda-forge older version, there's a fix. See https://mcmap.net/q/1321939/-prevent-conda-from-automatically-downgrading-python-package

Bolyard

Answer 2

ok, I solved this problem by upgrading pandas datareader

pip install pandas-datareader --upgrade

Teresetereshkova

Answer 3

The reason is that pandas removed urlencode from their library. Thus, with newer versions of pandas this will never work. Installing other libraries or upgrading will NOT address the issue.

https://github.com/pydata/pandas-datareader/pull/793/commits/558862104028dd7dbf5e845b3b6c5fcfc0d568e5

The fix is to use Python3's version of urlencode. Fortunately, Python3 seems to have a drop in replacement:

Replace this:

from pandas.io.common import urlencode

With:

from urllib.parse import urlencode

And use urlencode as usual

Default

I am having this exact same issue. I am also using yfinance with pandas-datareader. I also see the exact same error on "from pandas.io.common import urlencode" and i clicked on your link to github (which I must admit I am not very experienced with and struggle to really understand). I see on github "from pandas.io.common import urlencode" is red and a minus sign... and "from urllib.parse import urlencode" is green and a plus sign... I can deduct that it is saying the same thing you are saying... replace "x" with "y"... but how does one go about replacing x with y? – Altorelievo

Open your code in an coding editor/IDE and when you see X, use the cut feature to remove it and replace it with Y. Save then re-run. Not sure I see the problem beyond this. It's not related to git in any meaningful way. – Default

5 years later... still the same error, with pandas-datareader (0.8.1) the latest available at conda-forge, and anyway no mention of a fix in their history of revisions. This module doesn't seem very well maintained, or I'm missing something. – Enfield

Your missing something. The module has been removed because there is a duplicate outside of pandas that has a similar api. Try my solution above, and I think this will make more sense. – Default

Answer 4

The answer above is correct. I just wrote some code in that implements it:

import os

basePath = os.path.join(os.path.dirname(os.__file__),'site-packages','pandas_datareader','base.py')

# read base.py
with open(basePath, 'r') as f:
    lines = f.read()

find = 'from pandas.io.common import urlencode'

replace = """from urllib.parse import urlencode"""

# add new text
lines = lines.replace(find,replace)

# overwrite old 'basedatatypes.py'
with open(basePath, 'w') as f:   
    f.write(lines)

initPath = os.path.join(os.path.dirname(os.__file__),'site-packages','pandas_datareader','iex','__init__.py')
# read iex/__init__.py
with open(initPath, 'r') as f:
    lines = f.read()

# add new text
lines = lines.replace(find,replace)

# overwrite old 'basedatatypes.py'
with open(initPath, 'w') as f:   
    f.write(lines)

Corby