Upgrade python without breaking yum

Question

I recently installed Python 2.7.3 on a CentOS machine by compiling from source. Python 2.7.3 is installed at /opt/python2.7 and when I installed it I just changed /usr/bin/python to point to the new version. This apparently is wrong though because when I did it it broke yum. I would get the following.

There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   No module named yum

Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:
2.7.3 (default, May 15 2012, 17:45:42) 
[GCC 4.4.4 20100726 (Red Hat 4.4.4-13)]

If you cannot solve this problem yourself, please go to 
the yum faq at:
  http://yum.baseurl.org/wiki/Faq

I changed /usr/bin/python to point back to the python 2.6.6 but now 2.6.6 is the default version of python. Any idea how to fix this?

Rounder

Answer 1

I have written a quick guide on how to install the latest versions of Python 2 and Python 3 on CentOS 6 and CentOS 7. It currently covers Python 2.7.13 and Python 3.6.0:

# Start by making sure your system is up-to-date:
yum update
# Compilers and related tools:
yum groupinstall -y "development tools"
# Libraries needed during compilation to enable all features of Python:
yum install -y zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel expat-devel
# If you are on a clean "minimal" install of CentOS you also need the wget tool:
yum install -y wget

The next steps depend on the version of Python you're installing.

For Python 2.7.14:

wget http://python.org/ftp/python/2.7.14/Python-2.7.14.tar.xz
tar xf Python-2.7.14.tar.xz
cd Python-2.7.14
./configure --prefix=/usr/local --enable-unicode=ucs4 --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"
make && make altinstall

# Strip the Python 2.7 binary:
strip /usr/local/lib/libpython2.7.so.1.0

For Python 3.6.3:

wget http://python.org/ftp/python/3.6.3/Python-3.6.3.tar.xz
tar xf Python-3.6.3.tar.xz
cd Python-3.6.3
./configure --prefix=/usr/local --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"
make && make altinstall

# Strip the Python 3.6 binary:
strip /usr/local/lib/libpython3.6m.so.1.0

To install Pip:

# First get the script:
wget https://bootstrap.pypa.io/get-pip.py

# Then execute it using Python 2.7 and/or Python 3.6:
python2.7 get-pip.py
python3.6 get-pip.py

# With pip installed you can now do things like this:
pip2.7 install [packagename]
pip2.7 install --upgrade [packagename]
pip2.7 uninstall [packagename]

You are not supposed to change the system version of Python because it will break the system (as you found out). Installing other versions works fine as long as you leave the original system version alone. This can be accomplished by using a custom prefix (for example /usr/local) when running configure, and using make altinstall (instead of the normal make install) when installing your build of Python.

Having multiple versions of Python available is usually not a big problem as long as you remember to type the full name including the version number (for example "python2.7" or "pip2.7"). If you do all your Python work from a virtualenv the versioning is handled for you, so make sure you install and use virtualenv!

Volkslied

very nice writup = thank you very much. I would assume that python3 would be a similar set of steps correct? no major differences right? – Pindling

I have now rewritten the tutorial linked in my answer so that it covers both Python 2.7.3 and Python 3.3.0 properly. I also updated the contents with better information and I expanded the section about virtualenv. – Volkslied

Thanks a lot for all the upvotes, they are very much appreciated! – Volkslied

I made an install script in bash that used this guide, thanks. Feel free to put it up on your website if you like it! gist.github.com/timss/5122008 – Micron

Brilliant, but you might want to update it to install 2.7.4 instead. – Volkslied

@DanielEriksson Thanks, I wasn't aware of the 2.7.4 release. Easy to switch version though, it's nto really that version specific. – Micron

redhat 6.4 bundles python27 in it as far as I used it last time... why py2.6 still even exists..? everybody migrates to python3 quite actively already. py2.6 is dead! – Sunburst

Yes, I'm aware. I really should update/rewrite the post. Thanks for the nudge! – Volkslied

Thank you for your guide. When I try /usr/local/bin/python3.3 and >>> import pymysql I have ImportError: No module named 'pymysql'. If I invoke python as $ python (2.6 version) it can import this module. I wonder why Python 3 does not see it... – Skewness

When you install a Python package it is installed for the specific version of Python used during installation. So you need to install pymysql for 3.3 also. – Volkslied

I followed some dude's hacked up instructions, but this probably would have been a better idea. (lecloud.net/post/61401763496/…) – Perfidious

I've updated the guide with the latest versions of Python (2.7.6 and 3.3.4), plus info about compiling as a shared library, plus Unicode UTF-32 support, plus more libraries, plus setuptools instead of distribute, plus virtualenv and pyvenv, ... – Volkslied

Just used your guide on RHEL6 and it worked flawlessly. Thank you very much Daniel! – Puttergill

I just updated the guide (and moved it to a new domain) with info about Python 2.7.13 and Python 3.6.0. I also simplified the instructions a bit and added info about using get-pip.py to install pip, setuptools and wheel. – Volkslied

@DanielEriksson i have used your article to install the python 2.7.13 along with tthe 2.6.6 in CentOS. But when i install pip it gives the error . python2.7 : command not found. any idea? – Blount

@adeel Do you get the error message when you try to run the get-pip.py script, or later when you try to use the newly installed pip? Are you sure you have the location where you installed python2.7 (/usr/local/bin if you followed the guide) in your PATH? – Volkslied

@DanielEriksson its been 20 days lol . anyways it was saying that python2.7 not found. – Blount

by banging my head for almost 5 days i switched to the digital ocean server. I called godaddy they say they have no support of django in centos vps but you can do it own yourseld – Blount

this is great, but how would you surpass yum install dependency issue for it python 27(alternatives) and `python27(dist-packages), when i just installed python27, and still persists on my amazonlinux(has happened on several old, when yum updating from a 2013 kernel ugh, i know. – Beget

I think if you spawn a clone of your broken instance using AWS EC2 then you can immediately enable /etc/yum.conf to cache rpm. Force some sort of install of python26 or python27 used by yum backup all rpm and bring them to broken instance. I think it should work but require some patience and to install each rpm in the correct order. What amazon is going to do when Python 2.7 is supposed to reach end of life on December 31 2019... They should allow yum to work with Python 3 sooner than later!! – Gopak

for Python 2.7.x installation, LDFLAGS="-Wl,-rpath /usr/local/lib" is very very important, It saved my time, thanks – Provolone

1) sudo yum groupinstall -y "development tools" has thrown following error Warning: Group development does not have any packages to install. Maybe run: yum groups mark install (see man yum) No packages in any requested group available to install or update 2) this is working yum -y groupinstall base "Development tools" --setopt=group_package_types=mandatory,default,optional – Allwein

/usr/bin/install: cannot create regular file ‘/usr/local/bin/python3.8’: Permission denied – Skyscraper

Answer 2

vim `which yum`
modify #/usr/bin/python to #/usr/bin/python2.4

Slake

can you explain your answer? – Receptor

This is the correct answer, rather than lot of article insist to change the default python version in the system.. that didn't works for me.. ! – Francyne

Let's break the system by replacing Python, and then fix it by breaking yum? Not good advice. – Vallery

It's not really breaking yum, but I agree it's a risky solution. The main issue is that it's not clear what else you might break other than yum, and you might not immediately remember to look at the python version upgrade as a possible cause of future problems. This answer also doesn't explain what else it's recommending to do at all. – Grigri

Had to do the same change on /usr/libexec/urlgrabber-ext-down, after that it worked – Dwight

Answer 3

Put /opt/python2.7/bin in your PATH environment variable in front of /usr/bin...or just get used to typing python2.7.

Nitrochloroform

$ python --version Python 2.7.3 $ sudo python --version Python 2.6.6 – Rounder

or ln -s /usr/local/python7.6 /usr/local/python76 also. – Beget

Answer 4

pythonz, an active fork of pythonbrew, makes this a breeze. You can install a version with:

# pythonz install 2.7.3

Then set up a symlink with:

# ln -s /usr/local/pythonz/pythons/CPython-2.7.3/bin/python2.7 /usr/local/bin/python2.7
# python2.7 --version
Python 2.7.3

Bluepoint

"pythonz requires Python 2.6, 2.7 or higher" :\ – Beecham

Answer 5

ln -s /usr/local/bin/python2.7 /usr/bin/python

Antevert

Answer 6

Alright so for me, the error being fixed is when there are different versions of python installed and yum can't find a certain .so file and throws an exception.
yum wants 2.7.5 according to the error.

which python gives me /usr/bin/python
python --version gives me 2.7.5

The fix for me was append /lib64 to the LD_LIBRARY_PATH environment variable. The relevant content is /lib64/python2.7 and /lib64/python3.6.

export LD_LIBRARY_PATH=/lib64:$LD_LIBRARY_PATH

Fixed the yum error for me with multiple python versions installed.

Regnal

Answer 7

Daniel's answer is probably the most ideal one as it doesn't involve changing OS files. However, I found myself in a situation where I needed a 3rd party program which invoked python by calling usr/bin/python, but required Python 2.7.16, while the default Python was 2.7.5. That meant I had to make usr/bin/python point to a Python version of 2.7.16 version, which meant that yum wouldn't work.

What I ended up doing is editing the file /usr/bin/yum and replacing the shebang there to use to the system default Python (in my case, that meant changing #! /usr/bin/python to #! /usr/bin/python2). However, after that running yum gave me an error:

ImportError: No module named urlgrabber.grabber

I solved that by replacing the shebang in /usr/libexec/urlgrabber-ext-down the same way as in /usr/bin/yum. I.e., #! /usr/bin/python to #! /usr/bin/python2. After that yum worked.

This is a hack and should be used with care. As mentioned in other comments, modifying OS files should be last resort only.

Wither

Answer 8

I recommend, instead, updating the path in the associated script(s) (such as /usr/bin/yum) to point at your previous Python as the interpreter.

Ideally, you want to upgrade yum and its associated scripts so that they are supported by the default Python installed.

If that is not possible, the above is entirely workable and tested.

Change:

#!/usr/bin/python

to whatever the path is of your old version until you can make the above yum improvement.

Cases where you couldn't do the above are if you have an isolated machine, don't have the time to upgrade rpm manually or can't connect temporarily or permanently to a standard yum repository.

Kanter

I would very much recommend to do it the other way round: Change the interpreter line in the new scripts to point to the new Python version. Don't mess with OS scripts. – Tiros

Interestingly, the other answer of @Slake up there got +6 for pretty much the same approach. – Ava

Answer 9

If you want to try out rpm packages, you can install binary packages based on the newest Fedora rpms, but recompiled for RHEL6/CentOS6/ScientificLinux-6 on:

http://www.jur-linux.org/download/el-updates/6/

best regards,

Florian La Roche

Betroth

Answer 10

I read a piece with a comment that states the following commands can be run now. I have not tested myself so be careful.

$ yum install -y epel-release
$ yum install -y python36

Fritillary