how to succesfully compile python 3.x

Question

Upon attempting to compile python 3.7 I hit Could not import runpy module:

jeremyr@b88:$ wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
....
jeremyr@b88:~/Python-3.7.3$ ./configure --enable-optimizations    
jeremyr@b88:~/Python-3.7.3$ make clean 
jeremyr@b88:~/Python-3.7.3$ make -j32 
.... 

gcc -pthread     -Xlinker -export-dynamic -o Programs/_testembed Programs/_testembed.o libpython3.7m.a -lcrypt -lpthread -ldl  -lutil   -lm  
./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
    echo "generate-posix-vars failed" ; \
    rm -f ./pybuilddir.txt ; \
    exit 1 ; \
fi
Could not import runpy module
Traceback (most recent call last):
  File "/home/jeremyr/Python-3.7.3/Lib/runpy.py", line 15, in <module>
    import importlib.util
  File "/home/jeremyr/Python-3.7.3/Lib/importlib/util.py", line 14, in <module>
    from contextlib import contextmanager
  File "/home/jeremyr/Python-3.7.3/Lib/contextlib.py", line 4, in <module>
    import _collections_abc
SystemError: <built-in function compile> returned NULL without setting an error
generate-posix-vars failed
Makefile:603: recipe for target 'pybuilddir.txt' failed
make[1]: *** [pybuilddir.txt] Error 1
make[1]: Leaving directory '/home/jeremyr/Python-3.7.3'
Makefile:531: recipe for target 'profile-opt' failed
make: *** [profile-opt] Error 2

jeremyr@88:~/Python-3.7.3$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 8.11 (jessie)
Release:    8.11
Codename:   jessie

jeremyr@88:~/Python-3.7.3$ gcc --version 
gcc (Debian 4.9.2-10+deb8u2) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

jeremyr@88:~/Python-3.7.3$ sudo apt upgrade gcc
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... gcc is already the newest version.

jeremyr@b88:~/Python-3.7.3$ echo $PYTHONPATH

Any advice on how to overcome this and install python3.7 appreciated.

Edit - the solution listed below seems to work for various other python versions, so I changed title to python 3.x from 3.7

Narcisanarcissism

did you ever find a solution for this? – Improvised

iirc when I took out the --enable-optimizations that runpy error was overcome, then there was one more error, and then success. – Narcisanarcissism

ahhh ok, I ended up figuring out our gcc was also out of date but I found an old copy of python3 on that system which seems to work with the aws-cli I was trying to get working. (our python2 couldn't handle the new signaturev4) – Improvised

is this with universe 11.3? – Declare

what's universe 11.3? – Narcisanarcissism

I think I'm in universe 1.0 but I suppose it depends on where you stand w.r.t. the anthropic principle. – Narcisanarcissism

Answer 1

It seems the enable-optimizations was the problem,

jeremyr@b88:~/Python-3.7.3$ ./configure   
jeremyr@b88:~/Python-3.7.3$ make clean 

takes care of it in my case.

Narcisanarcissism

Thanks, I just came across this when compiling Python 3.5.9. – Titivate

I just came across this issue in python3.8.5 & centos7 and gcc 4.8.3. This fix in my case! Thanks,@jeremy_ruthan – Glycerite

Same here - this worked (without enable-optimizations) to compile a local/standalone Python 3.8.3 – Christabella

Answer 2

In case others come across this question: I encountered the same problem on Centos 7. I also had --enable-optimizations but didn't want to remove that flag. Updating my build dependencies and then re-running solved the problem. To do that I ran:

sudo yum groupinstall "Development Tools" -y

In case the yum group is not available, you can also install the pacakges individually using:

sudo yum install bison byacc cscope ctags cvs diffstat doxygen flex gcc gcc-c++ gcc-gfortran gettext git indent intltool libtool patch patchutils rcs redhat-rpm-config rpm-build subversion swig systemtap

Willywilly

I think this is a very good answer too. I had to restart my (virtual) machine after the install, but it seems to work. And why not run optimized code? – Nutrient

But it is possible some other packages need to be added as well, as I still get a fail. – Nutrient

Answer 3

For whomever MicGer's answer didn't work and would like to retain --enable-optimizations, check your gcc version. The error was solved for me on gcc 8.3.0.

Pithos

For centos7 one must install devtoolset. #55345873 – Impend

Answer 4

For me, CentOS 7, compiling Python3.9, the fix was updating the gcc and related packages.

Melodimelodia

Answer 5

Removing --enable-optimizations didn't solved the issue here, neither upgrading GCC from gcc-7.5.0 to gcc-8.2.1 trying building Python 3.6.13 on opensuse-15.2.

Here for some reason the build is picking a local python installation, so before calling make I have updated the PATH env to include the build directory (were python file is generated), for instance: export PATH=$(pwd):$PATH

Uncaredfor

Answer 6

#
# build python on centos 7
#

set -e
set -x

hub=/rpm/packages

mkdir -p ${hub}

cd $hub

yum groupinstall "Development Tools" -y

#
# download and install python deps
#   with saving deb to keep folder
#
yumdownloader --downloadonly --resolve -y \
    curl openssl-devel make kernel-devel \
    cmake pkgconfig mesa-libGL \
    bzip2-devel libffi-devel xz-devel zlib-devel \
    ncurses-devel gdbm-devel nss-tools readline-devel sqlite-devel tk tk-devel glibc-devel

yum install -y *.rpm

#
# update gcc
#

yum install -y centos-release-scl
yum install -y devtoolset-9

source /opt/rh/devtoolset-9/enable

echo "Current GCC: $(gcc --version)"


cd /tmp

PYTHON='3.8.17'

mkdir .Python3.8
cd .Python3.8

curl -O https://www.python.org/ftp/python/${PYTHON}/Python-${PYTHON}.tar.xz
tar -xf Python-${PYTHON}.tar.xz

cd Python-${PYTHON}

./configure \
    --enable-optimizations \
    --enable-loadable-sqlite-extensions \
    --prefix=/usr/local \
    --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"

# build
make -j 4
# make links
make altinstall

# check
python3.8 -V

Kidd