pip3 installs inside virtual environment with python3.6 failing due to ssl module not available

Question

(py36venv) vagrant@pvagrant-dev-vm:/vagrant/venvs$ pip3 install pep8

pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.

Collecting pep8 Could not fetch URL https://pypi.python.org/simple/pep8/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping

Could not find a version that satisfies the requirement pep8 (from versions: ) No matching distribution found for pep8

Background information - Trying to move to python 3.6.

Installed python3.6 using the below commands:

wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tgz

tar -xvf Python-3.6.0.tgz

cd Python-3.6.0
./configure --enable-optimizations
make -j8 sudo make altinstall python3.6

Created virtualenv by:

python3.6 -m venv py36venv

source py36venv/bin/activate

Tried to install pep8

(py36venv) pip3 install pep8

pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available. Collecting pep8

Could not fetch URL https://pypi.python.org/simple/pep8/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the

SSL module is not available. - skipping Could not find a version that satisfies the requirement pep8 (from versions: ) No matching distribution found for pep8

Wits

Do you have libssl-dev installed, does the Makefile point to the right ssl, is it enabled in the makefile? Also see: #5937837 – Jephum

Uncommented the following - **SSL=/usr/local/ssl _ssl _ssl.c \ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \ -L$(SSL)/lib -lssl -lcrypto ** – Wits

If you figured out a solution feel free to answer your own question. – Jephum

Answer 1

I followed the below steps for python3.6 installation in ubuntu 14.04 and virtualenv pip installs works fine.

Python 3.6 Installation:

sudo apt-get install python3-dev libffi-dev libssl-dev
wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tgz  
tar xvf Python-3.6.0.tgz
cd Python-3.6.0
./configure --enable-optimizations  
make -j8  
sudo make altinstall
python3.6

If seeing the following error --

zipimport.ZipImportError: can't decompress data; zlib not available make: *** [altinstall] Error 1

try:

sudo apt-get install zlib1g-dev

Validation:

Create virtualenv in python3.6:

python3.6 -m venv testenv
source testenv/bin/activate
pip install pep8

using pip:

(testenv) vagrant@pvagrant-dev-vm:~$ pip install pep8
*Collecting pep8
  Downloading pep8-1.7.0-py2.py3-none-any.whl (41kB)
    100% |████████████████████████████████| 51kB 4.1MB/s
Installing collected packages: pep8
Successfully installed pep8-1.7.0*

(testenv) vagrant@pvagrant-dev-vm:~$ pip list

pep8 (1.7.0) 
pip (9.0.1)
setuptools (28.8.0)

Wits

What does -j8 in make -j8 do? – Gabbey

-j [jobs], --jobs[=jobs] Specifies the number of jobs (commands) to run simultaneously. If there is more than one -j option, the last one is effective. If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously. Hope its helpful, for more information, please refer man pages on make – Wits

Note: It takes quite a long time to compile and install (depending on your system of course). It took ~20 minutes for me, running an AMD FX(tm)-9590 Eight-Core Processor, but it worked. – Addlebrained

Are you saying that if you compile with libssl-dev absent, that you need to reinstall? – Xanthophyll

@surfer190 Correct you are! I forgot to install libssl-dev and ran into the error described in the OP. After install libssl-dev, re-compiling, and re-installing, pip works without issue. – Addlebrained

Is libffi-dev actually needed? I seem to have built and installed without it. – Ruggles

Answer 2

I stumbled upon the same issue when I tried to create a virtual environment utilising python3.6.0. Here is my solution for Mac OS X 10.12.2 (Py_minion comment was pretty close):

Setup

I created the environment by the following steps: downloading python3.6.0 running

./configure --prefix=<some_path>`
make
make install
mkvirtualenv --python=<some_path/bin/python3.6> foo

So basically similar to: https://mcmap.net/q/20464/-use-different-python-version-with-virtualenv

An indication if you have the same issue as I had would be a similar line when running make

The necessary bits to build these optional modules were not found: _ssl

Solution

Install openssl

brew install openssl
brew unlink openssl && brew link openssl --force

Change Module/Setup or Module/Setup.dist

You can find those files in the directory of the downloaded Python version.

Comment in and, if necessary change, lines 209 - 211 (I had to change the SSL variable to my openssl location).

SSL=/usr/local/opt/openssl <---- THIS DEPENDS ON YOUR INSTALLATION
_ssl _ssl.c \
    -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
    -L$(SSL)/lib -lssl -lcrypto

Given that this was the location of openssl

Set environment variables

export CFLAGS="-I$(brew --prefix openssl)/include"
export LDFLAGS="-L$(brew --prefix openssl)/lib"

make and install again

Running

./configure --prefix=<some_path>`
make
make install
mkvirtualenv --python=<some_path/bin/python3.6> foo

again did the trick for me

Shogunate

For those that don't have sudo and brew: openssl can be acquired from openssl.org – Gabbey

Answer 3

Running make reported to me in the shell output:

The necessary bits to build these optional modules were not found:
_bz2                  _dbm                  _gdbm              
_sqlite3              _ssl                  _tkinter           
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

What solved the problem in my case (Linux Mint 18.1, openssl already installed) was editing the setup.py in the Python-3.6.0 folder adding there the path to where the openssl installation put the ssl.h file on my system into ( /usr/include/openssl/ssl.h ). Here the section in which I have added the line '/usr/include':

    # Detect SSL support for the socket module (via _ssl)
    search_for_ssl_incs_in = [
                          '/usr/local/ssl/include/',
                          '/usr/contrib/ssl/include/',
                          '/usr/include/'
                         ]
    ssl_incs = find_file('openssl/ssl.h', inc_dirs,
                         search_for_ssl_incs_in
                         )

Culpable

So happy I found your answer, it fixed this issue for me (Ubuntu 16.04) – Hypercritical

This works on my Ubuntu 16.04. Should be the most upvoted answer. – Gelatinoid

Answer 4

I have solved this problem on Ubuntu-16.04.1. First you need to install necessary libraries. To install open Terminal (Ctrl+Alt+T), then type;

 sudo apt-get install build-essential checkinstall
 sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev

After that go the directory where your python file is then reconfigure and reinstall python3.6 .

 cd /opt/Python3.6/
./configure
 make 
sudo make install

NOTE
If you installed Python3.6 via ppa, then reinstall it again;

sudo apt-get install python3.6

Now you should be able to use pip3.6

Egidio

Worked like a charm on debian (I installed python3.6 from source). Thank you. – Penney

Answer 5

I ran into the same error when building Python 3.6.1 from source under CentOS 7. For CentOS7, I had to first:

sudo yum install openssl-dev

Then:

./configure --enable-optimizations
make altinstall

Now pip3.6 works :-)

Locale

Same configuration :) – Daphne

WORKS, but for RHEL 7.3 it's sudo yum install openssl-devel – Eutherian

Answer 6

A complete script can be found HERE

Install Prerequisites

For RHEL/CentOS

sudo yum -y install gcc gcc-c++ zlib zlib-devel libffi-devel openssl-devel wget

For Ubuntu/Debian

sudo apt-get -y install build-essential python-dev python-setuptools python-pip python-smbus libncursesw5-dev libgdbm-dev libc6-dev zlib1g-dev libsqlite3-dev tk-dev libssl-dev openssl libffi-dev wget

Download Python

Modify for the version of python you want

Python Versions

cd /var/tmp sudo wget https://www.python.org/ftp/python/3.x.x/Python-x.x.x.tgz sudo tar xf Python-3.*.tgz cd Python-3*

Configure/Make/Install

sudo ./configure --enable-optimizations --enable-shared --prefix=/usr/local sudo make && make altinstall

Cleanup Shared Library & Add to Path

Stripping the shared library of debugging symbols can speed up execution when running parallel scripts.

sudo make && make altinstall sudo strip /usr/local/lib/libpython3.7m.so.1.0 sudo echo 'export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib' >> /etc/profile.d/python.sh sudo echo 'export PATH=${PATH}:~/usr/local/bin/' >> /etc/profile.d/python.sh sudo echo '/usr/local/lib' >> /etc/ld.so.conf sudo ldconfig

Reference

• Gist
• Stack Overflow
• Stack Overflow
• Daniel Erikson
• Unix StackExchange
• TLDP

Decoupage