Python SSL Certification Problems in Tensorflow

Question

I'm trying to download the MNIST data which is supposedly handled in:

tensorflow.examples.tutorials.mnist.input_data.read_data_sets()

As far as I'm aware read_data_sets sends a pull request to a server to download the (approx.) 1.5GB of data.

I keep getting this traceback error:

File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py",
line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)   File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)   File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
line 1026, in _send_output
    self.send(msg)   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
line 964, in send
    self.connect()   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
line 1400, in connect
    server_hostname=server_hostname)   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py",
line 401, in wrap_socket
    _context=self, _session=session)   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py",
line 808, in __init__
    self.do_handshake()   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py",
line 1061, in do_handshake
    self._sslobj.do_handshake()   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py",
line 683, in do_handshake
    self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)

It obviously has something to do with the SSL cert python uses... so I went to /Applications/Python 3.6/ and executed the "Install Certifactions.command" located there and got this error:

Traceback (most recent call last):   File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/shutil.py",
line 544, in move
    os.rename(src, real_dst) PermissionError: [Errno 13] Permission denied:
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/certifi-2015.04.28.dist-info/DESCRIPTION.rst'
-'/var/folders/mq/g_jy_1qx1vjdb3xmdh7y62y80000gn/T/pip-3m8ixnf5-uninstall/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/certifi-2015.04.28.dist-info/DESCRIPTION.rst'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):   File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/basecommand.py",
line 215, in main
    status = self.run(options, args)   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/commands/install.py",
line 342, in run
    prefix=options.prefix_path,   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/req/req_set.py",
line 778, in install
    requirement.uninstall(auto_confirm=True)   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/req/req_install.py",
line 754, in uninstall
    paths_to_remove.remove(auto_confirm)   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/req/req_uninstall.py",
line 115, in remove
    renames(path, new_path)   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/utils/__init__.py",
line 267, in renames
    shutil.move(old, new)   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/shutil.py",
line 559, in move
    os.unlink(src) PermissionError: [Errno 13] Permission denied: '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/certifi-2015.04.28.dist-info/DESCRIPTION.rst'

Is there something wrong with my filesystem permissions? I quite literally reinstalled OSX about a month ago on this computer...

Is there a way I can manually install the certs? Or issue the pull request to download the data to a non-https address?

Berenice

UPDATE: I was able to manually download the files from Yann Lecun's website and place them in the right directory. Program is working now, but would still like to get to the bottom of this. – Berenice

Answer 1

Install certificates, either double-clicking the file in /Applications/Python 3.6/Install Certificates.command (on macOS), or running this command from a bash terminal:

/Applications/Python\ 3.6/Install\ Certificates.command

And now the certificates are installed to download the data over HTTPS.

Source: dedicated issue on TensorFlow GitHub for official models.

Rumen

Answer 2

replacing https with http got it running

In contrib/learn/python/learn/datasets/mnist.py edit the line with SOURCE_URL='http://storage.googleapis.com/cvdf-datasets/mnist/'

Savadove

When I did that, I still got the error, but when I edited this file: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/keras/datasets/fashion_mnist.py and also deleted the s in https there, It worked. – Cly

This solution worked for me. In the case of a virtual environment, the location of the file is venv/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py. – Rumen