SSL: CERTIFICATE_VERIFY_FAILED with urllib

Question

I'm running into trouble with the module urllib (Python 3.6). Every time I use the module, I get a page's worth of errors.

what's wrong with urllib and how to fix it?

import urllib.request
url='https://www.goodreads.com/quotes/tag/artificial-intelligence'
u1 = urllib.request.urlopen(url)
print(u1)

That block of code likes to spit out this mouthful of stuff:

  Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/a-nguyen/Downloads/EzSorses/APAsauce.py", line 3, in <module>
    u1 = urllib.request.urlopen(url)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>

Seems like something's off with the module itself.

Remorse

Possible duplicate of urllib and "SSL: CERTIFICATE_VERIFY_FAILED" Error – Sarita

How much do you want to bet someone will suggest disabling the cert checking? Well, if someone does, don't do that ;) – Shwa

Answer 1

There is command line program that you can run on MacOsX that will install the certificates:

sudo /Applications/Python\ 3.6/Install\ Certificates.command

The sudo command will prompt for you password to elevate your privileges.

Iranian

Answer 2

a dirty (not safe in terms of security) but fast hack like this do the job for me:

import ssl

ssl._create_default_https_context = ssl._create_unverified_context

Alane

This is extremely dangerous as it can unknowingly impact all downstream users. It should never ever be done. At least create and use an unverified context instead as urlopen(url, context=ssl.SSLContext() instead. – Conversation

Answer 3

I had the same issue. My python version is 3.6.5 and I'm on Mac OSX. When I tried to connect to the HTTPS sites I started getting the

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)>

exception. So, it was so obvious to me that the python on my machine has no certificates. I just ran the file from the path /Applications/Python 3.6 with a file name Install Certificates.command. This solved my issue. I reffered to this post.

Posticous

This resolves the problem on Python 3.6. See the associated issue here: bugs.python.org/issue28150 – Revolving

where is this on linux rhel? – Anthropopathy

Answer 4

To fix the problem, open your bash and write:

cd /Applications/Python 3.6

then:

sudo Certificates.command

Dietsche

this didn't work for me, but this answer did https://mcmap.net/q/49223/-brew-installation-of-python-3-6-1-ssl-certificate_verify_failed-certificate-verify-failed – Benthos

And for me that answer did: https://mcmap.net/q/49223/-brew-installation-of-python-3-6-1-ssl-certificate_verify_failed-certificate-verify-failed - and not because I ran the code, but because I found out about the SSL_CERT_FILE variable, by running import ssl; print(ssl.get_default_verify_paths()) – Mahican