Ignore certificate validation with urllib3

Question

I'm using urllib3 against private services that have self signed certificates. Is there any way to have urllib3 ignore the certificate errors and make the request anyways?

import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001)
c.request('GET', '/')

When using the following:

import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001, cert_reqs='CERT_NONE')
c.request('GET', '/')

The following error is raised:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 67, in request
    **urlopen_kw)
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 80, in request_encode_url
    return self.urlopen(method, url, **urlopen_kw)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 415, in urlopen
    body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 267, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.3/http/client.py", line 1061, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.3/http/client.py", line 1099, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.3/http/client.py", line 1057, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.3/http/client.py", line 902, in _send_output
    self.send(msg)
  File "/usr/lib/python3.3/http/client.py", line 840, in send
    self.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 103, in connect
    match_hostname(self.sock.getpeercert(), self.host)
  File "/usr/lib/python3/dist-packages/urllib3/packages/ssl_match_hostname/__init__.py", line 32, in match_hostname
    raise ValueError("empty or no certificate")
ValueError: empty or no certificate

Using cURL I'm able to get the expected response from the service

$ curl -k https://10.0.3.168:9001/
Please read the documentation for API endpoints

Yogurt

Answer 1

Try following code:

import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001, cert_reqs='CERT_NONE',
                                assert_hostname=False)
c.request('GET', '/')

See Setting assert_hostname to False will disable SSL hostname verification

Chartism

Same error as above. Rather, it looks like I'm using an older version of urllib3, 1.5, as provided in Ubuntu. Even so that commit hasn't made it in to a "release" as of yet (last release was 1.6 which was 3 months ago, that merge was 2 months ago) – Yogurt

@MarcoCeppi I'll push a new release next week. We just had a couple of large outstanding issues resolved which I need to clean up before publishing. That said, you're welcome to use the master branch for now, it should be stable. – Avalos

@Avalos Thank you for that, unfortunately I'm bound by whatever is in the Ubuntu Archives so this won't affect me for quite some time. I've accepted this answer as the resolution, but I've ended up re-working our infrastructure to get around this problem. – Yogurt

@MarcoCeppi Sorry for the troubles. :) I'll try to speed it up for you. Also once the new version is out, assert_hostname is a parameter on the constructor too, iirc. So you won't need the extra line. – Avalos

If someone comes around: This also works like urllib3.PoolManager(cert_reqs='CERT_NONE') – Foxe

Answer 2

In this question I see many answers but, IMHO, too much unnecessary information that can lead to confusion.

Just add the cert_reqs='CERT_NONE' parameter

import urllib3
http = urllib3.PoolManager(cert_reqs='CERT_NONE')

Gothicism

Answer 3

I found the answer to my problem. The urllib3 documentation does not, in fact, completely explain how to suppress SSL certificate validation. What is missing is a reference to ssl.CERT_NONE.

My code has a boolean, ssl_verify, to indicate whether or not I want SSL validation. The code now looks like this:

import ssl
import urllib3

#
#
#
    if (ssl_verify):
        cert_reqs = ssl.CERT_REQUIRED
    else:
        cert_reqs = ssl.CERT_NONE
        urllib3.disable_warnings()

    http = urllib3.PoolManager(cert_reqs = cert_reqs)

    auth_url = f'https://{fmc_ip}/api/fmc_platform/v1/auth/generatetoken'
    type = {'Content-Type': 'application/json'}

    auth = urllib3.make_headers(basic_auth=f'{username}:{password}')
    headers = { **type, **auth }

    resp = http.request('POST',
                    auth_url,
                    headers=headers,
                    timeout=10.0)

Pianism

Answer 4

Try to instanciate your connection pool this way:

HTTPSConnectionPool(self.host, self.port, cert_reqs=ssl.CERT_NONE)

or this way:

HTTPSConnectionPool(self.host, self.port, cert_reqs='CERT_NONE')

Source: https://github.com/shazow/urllib3/blob/master/test/with_dummyserver/test_https.py

---

EDIT (after seeing your edit):

It looks like the remote host didn't send a certificate (is it possible?). This is the code (from urllib3) which raised an exception:

def match_hostname(cert, hostname):
    """Verify that *cert* (in decoded format as returned by
SSLSocket.getpeercert()) matches the *hostname*. RFC 2818 rules
are mostly followed, but IP addresses are not accepted for *hostname*.

CertificateError is raised on failure. On success, the function
returns nothing.
"""
    if not cert:
        raise ValueError("empty or no certificate")

So it looks like cert is empty, which means that self.sock.getpeercert() returned an empty string.

Levinson

That's really odd. It does, or should, send back a certificate. The services work via browsers. I mean, it's a self-signed certificate produced by the ssl-certs Debian package. – Yogurt