How to Use SSLContext with Python Requests Library?

Question

The certifi library often runs into issues with domain certificates, but in the standard urllib.request library has no issue.

If I set the context to use the certifi's file, I get the SSL error,

import ssl
import certifi
import requests.urllib as urlrq

resp = urlrq.urlopen(url="https://<web address>/rest/info?f=json", 
                                  context=ssl.create_default_context(cafile=certifi.where()))

if I use the standard library and set the context as follows:

import ssl
import certifi
import requests.urllib as urlrq

resp = urlrq.urlopen(url="https://<web address>/rest/info?f=json", 
                                  context=ssl.SSLContext(ssl.PROTOCOL_TLSv1))

No SSL error.

How can I get requests to honor this SSLContext?

Mccarver

Answer 1

In order to make the requests library use a custom ssl context, you need to create a custom HTTPAdapter class and override the init_poolmanager method to pass in extra arguments to the base class's implementation.

See sample code here:

from requests import Session
from requests.adapters import HTTPAdapter
import ssl


class CustomHTTPAdapter(HTTPAdapter):

    def init_poolmanager(self, *args, **kwargs):
        # this creates a default context with secure default settings,
        # which enables server certficiate verification using the
        # system's default CA certificates
        context = ssl.create_default_context()

        # alternatively, you could create your own context manually
        # but this does NOT enable server certificate verification
        # context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)

        super().init_poolmanager(*args, **kwargs, ssl_context=context)


def main():
    client_session = Session()
    client_session.mount("https://", CustomHTTPAdapter())

    # now you can use the client_session to make requests
    # r = client_session.get("https://<web address>/rest/info?f=json")

Bergh

Answer 2

Creating an ssl.SSLContext() on its own doesn't enable certificate verification or load CA certificates by default. This is why you're not seeing SSL errors. Using ssl.create_ssl_context() does set verification by default.

So the issue here isn't with SSLContext or certifi, it's with the website's certificate and how you're constructing your SSLContext. Next step would be to look into why the certificate the website is presenting isn't valid.

Hamlin

is there a way to garnish more information from urllib3/requests on what is the cause of the error? – Mccarver

Exact error message from requests: (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'), – Mccarver