Windows: Python SSL certificate verify failed

Question

I've installed Anaconda and had SSL problems when trying to do API calls via Jupyter Notebooks:

import requests
import certifi

r = requests.get('https://github.com/')
print(r)

This first produced a SSL connection error. Which I could solve after extensive search and the help of our IT department. The solution here was to add the company root certificate to certifi cert storage.

Now for other requests unfortunately I still have the same problems. Example code calling the Google Analytics API with google2pandas package:

from google2pandas import *

query = {
    'reportRequests': [{
        'viewId' : 37616054,

        'dateRanges': [{
            'startDate' : '8daysAgo',
            'endDate'   : 'today'}],

        'dimensions' : [
            {'name' : 'ga:date'}, 
            {'name' : 'ga:pagePath'},
            {'name' : 'ga:browser'}],

        'metrics'   : [
            {'expression' : 'ga:pageviews'}],

        'dimensionFilterClauses' : [{
            'operator' : 'AND',
            'filters'  : [
                {'dimensionName' : 'ga:browser',
                 'operator' : 'REGEXP',
                 'expressions' : ['Firefox']},

                {'dimensionName' : 'ga:pagePath',
                 'operator' : 'REGEXP',
                 'expressions' : ['iPhone']}]
        }]
    }]
}

# Assume we have placed our client_secrets_v4.json file in the current
# working directory.

conn = GoogleAnalyticsQueryV4(secrets='Analytics.json')
df = conn.execute_query(query)

Here I still get the SSL error I had on the simple call before as well:

C:\ProgramData\Anaconda3\lib\ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session) 848 # non-blocking 849 raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets") --> 850 self.do_handshake() 851 except (OSError, ValueError): 852 self.close()

C:\ProgramData\Anaconda3\lib\ssl.py in do_handshake(self, block)
1106 if timeout == 0.0 and block: 1107
self.settimeout(None) -> 1108 self._sslobj.do_handshake() 1109 finally: 1110 self.settimeout(timeout)

SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)

I believe there is another library in use, that doesn't rely on certifi? But I don't have any idea on where and how to add my root certificate, so all iPython requests will work.

Any ideas are appreciated.

Actinium

When you call the GoogleAnalytics API, as a part of SSL Handshake, it will be presenting it's certificate. You need to have the issuer certificate of the certificate presented for the Handshake to pass. So, you need to obtain it and add it to your store. – Khalilahkhalin

Hi @Jay, thanks for your reply. Unfortunately I don't fully understand the answer. I did add the company root certificate to certifi. Do I need to add it to a second store to get the handshake to verify? I don't have much knowledge of SSL in general. Outside of our company network the same requests from Python libraries work without any issues. – Actinium

Answer 1

One possible solution is to instruct Python to use your Windows Certificate Store instead of the built-in store in the certifi package. You can do that by installing:

pip install pip-system-certs

Python in then using the same certificates as your browsers do.

Possibility

FYI >>> this might be outdated as indicated here: https://mcmap.net/q/245672/-python-requests-with-wincertstore – Acarology

I tried this on Windows, Python 3.10, with Scoop, and it broke my installation of pip. Now when I use pip install I get TypeError: can only concatenate str (not "method") to str. BEWARE of this solution. – Brigidabrigit

Elaboration on my previous comment: I just found that ANY command in pip now raises the error above. So running this broke my pip. – Brigidabrigit

Apparently pip-system-certs (by the same author is the solution to replace python-certifi-win32, but I haven't tried it. – Thermionics

This worked for me for python v3.9 and windows 10 – Peroration

Answer 2

I added the CA certificate of my company in a configuration file used by requests library. You can check the path of this file by:

import requests as r
print(r.certs.where())

The path of the cacert.pem that python uses shall be printed, edit it and append the CA certificate to the bottom of it.

Payment

Can you please tell bottom of where? i have the .pem file from print(r.certs.where()) , and used r = requests.get("mycompany.com/home", cert="C:/.../site-packages/certifi/cacert.pem") . Still got the same error like above – Cleo

Open the .pem file you have found with a text editor, scroll to the bottom of the file and paste there your certificate – Payment

This Worked, thanks a lot! It looks like it's also possible to overwrite certifi.core.where function to make it use your company certificate file and avoid editing (I guess better for distributing the solution) see details here https://mcmap.net/q/394377/-python-requests-can-39-t-find-a-folder-with-a-certificate-when-converted-to-exe – Bully

Answer 3

The above solution is not very suitable due to security reasons. A better solution for this problem is:

• Go to https://letsencrypt.org/certificates/ and download "ISRG Root X1", "ISRG Root X2", "Let’s Encrypt R3" (.der files)
• Import the 3 certificates in your local Trusted Root Certification Authorities, follow this guide for that https://www.sonicwall.com/support/knowledge-base/how-can-i-import-certificates-into-the-ms-windows-local-machine-certificate-store/170504615105398/
• You may have to restart your system

Jacquard

"The above solution" is an ambiguous reference. Either link to a specific answer or remove that, please. – Azotize

Answer 4

You can monkey-patch the __init__ method of ssl.SSLSocket so that it always ignores SSL certificate verification by forcing the cert_reqs=CERT_NONE parameter.

Add this to the beginning of your script:

import ssl
orig_sslsocket_init = ssl.SSLSocket.__init__
ssl.SSLSocket.__init__ = lambda *args, cert_reqs=ssl.CERT_NONE, **kwargs: orig_sslsocket_init(*args, cert_reqs=ssl.CERT_NONE, **kwargs)

Saintebeuve

Thanks, but this would mean no more safety via SSL, right? – Actinium

Right. With this option, the connection will still be encrypted via SSL, but the certificate will not be verified. Use this method if you can't manage to install a proper root certificate. – Saintebeuve

Answer 5

httplib2 is the package that caused the problems. It has its own cacerts.txt, where I added the root certificate as well now.

Actinium

Answer 6

On every "request.SOMETHING", add in:

verify=False 

as a argument

for example

 re=requests.get(str(url)+"/"+str(f), headers=headers)

becomes

 re=requests.get(str(url)+"/"+str(f), headers=headers, verify=False)

Acus

Answer 7

Zscaler was causing similar issue on my Windows machine. I finally figured out that setting SSL_CERT_FILE at the beginning of my Python file helps: os.environ['SSL_CERT_FILE']='C:\\.certs\\mycert.pem'

Fields