certificate verify failed: unable to get local issuer certificate

Question

I am trying to get data from the web using python. I imported urllib.request package for it but while executing, I get error:

certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)

• I am using Python 3.7 on Mac OS High Sierra.
• I am trying to get CSV file from: https://s3.amazonaws.com/assets.datacamp.com/production/course_1606/datasets/winequality-red.csv

When I changed the URL to 'http' - I am able to get data. But, I believe, this avoids checking SSL certificate.

So I checked on the internet and found one solution: Run /Applications/Python\ 3.7/Install\ Certificates.command

This solved my problem. But I have no knowledge on SSL and the likes. Can you help me understand what it actually did to solve my issue.

If possible, please recommend me any good resource to learn about the security and certificates. I am new to this.

Thanks!

Note: I did go through the link - openssl, python requests error: "certificate verify failed"

My question differs from the one in link because, I want to know what actually happens when I install certifi package or run Install\ Certificates.command to fix the error. I have a poor understanding of securities.

Dainedainty

Possible duplicate of openssl, python requests error: "certificate verify failed" – Lotte

@Lotte - I read from the link provided you. Just to clear (I don't know SSL and the likes): 1. Is OpenSSL library native to the OS I am using or Python uses its own? 2. When my code is trying get data from a particular website, it checks for the website's certificate in the OpenSSL root and as it doesn't trust it by default, it throws me the error. Am I right? 3. If so, then what happens when I run install Certificates.command? – Dainedainty

1. Yes, pyopenssl is a wrapper to this. 2. Didn't know if requests uses pyopenssl at all, but faild to verify the certificate which results throwing the error. 3. Didn't know Certificats.command, assume update/install the needed Certificates, requests needs. – Lotte

Check this #50236617 It's answered here – Kozhikode

Answer 1

For anyone who still wonders on how to fix this, i got mine by installing the "Install Certificates.command"

Here is how I did,

Just double click on that file wait for it to install and in my case, you will be ready to go

Babara

How exactly do you install it? With brew? Please explain – Unfeeling

It's a .command file, which means you should just be able to double-click it and it will run in the Terminal. – Speaking

It was very useful for me. I had same issue (macOS high Sierra + Python 3.7). Thank you! – Bookbindery

but where do you get this file? – Wethington

@JosephAstrahan it is the standard python installation package from www.python.org . Also this is the official python release (I usually install this instead of the one from homebrew) – Niobe

I had permission errors so I had to first run sudo pip install --upgrade certifi in the terminal, and then double-click Install Certificates.command – Villalba

With brew, this worked for me: security find-certificate -a -p > ~/all_mac_certs.pem; export SSL_CERT_FILE=~/all_mac_certs.pem; export REQUESTS_CA_BUNDLE=~/all_mac_certs.pem – Burmese

Worked for me. Using Python 3.9. – Velez

@Unfeeling to run terminal > cd /Applications/{Python3.x} then terminal > ./Install\ Certificates.command – Nutria

My Python 3.10 installation was done using Scoop, on Windows, and I can't find the file Install Certificates.command. – Discard

Answer 2

Creating a symlink from OS certificates to Python worked for me:

ln -s /etc/ssl/* /Library/Frameworks/Python.framework/Versions/3.9/etc/openssl

(I'm on macOS, using pyenv)

Maganmagana

I'm using Python 3.9.3 through brew, and for me the command was ln -s /etc/ssl /usr/local/etc/[email protected] – Hyman

It works on mac air M1. – Alevin

Worked on Macbook 13 Intel 2020 Ventura 13.2.0 – Bootee

Worked on MacBook Pro 16 Intel Ventura 13.2.1 – Sidekick

worked for me, just make sure the version is correct ln -s /etc/ssl/* /Library/Frameworks/Python.framework/Versions/{python_version_you_have_installed}/etc/openssl – Accroach

Answer 3

I hit the same issue on OSX, while my code was totally fine on Linux, and you gave the answer in your question!

After inspecting the file you pointed to /Applications/Python 3.7/Install Certificates.command, it turned out that what this command replaces the root certificates of the default Python installation with the ones shipped through the certifi package.

certifi is a set of root certificates. Each SSL certificate relies a chain of trust: you trust one specific certificate because you trust the parent of that certificate, for which you trust the parent, etc. At some point, there is no "parent" and those are "root" certificates. For those, there is no other solution than bundling commonly trusted root certificates (usually big trust companies like eg. "DigiCert").

You can for instance see the root certificates in your browser security settings (for instance for Firefox->Preference->Privacy and security->view certificates->Authorities).

Coming back to the initial problem, and prior to running the .command file, executing this returns for me an empty list on a clean installation:

import os
import ssl                                        
openssl_dir, openssl_cafile = os.path.split(      
    ssl.get_default_verify_paths().openssl_cafile)
# no content in this folder
os.listdir(openssl_dir)
# non existent file
print(os.path.exists(os.path.join(openssl_dir, openssl_cafile)))

This means that there is no default certificate authority for the Python installation on OSX. A possible default is exactly the one provided by the certifi package.

After that, you just can create an SSL context that has the proper default as the following (certifi.where() gives the location of a certificate authority):

import platform
# ...

ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS)
ssl_context.verify_mode = ssl.CERT_REQUIRED
ssl_context.check_hostname = True
ssl_context.load_default_certs()

if platform.system().lower() == 'darwin':
    import certifi
    ssl_context.load_verify_locations(
        cafile=os.path.relpath(certifi.where()),
        capath=None,
        cadata=None)

and make request to an url from python like this:

import urllib
# previous context
https_handler = urllib.request.HTTPSHandler(context=ssl_context)

opener = urllib.request.build_opener(https_handler)
ret = opener.open(url, timeout=2)

Niobe

Answer 4

For those who this problem persists: - Python 3.6 (some other versions too?) on MacOS comes with its own private copy of OpenSSL. That means the trust certificates in the system are no longer used as defaults by the Python ssl module. To fix that, you need to install a certifi package in your system.

You may try to do it in two ways:

1) Via PIP:

pip install --upgrade certifi

2) If it doesn't work, try to run a Cerificates.command that comes bundled with Python 3.6 for Mac:

open /Applications/Python\ 3.6/Install\ Certificates.command

One way or another, you should now have certificates installed, and Python should be able to connect via HTTPS without any issues.

Hope this helped.

Hospice

After so many attempts and suggestions from various sources, #2 worked for me! Thanks so much! – Munniks

This is the actual fix, without having to adjust your code. Thanks! Works on M1 Macbook Pro with macOS Ventura – Zouave

I had the same error while connecting to SMTP (from Mac). The most simple solution that I have seen in this thread. – Bluma

Still working in 2024 with python3.12 – Audieaudience

Had this problem in Pycharm on MacOS. Had to restart the PC after this for it to work – Constitutional

Answer 5

This worked in all OS:

import ssl
import certifi
from urllib.request import urlopen

request = "https://example.com"
urlopen(request, context=ssl.create_default_context(cafile=certifi.where()))

Leland

What is request defined as? – Bussell

Copy-pasted this into my python 3.10 installation on MacOS 13.0.1 and got the same error I've started seeing recently: urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)> – Comanchean

Answer 6

Environment: Mac, Python 3.10, iTerm,

1. Search in Finder: Install Certificates.command
   
2. Get Info
3. Open with: iTerm.app
   
4. double click 'Install Certificates.command'

Waiting for install the certificates. Solve it.

Unrighteous

On thing worked for me. – Thinner

Thanks so much, finally an answer that doesn't involve copying cryptic commands – Aviator

Answer 7

As the question don't have the tag [macos] I'm posting a solution for the same problem under ubuntu :

sudo apt install ca-certificates
sudo update-ca-certificates --fresh
export SSL_CERT_DIR=/etc/ssl/certs

Solution come form Zomatree on Github.

Estabrook

This is the answer for Debian. Thanks – Fibula

Answer 8

This worked for me on Mac OS:

ln -s /etc/ssl/* /Library/Frameworks/Python.framework/Versions/Current/etc/openssl/

Natalee

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Koerlin

ran that exact command and it worked for me! thanks – Brat

Answer 9

I would like to provide a reference. I use cmd + space, then type Install Certificates.command, and then press Enter. After a short while, the command line interface pops up to start the installation.

 -- removing any existing file or link
 -- creating symlink to certifi certificate bundle
 -- setting permissions
 -- update complete

Finally, it fixes the errors.

Trudey

Answer 10

Suddenly I started facing this issue in my windows environment. To aggravate, it was showing up when I ran pip as well, so the issue was not with the remote server certificate.

After trying many different things, I've found the solution combining bit and pieces from multiple answers:

• Add trusted hosts to pip.ini: pip config set global.trusted-host "pypi.org files.pythonhosted.org pypi.python.org" (doesn't work only passing as pip install parameter)

• Update system certificates: pip install pip-system-certs (doesn't work installing python-certifi-win32)

Now https requests are working again \o/

Gumbo

My company uses Zscaler and this was all it took. Thanks Orez. – Pacifa

My company uses Zscaler and this failed to fix the issue. I'm still getting ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129). Also I get a permission denied when I execute /Applications/Python\ 3.7/Install\ Certificates.command :( – Kendo

Second option worked for me that uses Zscaler but now Zscaler is blocking me. At least 1 problem is now fixed. Thanks – Mychal

These two commands saved me in my Powershell/Windows environment. – Purify

Answer 11

Certifi provides Mozilla’s carefully curated collection of Root Certificates for validating the trustworthiness of SSL certificates while verifying the identity of TLS hosts. It has been extracted from the Requests project.

pip install certifi

Or running the program code below:

# install_certifi.py
#
# sample script to install or update a set of default Root Certificates
# for the ssl module.  Uses the certificates provided by the certifi package:
#       https://pypi.python.org/pypi/certifi

import os
import os.path
import ssl
import stat
import subprocess
import sys

STAT_0o775 = ( stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
             | stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP
             | stat.S_IROTH |                stat.S_IXOTH )


def main():
    openssl_dir, openssl_cafile = os.path.split(
        ssl.get_default_verify_paths().openssl_cafile)

    print(" -- pip install --upgrade certifi")
    subprocess.check_call([sys.executable,
        "-E", "-s", "-m", "pip", "install", "--upgrade", "certifi"])

    import certifi

    # change working directory to the default SSL directory
    os.chdir(openssl_dir)
    relpath_to_certifi_cafile = os.path.relpath(certifi.where())
    print(" -- removing any existing file or link")
    try:
        os.remove(openssl_cafile)
    except FileNotFoundError:
        pass
    print(" -- creating symlink to certifi certificate bundle")
    os.symlink(relpath_to_certifi_cafile, openssl_cafile)
    print(" -- setting permissions")
    os.chmod(openssl_cafile, STAT_0o775)
    print(" -- update complete")

if __name__ == '__main__':
    main()

Brew has not run the Install Certificates.command that comes in the Python3 bundle for Mac.

Judaism

Answer 12

The cause for this error in my case was that OPENSSLDIR was set to a path which did not contain the actual certificates, possibly caused by some upgrading / reinstallation.

To verify this if this might be the case for you, try running:

openssl s_client -CApath /etc/ssl/certs/ -connect some-domain.com:443

If you remove the -CApath /etc/ssl/certs/ and get a 20 error code, then this is the likely cause. You can also check what the OPENSSLDIR is set to by running openssl version -a.

Since changing the OPENSSLDIR requires re-compilation, I found the easiest solution to be just creating a symlink in the existing path: ln -s /etc/ssl/certs your-openssldir/certs

Balough

I get error_20 with one version of openssl in one machine, but not the others. The -CApath thing is irrelevant – Olibanum

Could be that the two versions of openssl each look in different CA paths? – Balough

Turns out the system’s OpenSSL certs were old, and installing OpenSSL from source doesn’t bring new certs. An os upgrade solved it (it was a supercomputer with centos 7 on all nodes) – Olibanum

Answer 13

Paste the following code at the start:

# paste this at the start of code
import ssl 

try:
    _create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
    pass
else:
    ssl._create_default_https_context = _create_unverified_https_context

Skipjack

Am I correct in assuming, this avoids checking the SSL certrificate's validity? – Hellebore

This skips the certificate validation. – Rie

Don't do this! "My house key doesn't work! Just leave the door unlocked all the time." – Greenfinch

Answer 14

I had the error with conda on linux. My solution was simple.

conda install -c conda-forge certifi

I had to use the conda forge since the default certifi appears to have problems.

Ketubim

Answer 15

This page is the top google hit for "certificate verify failed: unable to get local issuer certificate", so while this doesn't directly answer the original question, below is a fix for a problem with the same symptom. I ran into this while trying to add TLS to an xmlrpc service. This requires use of the fairly low-level ssl.SSLContext class. The error indicates that a certificate is missing. The fix was to do several things when constructing SSLContext objects:

First, in the client:

def get_client():
    context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    # Load the default certs:
    context.load_default_certs()

    # Optionally, install the intermediate certs.
    # This _should_ be handled by the server, but
    # maybe helpful in some cases.
    # context.load_verify_locations('path/to/ca_bundle.crt')
    return xmlrpc.client.ServerProxy('https://server.yourdomain.com/', context=context)

In the server, you need to install the intermediate certs in the context:

class SecureXMLRPCServer(socketserver.TCPServer, 
        xmlrpc.server.SimpleXMLRPCDispatcher):
    # https://gist.github.com/monstermunchkin/1100226
    allow_reuse_address = True

    def __init__(self, addr, certfile, keyfile=None,
            ca_bundle=None,
            requestHandler=xmlrpc.server.SimpleXMLRPCRequestHandler,
            logRequests=True, allow_none=False, encoding=None, 
            bind_and_activate=True, ssl_version=ssl.PROTOCOL_TLSv1_2):
        self.logRequests = logRequests

        # create an SSL context
        self.context = ssl.SSLContext(ssl_version)
        self.context.load_default_certs()

        # The server is the correct place to load the intermediate CA certificates:
        self.context.load_verify_locations(ca_bundle)
        self.context.load_cert_chain(certfile=certfile, keyfile=keyfile)

        xmlrpc.server.SimpleXMLRPCDispatcher.__init__(self, allow_none, 
                encoding)
        # call TCPServer constructor
        socketserver.TCPServer.__init__(self, addr, requestHandler, 
                bind_and_activate)

        if fcntl is not None and hasattr(fcntl, 'FD_CLOEXEC'):
            flags = fcntl.fcntl(self.fileno(), fcntl.F_GETFD)
            flags |= fcntl.FD_CLOEXEC
            fcntl.fcntl(self.fileno(), fcntl.F_SETFD, flags)

    def get_request(self):
        newsocket, fromaddr = self.socket.accept()
        # create an server-side SSL socket
        sslsocket = self.context.wrap_socket(newsocket, server_side=True)
        return sslsocket, fromaddr

Tess

Answer 16

Caveat: I am not super knowledgeable about certificates, but I think this is worth checking early.

Before spending any time reconfiguring your code/packages/system, make sure it isn't an issue with the server you are trying to download from.

I think the error can be misleading because "unable to get local issuer certificate" makes it seems like it's a problem with your local machine, but that may not necessarily be the case.

Try changing the page you are trying to load to something that is probably good, like https://www.google.com and see if the issue persists. Additionally, check the domain that's giving you problems against the search tool at https://www.digicert.com/help/.

In my case, DigiCert's tool told me that "The certificate is not signed by a trusted authority (checking against Mozilla's root store)." That would explain why I seemed to have the root certificates installed but still had the error. When I tested loading a different site with HTTPS, I had no issues.

If this case applies to you, then I think you probably have 3 logical options (in order of preference): 1) fix the server if it's under your control, 2) disable certificate checking while continuing to use HTTPS, 3) skip HTTPS and go to HTTP.

Chiastolite

I still don't understand this. Until a couple of days before my program worked just fine. Then suddenly out of the blue I get this error message. I am still not sure if the problem lies with myself or the site I am trying to reach. I somehow can get a response when sending a GET request to Google, but not to the (unrelated URLs) of two sites I try to reach... this is driving me nuts – Nickinickie

Answer 17

Experienced this on Windows and after struggling with this, I downloaded the chain of SSL Certificates for the webpage

Steps for this on Chrome - (the padlock on the top left -> tap "Connection is secure" -> tap "Certificate is valid") To view the certificate chain, select the Certification path. To download each certificate, view the certificate in "Certification Path" tab open the "details" tab then copy to file

Once downloaded, open where you save the certificates, then compile into one .PEM file

Use this as an example:

    openssl x509 -in inputfilename.cer -inform DER -outform PEM  >> .pem

The order of this matters, start with the lowest certificate in the chain otherwise your bundle will be invalid

Finally

    response = requests.get('enter/urll/here', verify ='/path/to/created bundle')

Inquiline

Answer 18

Simply run this:

pip install --trusted-host=pypi.org --trusted-host=files.pythonhosted.org --user pip-system-certs'

Diarmit

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Koerlin

This command legit does nothing – Sheepshanks

Answer 19

For me all the suggested solutions didn't work. However, I was running the code in a terminal from my companies' PC, which has an IT security software package installed called ZScaler. Disabling the ZScaler software solved all my issues.

Thorite

Same problem here, with Z-Scaler as well but when I turned it OFF, I still get the same error – Comanchean

I was facing the same issue - turning off ZScaler didn't help, but the issue was solved when I quit ZScaler and ran it again. – Grazia

Here are the official Zscaler recommendations community.zscaler.com/t/… – Jilli

Answer 20

I experienced this error after installing Python 3.11 with Homebrew, and also when installing with Pyenv (which was itself installed with Homebrew).

I resolved it by reinstalling Homebrew's ca-certificates and openssl packages:

brew reinstall ca-certificates openssl

Witherspoon

Answer 21

For me the problem was that I was setting REQUESTS_CA_BUNDLE in my .bash_profile

/Users/westonagreene/.bash_profile:
...
export REQUESTS_CA_BUNDLE=/usr/local/etc/openssl/cert.pem
...

Once I set REQUESTS_CA_BUNDLE to blank (i.e. removed from .bash_profile), requests worked again.

export REQUESTS_CA_BUNDLE=""

The problem only exhibited when executing python requests via a CLI (Command Line Interface). If I ran requests.get(URL, CERT) it resolved just fine.

Mac OS Catalina (10.15.6). Pyenv of 3.6.11. Error message I was getting: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)

This answer elsewhere: https://mcmap.net/q/50025/-scraping-ssl-certificate_verify_failed-error-for-http-en-wikipedia-org-duplicate

Disclamation

Answer 22

This might be not directly related to the issue experienced by the user, but this error also appears if your machine that sends a request is not allowed to communicate with that address at all, perhaps due to security rules of your company or internet provider. Please try to use "curl https://your-resuouce-name" and ensure you have read access before actually fixing the certificates.

Winou

Answer 23

I recently had this issue while connecting to MongoDB Atlas. I updated to the latest certifi python package and it works now.

(python 3.8, upgraded to certifi 2020.4.5.1, previously certifi version 2019.11.28)

Tung

Answer 24

I ran into this on Ventura with python 3.9-10, even though I had already tried this:

1. added my private CA certificates to /etc/ssl/cert.pem, /etc/ssl/certs/
2. added my private CA certificates to the certifi specific cert.pem file
3. added my private CA certificates to my keychain into the 'System' bucket
4. set REQUESTS_CA_BUNDLE=/etc/ssl/cert.pem

This made requests work, but HTTPSConnection and urllib3 failed validation, so it turns out there is yet a place to add CA certificates:

5. /usr/local/etc/ca-certificates/cert.pem

I believe this is because I have installed openssl via brew, and this sets up the above file, and adds a symlink from /usr/local/etc/[email protected]/cert.pem.

So if anyone experiences certificate validation failing after having installed openssl via brew, then this is likely the explanation.

Ela

Answer 25

I tried several of the above. What worked for me is this :

Add the python certs to your ~.zshrc if you don't have them there:

export PIP_DEFAULT_CERT=/etc/ssl/certs/ca-certs.pem

• Prevent SSL issues when running Python scripts that use the requests module.

export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certs.pem

Iredale

Answer 26

The only solution that worked for me was to set SSL_CERT_DIR environment variable in PyCharm's "Run/Debug configurations":

SSL_CERT_DIR=/etc/ssl/certs/

Whipcord

Answer 27

Another solution would be setting SSL Context:

import ssl

def get_ssl_context() -> ssl.SSLContext:
  return ssl.SSLContext()

And then using it while making a new request:

urllib.request.urlopen(request, context=get_ssl_context())

Whipcord

Answer 28

I had a similar situation with @Max which he presented here, so I thought that I might be of help if others will run into mine.

I had to install a newer version of python on a system, with SSL support which required a newer version of openssl than the one already installed. After installing openssl in a custom folder and looked at it's content I saw no cert.pem file.

I wanted to get the latest one and because I was using python already I used certifi:

pip install -U certifi

to get the latest version of it, then ran python and in the console:

>>> import certifi
>>> certifi.where()
'/home/.../site-packages/certifi/cacert.pem'

after a

cp /home/.../site-packages/certifi/cacert.pem /usr/local/openssl/cert.pem

everything went back to normal and the error was gone. (Update the paths to match yours.)

Musset

Answer 29

ln -s /etc/ssl/* /Library/Frameworks/Python.framework/Versions/Your Version/etc/openssl it works for me

Unsnap

This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review – Winona

Answer 30

I got the below permission error when running the Install Certificates.command

PermissionError: [Errno 13] Permission denied: '../../lib/python3.10/site-packages/certifi/cacert.pem' -> 'cert.pem'
logout

But, I solve this issue by adding ssl, hope it helps.

import ssl
ssl._create_default_https_context = ssl._create_unverified_context

Chuckhole

Note: this isn't "solving" the issue, it is just suppressing it. Disabling verification isn't a solution ... but a hack. And how does this relate to the question here in the first place? – Younts