Using client certificates with urllib2

Question

I need to create a secure channel between my server and a remote web service. I'll be using HTTPS with a client certificate. I'll also need to validate the certificate presented by the remote service.

1. How can I use my own client certificate with urllib2?

2. What will I need to do in my code to ensure that the remote certificate is correct?

Karat

Answer 1

Here's a bug in the official Python bugtracker that looks relevant, and has a proposed patch.

Got

Answer 2

Because alex's answer is a link, and the code on that page is poorly formatted, I'm just going to put this here for posterity:

import urllib2, httplib

class HTTPSClientAuthHandler(urllib2.HTTPSHandler):
    def __init__(self, key, cert):
        urllib2.HTTPSHandler.__init__(self)
        self.key = key
        self.cert = cert

    def https_open(self, req):
        # Rather than pass in a reference to a connection class, we pass in
        # a reference to a function which, for all intents and purposes,
        # will behave as a constructor
        return self.do_open(self.getConnection, req)

    def getConnection(self, host, timeout=300):
        return httplib.HTTPSConnection(host, key_file=self.key, cert_file=self.cert)

opener = urllib2.build_opener(HTTPSClientAuthHandler('/path/to/file.pem', '/path/to/file.pem.') )
response = opener.open("https://example.org")
print response.read()

Shock

Thank you so much for this! I've been tracking down a bug for days using a private certificate and certificate authority to connect to an Apache server using the requests library in python. I could verify that my certificate was working in the browser however it failed every time with a "handshake error" I ended up using the class above to prove that the client certificate was working and authenticating properly. Turned out there was a bug in the version of the requests library I was using 1.2.3 to be exact. Hopefully this comment helps other people who are encountering the same problem. – Clemenciaclemency

I am new to Python, so I would like to know- when is the https_open() function called here? If possible please describe what is the execution flow of this code? – Imposition

So I have followed these instructions and I am getting a urlopen error: "ssl certificate verify failed" when I use python 2.7.9 or greater. I believe that I need to add a SSLContext with my own cert chain, but am not 100% sure, do you mind giving any advice or hints on working that into your example here. Thanks! – Scarberry

Answer 3

Per Antoine Pitrou's response to the issue linked in Hank Gay's answer, this can be simplified somewhat (as of 2011) by using the included ssl library:

import ssl
import urllib.request

context = ssl.create_default_context()
context.load_cert_chain('/path/to/file.pem', '/path/to/file.key')
opener = urllib.request.build_opener(urllib.request.HTTPSHandler(context=context))
response = opener.open('https://example.org')
print(response.read())

(Python 3 code, but the ssl library is also available in Python 2).

The load_cert_chain function also accepts an optional password parameter, allowing the private key to be encrypted.

Tapir

Per docs.python.org/2.7/library/ssl.html the create_default_context function is introduced in 2.7.9, so this doesn't work in (admittedly very) old Python 2 versions. – Forecastle

Answer 4

check http://www.osmonov.com/2009/04/client-certificates-with-urllib2.html

Siberson

dead link, last archived content web.archive.org/web/20190302113828/http://www.osmonov.com/2009/… which is just what @Shock is showing us, above – Krona