Error using httlib's HTTPSConnection with PKCS#12 certificate

Question

I'm trying to use httplib's HTTPSConnection for client validation, using a PKCS #12 certificate. I know the certificate is good, as I can connect to the server using it in MSIE and Firefox.

Here's my connect function (the certificate includes the private key). I've pared it down to just the basics:

def connect(self, cert_file, host, usrname, passwd):
    self.cert_file = cert_file
    self.host = host

    self.conn = httplib.HTTPSConnection(host=self.host, port=self.port, key_file=cert_file, cert_file=cert_file)

    self.conn.putrequest('GET', 'pathnet/,DanaInfo=200.222.1.1+')
    self.conn.endheaders()
    retCreateCon = self.conn.getresponse()

    if is_verbose:
        print "Create HTTPS connection, " + retCreateCon.read()

(Note: No comments on the hard-coded path, please - I'm trying to get this to work first; I'll make it pretty afterwards. The hard-coded path is correct, as I connect to it in MSIE and Firefox. I changed the IP address for the post.)

When I try to run this using a PKCS#12 certificate (a .pfx file), I get back what appears to be an openSSL error. Here is the entire error traceback:

  File "Usinghttplib_Test.py", line 175, in 
    t.connect(cert_file=opts["-keys"], host=host_name, usrname=opts["-username"], passwd=opts["-password"])
  File "Usinghttplib_Test.py", line 40, in connect
    self.conn.endheaders()
  File "c:\python26\lib\httplib.py", line 904, in endheaders
    self._send_output()
  File "c:\python26\lib\httplib.py", line 776, in _send_output
    self.send(msg)
  File "c:\python26\lib\httplib.py", line 735, in send
    self.connect()
  File "c:\python26\lib\httplib.py", line 1112, in connect
    self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
  File "c:\python26\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "c:\python26\lib\ssl.py", line 113, in __init__
    cert_reqs, ssl_version, ca_certs) ssl.SSLError: [Errno 336265225] _ssl.c:337: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib

Notice, the openSSL error (the last entry in the list) notes "PEM lib", which I found odd, since I'm not trying to use a PEM certificate.

For kicks, I converted the PKCS#12 cert to a PEM cert, and ran the same code using that. In that case, I received no error, I was prompted to enter the PEM pass phrase, and the code did attempt to reach the server. (I received the response "The service is not available. Please try again later.", but I believe that would be because the server does not accept the PEM cert. I can't connect in Firefox to the server using the PEM cert either.)

Is httplib's HTTPSConnection supposed to support PCKS#12 certificates? (That is, pfx files.) If so, why does it look like openSSL is trying to load it inside the PEM lib? Am I doing this all wrong?

Any advice is welcome.

EDIT: The certificate file contains both the certificate and the private key, which is why I'm providing the same file name for both the HTTPSConnection's key_file and cert_file parameters.

Sirloin

Answer 1

On the openSSL mailing list, I chatted with Mounir Idrassi. He noted that openSSL does support PKCS#12 files, and - based on the error message I'm receiving - it appears that httplib is calling the wrong function to load the key.

In his words:

Concerning the error you are getting, it appears that the phython module you are using is calling SSL_CTX_use_PrivateKey_file by giving it the PKCS#12 file name. This is does not because SSL_CTX_use_PrivateKey_file only accepts two formats : SSL_FILETYPE_PEM and SSL_FILETYPE_ASN1.

(I'm giving httplib the PKCS#12 file name as key file, because this file format includes both the cert and the private key in the same file.)

In order to correct this, you have two solutions : - Either feed the python module with the private key in a PEM file. - Or modify the source code of this python module in order to use the PKCS#12 functions I mentioned above to extract the private key as an EVP_PKEY and then call SSL_use_PrivateKey instead of SSL_CTX_use_PrivateKey_file, along with SSL_use_certificate for setting the associated certificate.

(I tried the former and wasn't able to get it to work. Doesn't necessarily mean it won't work; only that I wasn't able to.)

Sirloin

Answer 2

This is no surprise. The Python library reference docs are pretty clear about this. From http://docs.python.org/library/httplib.html:

class httplib.HTTPSConnection(host[, port[, key_file[, cert_file[, strict[, timeout[, source_address]]]]]])

A subclass of HTTPConnection that uses SSL for communication with secure servers. Default port is 443. key_file is the name of a PEM formatted file that contains your private key. cert_file is a PEM formatted certificate chain file.

Bearcat

+1 for RTFM. Where were you last year when I was working on this? :) – Sirloin

Hi Remi, I was a bit slow off the mark ;-) - I'm fairly new to SO. Just came across your post searching on a similar error string. Turns out I had just mangled argument order somewhere along the call chain and never passed the key path into httplib. – Bearcat