How to establish TLS session in python using PKCS11

Question

I’m trying to establish TLS channel between a client and a web server that are under my control. Both the client and server authenticates themselves using certificates that I’ve created under private PKI scheme. Client key and certificate are stored on usb dongle type HSM. Python is the main application language.

I’m able to do all required crypto operations for my project using python-pkcs11 package such as AES encryption, HMAC signing, RSA signing, and etc. However, I couldn’t find a way to “bind” pkcs11 to any TLS library. What I mean is a “Pythonic” way of calling a function that handles pkcs11 layer and establishes a TLS channel. Requests does not support pkcs11. libcurl has support for pkcs11 but it’s not implemented in pycurl, neither pyopenssl.

I’m able to do it openssl’s s_client CLI tool using engine api:

openssl s_client -engine pkcs11 -verify 2 -CAfile path/to/CA.pem -keyform engine -key "pkcs11:...;object=rsa;type=private" -cert path/to/client-cert.pem -connect localhost:8443

An example of what I’m looking for:

do_tls_with_pkcs(key=’pkcs11:URL’, cert=’cert.pem’, verify=’CA-cert.pem’)

As far as I could search around, no such library exists yet. Now I’m looking for a workaround.

I have read that if openssl, libp11, and python are compiled in such a way it is possible to abstract all of this, hence simple requests calls would go through HSM, transparent to application code. Although, I couldn’t find any material on how to do it.

November

Answer 1

I faced a similar problem as I wanted to use a PKCS#11 token (YubiKey, PIV applet) along Python requests.

I came up with https://github.com/cedric-dufour/scriptisms/blob/master/misc/m2requests.py

It's imperfect, in the sense that it does not use connections pools and does not support HTTP streams or proxying - like requests's stock HTTPS adapter does - but it does the job for simple connections to backends that require mTLS.

Ablaze

I followed a different path so I can't confirm this solution but thanks anyway. – November

@B.Ceylan, may I ask you which path you followed? I am looking for a similar solution. – Tudor

Answer 2

You can establish a mTLS session in Python using PKCS#11 with the M2Crypto library. This solution has been tested with a SafeNet eToken 5110+ FIPS on macOS Sonoma. Below is a working example of how to achieve this:

from M2Crypto import Engine, SSL, m2, m2urllib2

# Function to build an opener for mTLS using PKCS#11
def build_p11_mtls_opener(pkcs11_module, pkcs11_pin, private_key_uri, certificate_uri):
    # Initialize the PKCS#11 engine
    pkcs11_engine = Engine.Engine("pkcs11")
    pkcs11_engine.ctrl_cmd_string('MODULE_PATH', pkcs11_module)
    pkcs11_engine.ctrl_cmd_string('PIN', pkcs11_pin)
    pkcs11_engine.init()

    # Load private key and certificate
    private_key = pkcs11_engine.load_private_key(private_key_uri)
    certificate = pkcs11_engine.load_certificate(certificate_uri)

    # Create SSL context
    ssl_context = SSL.Context()

    # Disable server-side SSL certificate validation.
    ssl_context.set_verify(SSL.verify_none, 9)
    # IMPORTANT: For production, instead of the previous:
    # ssl_context.set_verify(SSL.verify_peer | SSL.verify_fail_if_no_peer_cert, 9)
    # ssl_context.set_default_verify_paths()

    # Set client-side certificate and key
    m2.ssl_ctx_use_x509(ssl_context.ctx, certificate.x509)
    m2.ssl_ctx_use_pkey_privkey(ssl_context.ctx, private_key.pkey)

    # Build opener with SSL context
    opener = m2urllib2.build_opener(ssl_context)

    return opener

# PKCS#11 module and credentials
pkcs11_module = '/Library/Frameworks/eToken.framework/Versions/A/libeToken.dylib'
pkcs11_pin = 'PE*jZC4k'
private_key_uri = "pkcs11:id=%48%F1%E3%A3%00%CD%3F%C8%17%88%01%04%46%01%22%A0%1E%8E%82%08;type=private"
certificate_uri = "pkcs11:id=%48%F1%E3%A3%00%CD%3F%C8%17%88%01%04%46%01%22%A0%1E%8E%82%08;type=cert"

# Create opener
opener = build_p11_mtls_opener(pkcs11_module, pkcs11_pin, private_key_uri, certificate_uri)

# Open URL and print response
opener_open = opener.open("https://localhost:8443/")
print(opener_open.read().decode('utf-8'))

This script demonstrates how to set up a mutual TLS (mTLS) connection using PKCS#11 tokens for client-side authentication. The M2Crypto library handles the SSL/TLS connection setup, private key, and certificate loading from the PKCS#11 module.

For quickly setting up an SSL server appropriate for the previous example, you can use the following commands:

openssl req -new -x509 -days 365 -nodes -out server.crt -keyout server.key -subj "/CN=localhost"
openssl s_server -accept 8443 -cert server.crt -key server.key -Verify 1 -www

These commands will generate a self-signed certificate and start an SSL server listening on port 8443, which you can use to test the client script.

Guinna