Suds over https with cert

Question

I have soap service under Apache with ssl, suds works greate without ssl.
I have client certificate (my.crt and user.p12 files).
How I need to configure suds client ot make it work with service over https?

without certs i see

urllib2.URLError: <urlopen error [Errno 1] _ssl.c:499: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure>

Brade

It looks like it's relying on urllib2, which doesn't support such options. Note that urllib2 doesn't even verify the server certificate (see documentation), which you'd really need to do if you're serious about using HTTPS. – Elate

yep, but I can create my own transport based on other python library, which will use client certificate. What library you recomend instead of urllib2? – Brade

There was a discussion here: #6167648 and #1087727 – Elate

Answer 1

It sounds like you want to authenticate using a client certificate, not a server certificate as was stated in some of the comments. I had the same issue and was able to write a custom transport for SUDS. Here's the code that works for me.

You'll need your certificates in PEM format for this to work; OpenSSL can easily perform this conversion, though I don't remember the exact syntax.

import urllib2, httplib, socket
from suds.client import Client
from suds.transport.http import HttpTransport, Reply, TransportError

class HTTPSClientAuthHandler(urllib2.HTTPSHandler):
    def __init__(self, key, cert):
        urllib2.HTTPSHandler.__init__(self)
        self.key = key
        self.cert = cert

    def https_open(self, req):
        #Rather than pass in a reference to a connection class, we pass in
        # a reference to a function which, for all intents and purposes,
        # will behave as a constructor
        return self.do_open(self.getConnection, req)

    def getConnection(self, host, timeout=300):
        return httplib.HTTPSConnection(host,
                                       key_file=self.key,
                                       cert_file=self.cert)

class HTTPSClientCertTransport(HttpTransport):
    def __init__(self, key, cert, *args, **kwargs):
        HttpTransport.__init__(self, *args, **kwargs)
        self.key = key
        self.cert = cert

    def u2open(self, u2request):
        """
        Open a connection.
        @param u2request: A urllib2 request.
        @type u2request: urllib2.Requet.
        @return: The opened file-like urllib2 object.
        @rtype: fp
        """
        tm = self.options.timeout
        url = urllib2.build_opener(HTTPSClientAuthHandler(self.key, self.cert))
        if self.u2ver() < 2.6:
            socket.setdefaulttimeout(tm)
            return url.open(u2request)
        else:
            return url.open(u2request, timeout=tm)

# These lines enable debug logging; remove them once everything works.
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger('suds.client').setLevel(logging.DEBUG)
logging.getLogger('suds.transport').setLevel(logging.DEBUG)

c = Client('https://YOUR_URL_HERE',
    transport = HTTPSClientCertTransport('PRIVATE_KEY.pem',
                                         'CERTIFICATE_CHAIN.pem'))
print c

Mota

excellent answer. this would be a good example for using client certificates. needs to be there in the SUDS docs. :-) – Xenolith

"It sounds like you want to authenticate using a client certificate, not a server certificate as was stated in some of the comments.". This code doesn't authenticate the server: essentially, you're sending your client cert to something, but you haven't verified what that something was. None of this authenticates the server, which should be done first, whether you use client certs or not. – Elate

There is a small copyNpaste bug. instead of twice the param 'YOUR_KEY_AND_CERT.pem' it should be two filenames. First one to yout private key file, second one to the certificate chain file. both in unsecured .pem format. – Swinish

@willsteel Actually, you can have both the private key and the certificate (public key) in the same PEM file -- you just append them one after the other. – Mota

very util for this question: PFX or P7B or DER to PEM converter: sslshopper.com/ssl-converter.html – Venepuncture

I have just a .crt (Public) file and When I convert this to .pem file,I have a .pem file and in your code is dual cert file to connect ('YOUR_KEY_AND_CERT.pem', 'YOUR_KEY_AND_CERT.pem') ,how I I can use this code that you've written? I import both the same? – Sladen

SAXParseException: <unknown>:7:11: not well-formed (invalid token) , encountering this error, I converted the pfx file with password that I have to public and private pem keys. – Fauman

Is this answer useful for python3 ? – Smelter

Answer 2

Another workaround is to use requests library as transport which has better support for ssl. This is what I'm using now to access SOAP services through https using suds:-

import requests
from suds.transport.http import HttpAuthenticated
from suds.transport import Reply, TransportError

class RequestsTransport(HttpAuthenticated):
    def __init__(self, **kwargs):
        self.cert = kwargs.pop('cert', None)
        # super won't work because not using new style class
        HttpAuthenticated.__init__(self, **kwargs)

    def send(self, request):
        self.addcredentials(request)
        resp = requests.post(request.url, data=request.message,
                             headers=request.headers, cert=self.cert)
        result = Reply(resp.status_code, resp.headers, resp.content)
        return result

And then you can instantiate suds client as:-

headers = {"Content-TYpe" : "text/xml;charset=UTF-8",
           "SOAPAction" : ""}
t = RequestsTransport(cert='/path/to/cert', **credentials)
client = Client(wsdl_uri, location=send_url, headers=headers,
                transport=t))

Update

We're now using Zeep, which use requests underneath.

Repeated

+1 Nice answer. I visited this question almost a year ago and used the urllib2 described in @nitwit's answer. I'm refactoring that code today and decided to switch the whole thing over to requests. Quick question: what is **credentials supposed to be on line 2? I removed it and I'm not having problems, but I'm still curious. – Biotin

That if your soap endpoint require http basic authentication, so credentials = {'username': 'yourname', 'password': 'yourpass'}. – Repeated

I'm trying to run this and getting a ssl.SSLError: [Errno 1] _ssl.c:1359: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure. I could access the same wsdl_url with a simple requests.get(wsdl_url, cert='/path/to/cert.pem', verify=False). Shouldn't the request to get the WSDL file be made with a GET? – Kermit

In fact, I noticed that t.send is not even being called here. Help! – Kermit

Shouldn't the "verify" keyword argument be replaced by "cert" keyword argument in requests.post? docs.python-requests.org/en/latest/user/advanced/… – Trabue

@GiovanniP, yes, you also need to override the open() method. I've edited the answer, I hope it gets accepted. – Trabue

As my suggested edit can take ages before getting approved, I've also added a new answer. – Trabue

Answer 3

Based on @k4ml answer, I've only added the open() which allows to fetch the WSDL using the certificate.

This method should fix the suds.transport.TransportError: HTTP Error 403: Forbidden when trying to fetch a WSDL (at Client creation) served behind a HTTPS service.

import requests
from suds.transport.http import HttpAuthenticated
from suds.transport import Reply, TransportError

class RequestsTransport(HttpAuthenticated):
    def __init__(self, **kwargs):
        self.cert = kwargs.pop('cert', None)
        # super won't work because not using new style class
        HttpAuthenticated.__init__(self, **kwargs)

    def open(self, request):
        """
        Fetches the WSDL using cert.
        """
        self.addcredentials(request)
        resp = requests.get(request.url, data=request.message,
                             headers=request.headers, cert=self.cert)
        result = io.StringIO(resp.content.decode('utf-8'))
        return result

    def send(self, request):
        """
        Posts to service using cert.
        """
        self.addcredentials(request)
        resp = requests.post(request.url, data=request.message,
                             headers=request.headers, cert=self.cert)
        result = Reply(resp.status_code, resp.headers, resp.content)
        return result

Side note, I've also made a suggested edit to k4ml's answer, but it can take ages before it gets approved.

Trabue

When I do a client.service.something call and resp.status_code is 401, but the service call just returns None. Shouldn't it return a code or throw an exception? – Kappa

You're right, perhaps it could throw a TransportError or another exception, lust like in HttpTransport.send(). I don't have anything to test and confirm at the moment. – Trabue

If you get a xml.sax._exceptions.SAXParseException: <unknown>:1:1: not well-formed (invalid token) error may be you must replace io.StringIO(resp.content.decode('utf-8')) with StringIO.StringIO(resp.content) ... See also https://mcmap.net/q/470869/-python-suds-and-client-certificate-saxparseexception-not-well-formed-invalid-token – Fromm

Answer 4

Extending @k4ml solution, using cert + key This will solve exceptions like:

requests.exceptions.SSLError: [SSL] PEM lib (_ssl.c:2599)

Solution:

import requests

from suds.client import Client
from suds.transport.http import HttpAuthenticated
from suds.transport import Reply, TransportError


class RequestsTransport(HttpAuthenticated):

    def __init__(self, **kwargs):
        self.cert = kwargs.pop('cert', None)
        HttpAuthenticated.__init__(self, **kwargs)

    def send(self, request):
        self.addcredentials(request)
        resp = requests.post(
            request.url,
            data=request.message,
            headers=request.headers,
            cert=self.cert,
            verify=True
        )
        result = Reply(resp.status_code, resp.headers, resp.content)
        return result



t = RequestsTransport(cert=('<your cert.pem path>', 'your key.pem path'))
headers = {"Content-Type": "text/xml;charset=UTF-8", "SOAPAction": ""}
client = Client(wsdl_url, headers=headers, transport=t)

Botel

Answer 5

SSL security feature is auto enabled python 2.7.9+ which breaks suds and other python libraries. I am sharing a patch which can fix this:

Locate you suds library and replace u2handlers function in suds/trasnport/http.py file with following line:

import ssl
def u2handlers(self):
        """
        Get a collection of urllib handlers.

        @return: A list of handlers to be installed in the opener.
        @rtype: [Handler,...]

        """
        handlers = []
        unverified_context = ssl.create_default_context()
        unverified_context.check_hostname = False
        unverified_context.verify_mode = ssl.CERT_NONE
        unverified_handler = urllib2.HTTPSHandler(context=unverified_context)
        handlers.append(unverified_handler)
        handlers.append(urllib2.ProxyHandler(self.proxy))
        #handlers.append(urllib2.ProxyHandler(self.proxy))
        return handlers 

Note: It's not a recommended way of doing it.

Argal