Using a self-signed certificate with .NET's HttpWebRequest/Response

Question

I'm trying to connect to an API that uses a self-signed SSL certificate. I'm doing so using .NET's HttpWebRequest and HttpWebResponse objects. And I'm getting an exception that:

The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

I understand what this means. And I understand why .NET feels it should warn me and close the connection. But in this case, I'd like to just connect to the API anyway, man-in-the-middle attacks be damned.

So, how do I go about adding an exception for this self-signed certificate? Or is the approach to tell HttpWebRequest/Response not to validate the certificate at all? How would I do that?

Menander

Answer 1

@Domster: that works, but you might want to enforce a bit of security by checking if the certificate hash matches what you expect. So an expanded version looks a bit like this (based on some live code we're using):

static readonly byte[] apiCertHash = { 0xZZ, 0xYY, ....};

/// <summary>
/// Somewhere in your application's startup/init sequence...
/// </summary>
void InitPhase()
{
    // Override automatic validation of SSL server certificates.
    ServicePointManager.ServerCertificateValidationCallback =
           ValidateServerCertficate;
}

/// <summary>
/// Validates the SSL server certificate.
/// </summary>
/// <param name="sender">An object that contains state information for this
/// validation.</param>
/// <param name="cert">The certificate used to authenticate the remote party.</param>
/// <param name="chain">The chain of certificate authorities associated with the
/// remote certificate.</param>
/// <param name="sslPolicyErrors">One or more errors associated with the remote
/// certificate.</param>
/// <returns>Returns a boolean value that determines whether the specified
/// certificate is accepted for authentication; true to accept or false to
/// reject.</returns>
private static bool ValidateServerCertficate(
        object sender,
        X509Certificate cert,
        X509Chain chain,
        SslPolicyErrors sslPolicyErrors)
{
    if (sslPolicyErrors == SslPolicyErrors.None)
    {
        // Good certificate.
        return true;
    }

    log.DebugFormat("SSL certificate error: {0}", sslPolicyErrors);

    bool certMatch = false; // Assume failure
    byte[] certHash = cert.GetCertHash();
    if (certHash.Length == apiCertHash.Length)
    {
        certMatch = true; // Now assume success.
        for (int idx = 0; idx < certHash.Length; idx++)
        {
            if (certHash[idx] != apiCertHash[idx])
            {
                certMatch = false; // No match
                break;
            }
        }
    }

    // Return true => allow unauthenticated server,
    //        false => disallow unauthenticated server.
    return certMatch;
}

Pademelon

Probably someone who preferred the correct way below. Anyway, this hack works in a pinch, but you probably shouldn't be coding these kinds of exceptions in... either just disable the checking all together (via the suggestion directly below) or actually instruct your computer to trust the certificate... – Cimmerian

@BrainSlugs83: Disabling is certainly an option too, but adding the cert to the machine-level root authorities store can only be done by administrators. My solution works either way. – Pademelon

And I fully understand that, but you asked, and that's still my guess as to why someone down-voted your answer. And regardless of it being more work, IMHO wgthom's answer below is still the most correct one. – Cimmerian

btw, be careful, i think that ServerCertificateValidationCallback is STATIC, and not even threadlocal. if i'm not wrong, then once set, it stays set until you clear it. if you want to use it for one connection only and not in all others, be very careful with parallel requests.. – Mccaskill

This is the best way to do this. If you remove the check against sslPolicyErrors, you can actually ensure the API certificate is always the expected one. One thing to note is that the certificate fingerprint in the code above is a const byte array. This will not compile as written. Try a static readonly byte array instead. The compiler chokes on this because it requires the new() operator. – Windsor

Thanks for noticing the typo @Centijo, fixed. – Pademelon

Answer 2

Turns out, if you just want to disable certificate validation altogether, you can change the ServerCertificateValidationCallback on the ServicePointManager, like so:

ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };

This will validate all certificates (including invalid, expired or self-signed ones).

Menander

Perfect for some quick testing against development machines. Thanks. – Boffa

What scope does this affect - everything in the appdomain? everything on the apppool? everything on the machine? – Postcard

But be carefull! The RL experience shows that this development hacks often makes it's way into the release product: The most dangerous code in the world – Amphicoelous

This is a hack useful in development so putting a #if DEBUG #endif statement around it is the least you should do to make this safer and stop this ending up in production. – Hoi

Unless this guy removes this answer, we will see a funny fact that a wrong answer receives far more votes than the correct one. – Doorkeeper

This is perfect for a keep alive request that does not need validation. – Hydrotherapy

Sadly more votes for a hack rather than the real solution above by @devstuff. Fine for dev but never let this code into production! – Holsworth

Answer 3

Note, that in .NET 4.5 you can override SSL validation per HttpWebRequest itself (and not via global delegate which affects all requests):

http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.servercertificatevalidationcallback.aspx

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.ServerCertificateValidationCallback = delegate { return true; };

Hybrid

Please upvote this; this is worth upgrading to 4.5 for! – Tobin

@FlorianWinter Yes, you have to adopt the logic from user devstuff – Underplot

Answer 4

Add the self signed cert to the Local Computer Trusted Root Certification Authorities

You can import the cert by running the MMC as Administrator.

How to: View Certificates with the MMC Snap-in

Illnatured

IMHO this is the most correct way; people are just too lazy so they code in special exceptions for things they probably shouldn't. – Cimmerian

Does that method work for Windows Mobile 6.5? How about 7? In my case, I didn't want to have to add a local certificate to every mobile device I planned to run a development version on. A good exception, in this case, makes deployment a ton easier. Laziness or efficiency, you tell me. – Menander

@domster You're using SSL certs for a reason - to verify endpoints. If you develop code that specifically works around that, you're not testing it properly and risk leaking that code into a live environment. If installing a cert on the client is really too much work, why not just pay for a cert from an issuer trusted by all the devices? – Forestay

@Forestay If I remember this specific case, I would have needed several wildcard certs (there were half a dozen TLDs it was connecting to, all under our control). That's a hard-to-justify cost for a development environment. In this case, the only code being "worked around" and not tested is that an exception is not thrown where it otherwise would be. You should be testing that specific exception path regardless of whether you're using this workaround. And, finally, if you can't keep development code out of production, you have much bigger problems than SSL validation. – Menander

for webapps, be sure to recycle your apppool or restart your website. personally, i just recompiled, and then it worked. for our wsdl stuff, the cert verification appears to happen on initialization and cached. – Undershot

I've added the CA file into trusted CAs using MMC but the .NET app wouldn't accept it. The app is not mine :( The CA cert is ok, because it works if I add it into Firefox or into Cygwin and test. Any ideas why? – Adelinaadelind

Answer 5

The scope of the validation callback used in Domster's answer can be limited to a specific request using the sender parameter on the ServerCertificateValidationCallback delegate. The following simple scope class uses this technique to temporarily wire up a validation callback that only executes for a given request object.

public class ServerCertificateValidationScope : IDisposable
{
    private readonly RemoteCertificateValidationCallback _callback;

    public ServerCertificateValidationScope(object request,
        RemoteCertificateValidationCallback callback)
    {
        var previous = ServicePointManager.ServerCertificateValidationCallback;
        _callback = (sender, certificate, chain, errors) =>
            {
                if (sender == request)
                {
                    return callback(sender, certificate, chain, errors);
                }
                if (previous != null)
                {
                    return previous(sender, certificate, chain, errors);
                }
                return errors == SslPolicyErrors.None;
            };
        ServicePointManager.ServerCertificateValidationCallback += _callback;
    }

    public void Dispose()
    {
        ServicePointManager.ServerCertificateValidationCallback -= _callback;
    }
}

The above class can be used to ignore all certificate errors for a specific request as follows:

var request = WebRequest.Create(uri);
using (new ServerCertificateValidationScope(request, delegate { return true; }))
{
    request.GetResponse();
}

Filamentary

This answer needs more up-votes :) It's the most reasonable answer to skip certificate validation for a single request using an HttpWebRequest object. – Talie

I added this and I'm still getting The request was aborted: Could not create SSL/TLS secure channel. – Langur

This doesn't really solve the problem in a multi-threaded environment. – Centro

maaan!!!, a 5 years old post save my day, i have problem connecting to old-satellite-modem-appliance with invalid certificate!! Thank you!! – Accessary

I'm confused/slightly worried! Doesn't returning SslPolicyErrors.None in the case where there was no previous callback mean that we end up overriding the default policy with an 'accept all' policy? cf. this question and its various answers: https://mcmap.net/q/67660/-how-to-call-the-default-certificate-check-when-overriding-servicepointmanager-servercertificatevalidationcallback-in-c . I'd be very happy to be told why I'm wrong and this code is fine! – Peal

SslPolicyErrors.None isn't returned, it's compared with the errors callback argument. – Filamentary

Answer 6

Just building on answer from devstuff to include subject and issuer...comments welcome...

public class SelfSignedCertificateValidator
{
    private class CertificateAttributes
    {
        public string Subject { get; private set; }
        public string Issuer { get; private set; }
        public string Thumbprint { get; private set; }

        public CertificateAttributes(string subject, string issuer, string thumbprint)
        {
            Subject = subject;
            Issuer = issuer;                
            Thumbprint = thumbprint.Trim(
                new char[] { '\u200e', '\u200f' } // strip any lrt and rlt markers from copy/paste
                ); 
        }

        public bool IsMatch(X509Certificate cert)
        {
            bool subjectMatches = Subject.Replace(" ", "").Equals(cert.Subject.Replace(" ", ""), StringComparison.InvariantCulture);
            bool issuerMatches = Issuer.Replace(" ", "").Equals(cert.Issuer.Replace(" ", ""), StringComparison.InvariantCulture);
            bool thumbprintMatches = Thumbprint == String.Join(" ", cert.GetCertHash().Select(h => h.ToString("x2")));
            return subjectMatches && issuerMatches && thumbprintMatches; 
        }
    }

    private readonly List<CertificateAttributes> __knownSelfSignedCertificates = new List<CertificateAttributes> {
        new CertificateAttributes(  // can paste values from "view cert" dialog
            "CN = subject.company.int", 
            "CN = issuer.company.int", 
            "f6 23 16 3d 5a d8 e5 1e 13 58 85 0a 34 9f d6 d3 c8 23 a8 f4") 
    };       

    private static bool __createdSingleton = false;

    public SelfSignedCertificateValidator()
    {
        lock (this)
        {
            if (__createdSingleton)
                throw new Exception("Only a single instance can be instanciated.");

            // Hook in validation of SSL server certificates.  
            ServicePointManager.ServerCertificateValidationCallback += ValidateServerCertficate;

            __createdSingleton = true;
        }
    }

    /// <summary>
    /// Validates the SSL server certificate.
    /// </summary>
    /// <param name="sender">An object that contains state information for this
    /// validation.</param>
    /// <param name="cert">The certificate used to authenticate the remote party.</param>
    /// <param name="chain">The chain of certificate authorities associated with the
    /// remote certificate.</param>
    /// <param name="sslPolicyErrors">One or more errors associated with the remote
    /// certificate.</param>
    /// <returns>Returns a boolean value that determines whether the specified
    /// certificate is accepted for authentication; true to accept or false to
    /// reject.</returns>
    private bool ValidateServerCertficate(
        object sender,
        X509Certificate cert,
        X509Chain chain,
        SslPolicyErrors sslPolicyErrors)
    {
        if (sslPolicyErrors == SslPolicyErrors.None)
            return true;   // Good certificate.

        Dbg.WriteLine("SSL certificate error: {0}", sslPolicyErrors);
        return __knownSelfSignedCertificates.Any(c => c.IsMatch(cert));            
    }
}

Doradorado

Answer 7

To add as a possible help to someone else... If you want it to prompt the user to install the self-signed cert, you can use this code (modified from above).

Does not require admin rights, installs to the local users trusted profiles:

    private static bool ValidateServerCertficate(
        object sender,
        X509Certificate cert,
        X509Chain chain,
        SslPolicyErrors sslPolicyErrors)
    {
        if (sslPolicyErrors == SslPolicyErrors.None)
        {
            // Good certificate.
            return true;
        }

        Common.Helpers.Logger.Log.Error(string.Format("SSL certificate error: {0}", sslPolicyErrors));
        try
        {
            using (X509Store store = new X509Store(StoreName.My, StoreLocation.CurrentUser))
            {
                store.Open(OpenFlags.ReadWrite);
                store.Add(new X509Certificate2(cert));
                store.Close();
            }
            return true;
        }
        catch (Exception ex)
        {
            Common.Helpers.Logger.Log.Error(string.Format("SSL certificate add Error: {0}", ex.Message));
        }

        return false;
    }

This seems to work well for our application, and if the user presses no, the communication will not work.

Update: 2015-12-11 - Changed StoreName.Root to StoreName.My - My will install into the local users store, instead of Root. Root on some systems will not work, even if you "run as administrator"

Emboss

This would be awesome if it worked on Compact Framework winCE. store.Add(..) is not available. – Gens

Answer 8

First of all - I apologize, because I have used the solution that was described by @devstuff. However, I have found some ways to improve it.

• adding self-signed certificates handling
• comparison by the Raw data of certificates
• actual certificate authority validation
• some additional comments and improvements

Here is my modification:

private static X509Certificate2 caCertificate2 = null;

/// <summary>
/// Validates the SSL server certificate.
/// </summary>
/// <param name="sender">An object that contains state information for this validation.</param>
/// <param name="cert">The certificate used to authenticate the remote party.</param>
/// <param name="chain">The chain of certificate authorities associated with the remote certificate.</param>
/// <param name="sslPolicyErrors">One or more errors associated with the remote certificate.</param>
/// <returns>Returns a boolean value that determines whether the specified certificate is accepted for authentication; true to accept or false to reject.</returns>
private static bool ValidateServerCertficate(
        object sender,
        X509Certificate cert,
        X509Chain chain,
        SslPolicyErrors sslPolicyErrors)
{
    if (sslPolicyErrors == SslPolicyErrors.None)
    {
        // Good certificate.
        return true;
    }

    // If the following line is not added, then for the self-signed cert an error will be (not tested with let's encrypt!):
    // "A certificate chain processed, but terminated in a root certificate which is not trusted by the trust provider. (UntrustedRoot)"
    chain.ChainPolicy.VerificationFlags = X509VerificationFlags.AllowUnknownCertificateAuthority;

    // convert old-style cert to new-style cert
    var returnedServerCert2 = new X509Certificate2(cert);

    // This part is very important. Adding known root here. It doesn't have to be in the computer store at all. Neither do certificates.
    chain.ChainPolicy.ExtraStore.Add(caCertificate2);

    // 1. Checks if ff the certs are OK (not expired/revoked/etc) 
    // 2. X509VerificationFlags.AllowUnknownCertificateAuthority will make sure that untrusted certs are OK
    // 3. IMPORTANT: here, if the chain contains the wrong CA - the validation will fail, as the chain is wrong!
    bool isChainValid = chain.Build(returnedServerCert2);
    if (!isChainValid)
    {
        string[] errors = chain.ChainStatus
            .Select(x => String.Format("{0} ({1})", x.StatusInformation.Trim(), x.Status))
            .ToArray();

        string certificateErrorsString = "Unknown errors.";

        if (errors != null && errors.Length > 0)
        {
            certificateErrorsString = String.Join(", ", errors);
        }

        Log.Error("Trust chain did not complete to the known authority anchor. Errors: " + certificateErrorsString);
        return false;
    }

    // This piece makes sure it actually matches your known root
    bool isValid = chain.ChainElements
        .Cast<X509ChainElement>()
        .Any(x => x.Certificate.RawData.SequenceEqual(caCertificate2.GetRawCertData()));

    if (!isValid)
    {
        Log.Error("Trust chain did not complete to the known authority anchor. Thumbprints did not match.");
    }

    return isValid;
}

setting certificates:

caCertificate2 = new X509Certificate2("auth/ca.crt", "");
var clientCertificate2 = new X509Certificate2("auth/client.pfx", "");

passing delegate method

ServerCertificateValidationCallback(ValidateServerCertficate)

client.pfx is generated with KEY and CERT as such:

openssl pkcs12 -export -in client.crt -inkey client.key -out client.pfx

Isley

Answer 9

One thing to keep in mind is that having the ServicePointManager.ServerCertificateValidationCallback does not seem to mean that the CRL check and servername validation are not done, it only provides a means to override their result. So your service might still take a while to get a CRL, you'll only know afterwards that it failed some checks.

Dhiman

Answer 10

I was running into the same problem as the OP where the web request would throw that exact exception. I had everything setup correctly I thought, the certificate was installed, I could locate it in the machine store just fine and attach it to the web request, and I had disabled the verification of certificates on the request context.

It turned out that I was running under my user account, and that the certificate was installed to the machine store. This caused the web request to throw this exception. To solve the problem I had to either be running as administrator or install the certificate to the user store and read it from there.

It would seem that C# is able to find the certificate in the machine store even though it can't be used with a web request, and that this results in the OP's exception being thrown once the web request is issued.

Teleran

For Windows services you can setup separate certificate configurations per each service. If you're writing not a desktop app but a service, the CA cert can be imported in MMC for the service daemon specifically. What is the difference between user account and machine account? I thought everything in machine account applies to user automatically. – Adelinaadelind