.NET application fails to send client certificate - Win 7 vs Win XP?

Question

I am developing an ASP.NET web application that sends a request to another server using HttpWebRequest. It sends the request over HTTPS, and the remote server requires a client certificate. The request fails in the .NET application, apparently unable to send the correct client certificate. I am able to successfully connect and send the client certificate if I simply visit the url with a web browser (Chrome specifically).

The code below is a simple reproduction, with just a basic GET request.

var r = WebRequest.Create(url) as HttpWebRequest;
r.ClientCertificates = new X509CertificateCollection { myX509Cert };
using (var resp = r.GetResponse() as HttpWebResponse) {
    ...
}

I get our favorite exception, "Could not create SSL/TLS secure channel". Typically, these types of problems point to issues with permissions on your certificate's private key. I tried everything I could think of to ensure this is all configured correctly, but perhaps I missed something. Long story short, the remote server is sending a TLS CertificateRequest with a list that that does seem to properly identify my client certificate, but my appliation fails to respond with any client certificate.

Here is my setup:

• Windows 7 professional 64 bit
• Able to reproduce the problem in an ASP.NET MVC 3 / .NET 4 application running in the Visual Studio dev server, ASP.NET WebForms / .NET 3.5 application running in local IIS, and in a .NET console application / .NET 4
• Microsoft .NET Framework 4.5 was recently installed. Haven't examined yet whether this could be a problem

Here is everything I've tried, and what I know:

• This code seemed to work fine when running on a Windows XP machine
• I ensured that my client certificate is imported into the Local Computer, Personal Certificates store, with the Private Key permissions properly configured for myself and all relevant IIS users
• I attempted reinstalling the certificate on my machine a couple of times
• I have confirmed that the .NET application can access the X509 certificate and HasPrivateKey = true
• Ensured that my client certificate is valid. It's actually an SSL cert for the web server on which this application will be running
• I set PreAuthenticate = true in the request object. Did not make a difference
• I tried setting ServicePointManager.Expect100Continue = false, did not make a difference
• I tried setting ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3, but apparently the remote server requires TLS so that's not helping
• I set a ServicePointManager.ServerCertificateValidationCallback delegate to always return true. But the request fails earlier in the TLS handshake, before it even gets to the point of calling this delegate
• When I go to the URL in Chrome, it asks me to supply a client certificate, presents the correct client cert as a choice, I choose that cert and I get a valid response. Also works properly when I construct a request in Fiddler. So it definitely seems to be specifically a problem with my .NET code, not the certificate itself, or the remote server, etc.
• The vendor I am working with has the same service set up on a different remote server, and that service requires a different client cert. So I tried the same thing with the different URL and client cert and got the same failure

I added System.Net trace and saw this:

SecureChannel#26717201 - We have user-provided certificates. The server has specified 6 issuer(s). Looking for certificates that match any of the issuers.

SecureChannel#26717201 - Left with 0 client certificates to choose from.

...

InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=CertUnknown).

I enabled full SCHANNEL logging, and saw this warning:

The remote server has requested SSL client authentication, but no suitable client certificate could be found. An anonymous connection will be attempted. This SSL connection request may succeed or fail, depending on the server's policy settings.

I ran Wireshark, and saw the remote server sent a CertificateRequest, and it seems to have a Distinguished Name entry with the CN\OU\O values of my client certificate specified exactly. After that, my application sends a Certificate response with no certificates.

It seems like I am missing something with setting up this certificate to work properly with .NET applications in Windows 7. My best guess is that there is something different on my new Windows 7 machine, compared to the XP machine, that is causing this to fail now. I don't have access to a Windows XP environment at the moment to confirm this; I will in a couple of days but would really like to resolve this ASAP.

Any ideas would be much appreciated. Thanks!

EDIT As described above, when connecting to the URL in Chrome, the browser asks me for a client certificate, and I can provide the correct certificate and connect successfully. I cannot do this successfully in Internet Explorer (9), however. I simply get "Internet Explorer cannot display the webpage", with no other prompts or explanation. I was told that this may be relevant as WebRequest.Create has similar behavior to IE. I am looking into what this might mean, but would value any thoughts on this.

EDIT Also, I should note that the remote server uses a self-signed SSL certificate. I originally thought that might be the problem, so I added the cert as a trusted root in MMC so that the certificate appears as valid on my machine. This did not fix the problem.

Jeopardy

Does the certificate include the "Client Authentication (1.3.6.1.5.5.7.3.2)" Enhanced Key Usage extension? If not, it may be that Chrome is being naughty in allowing it to be selected. – Aimee

It turns out that the client cert does not have this Enhanced Key Usage extension. I'm pretty sure this code was working correctly in the past when it was pointed to a different remote URL and a different client certificate, even though that client cert did not have the Client Authentication enhanced key usage. This gives me a path to investigate though, thanks. – Jeopardy

Actually Damien, I was wrong - all of these certs do include the Client Authentication enhanced key usage extension. – Jeopardy

There is absolutely no need to wiki this. You're all good. – Sandrasandro

Answer 1

This turned out be a rather simple problem, but it was hard to spot. The remote server that my application had my client certificate in its keystore, but not any of the root certificates in my client cert's trust chain.

I was able to use my code to successfully send a request to a different server that required client certificates. I took a capture in Wireshark while sending this successful request, and also took a capture while sending the failed request to the other server. In the Wireshark capture, I found the "Server Hello" and compared the messages sent from the remote servers. The "good" remote server was sending my client certificate and also also its root certificate in the "Certificate Request" portion of that message. The "bad" remote server was only sending my client certificate.

This reminded me that the System.Net diagnostic trace read "The server has specified 6 issuer(s) ... Left with 0 client certificates to choose from". So it turns out that "issuer" is the key term here. It was just easy to overlook initially because in my initial analysis of the TLS handshake, the server was sending my client cert in the Certificate Request. In hindsight, it makes sense that the server should be sending your root certificate, not your client certificate itself.

Jeopardy

Can you help clarify what you did to solve this issue? I'm using a self signed certificate and just use the following to validate the cert: ServicePointManager.ServerCertificateValidationCallback = delegate(object s, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) { return true; }; My host has updated to .net 4.5 and now my services returns the following error "Could not create SSL/TLS secure channel" – Nunci

This resolved my issue too. It took me 2 entire week to figure this out. The remote server was only sending the client certificate,not including RSA CA root certificate. Once i read this answer and suggested the solution to include RSA CA root certificate on "certificate request" handshake, it solved the problem. Thank you!!! – Pursuivant

Answer 2

I would like to add another "solution" to the problem.

The server can fail to send the correct list of issuers if that list is too long. This seems to be a Microsoft limitation. http://support.microsoft.com/kb/933430

Source: http://netsekure.org/2011/04/tls-client-authentication-and-trusted-issuers-list/

The solution is to ask the server not to send the list at all. (By editing a registry value)

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL

Value name: SendTrustedIssuerList Value type: REG_DWORD Value data: 0 (False)

Bac

Answer 3

For me, these exact symptoms were being caused by using TLS 1.0, which the server wasn't allowing. This can be found in the trace logs as such:

System.Net Information: 0 : ProcessAuthentication(Protocol=Tls, Cipher=TripleDes 168 bit strength, Hash=Sha1 160 bit strength, Key Exchange=RsaKeyX 2048 bit strength).

TLS 1.0 is the default for .NET 4.5, but it can be overridden by setting:

ServicePointManager.SecurityProtocol = 
    SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;

There are also some registry flags that allow setting this without changing existing code.

Rolf