I have a need to ensure that an SMTP server certificate is signed by a public certificate authority. I would like to use phpseclib or some other trusted library. I believe I can use the root certificates extracted from Firefox.
There are some home-brew approaches here to check cert dates and other metadata, but it does not look like that does any signature checking as such (other than ensuring that OpenSSL does it). In any case, I want to use a library - I want to write as little certificate-handling code as possible, since I am not a cryptographer.
That said, the answers on the above link were still very useful, as it helped me get some code to fetch a certificate from a TLS conversation:
$url = "tcp://{$domain}:{$port}";
$connection_context_option = [
'ssl' => [
'capture_peer_cert' => true,
'verify_peer' => false,
'verify_peer_name' => false,
'allow_self_signed' => true,
]
];
$connection_context = stream_context_create($connection_context_option);
$connection_client = stream_socket_client($url, $errno, $errstr, 30, STREAM_CLIENT_CONNECT, $connection_context);
stream_set_timeout($connection_client, 2);
fread($connection_client, 10240);
fwrite($connection_client,"HELO alice\r\n");
fread($connection_client, 10240);
fwrite($connection_client, "STARTTLS\r\n");
fread($connection_client, 10240);
$ok = stream_socket_enable_crypto($connection_client, TRUE, STREAM_CRYPTO_METHOD_SSLv23_CLIENT);
if ($ok === false)
{
return false;
}
$connection_info = stream_context_get_params($connection_client);
openssl_x509_export($info["options"]["ssl"]["peer_certificate"], $pem_encoded);
(Note that I have turned off certificate validation here deliberately. This is because I have no control over what hosts this runs on, and their certificates may be old or misconfigured. Therefore, I wish to fetch the certificate regardless of the verification on the connection I am using, and then verify it myself using a cacert.pem
that I will supply.)
That will give me a certificate like this. This one is for Microsoft's Live.com email server at smtp.live.com:587
:
-----BEGIN CERTIFICATE-----
MIIG3TCCBcWgAwIBAgIQAtB7LVsRCmgbyWiiw7Sf5jANBgkqhkiG9w0BAQsFADBN
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMScwJQYDVQQDEx5E
aWdpQ2VydCBTSEEyIFNlY3VyZSBTZXJ2ZXIgQ0EwHhcNMTcwOTEzMDAwMDAwWhcN
MTkwOTEzMTIwMDAwWjBqMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3Rv
bjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0
aW9uMRQwEgYDVQQDEwtvdXRsb29rLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEP
ADCCAQoCggEBAIz2tovvgBmK4sOHgpyzCdtXrI0XOujctf6LHMj16wzUnMEatioS
tH0Pz0dKkCr/0yd9qtXbGhD1o6WhFsd7k651K9MZ98+uQ29SzTIAl6y1gkaBbp4h
MFXcE5EpRNHHmK8t2OR7hzmrvvNr6OTYv7BhVCw9pSrQqEFNno0K2TQRhAD9uzrL
OY+rBBVedCXWXH7uhZoZ6joUU7CEA5pPMzKPL1ro+Eorc8vt5FYOC+oAT587+b1M
z+jbZVQlq0qaMkBKRtUIII78MYY0n8DopGqHyzwqWoGySHJNC8256q+MwsZQvvQ3
vmy/rf61h2sg1tU0s7O88Yufxp0LSaMMzZcCAwEAAaOCA5owggOWMB8GA1UdIwQY
MBaAFA+AYRyCMWHVLyjnjUY4tCzhxtniMB0GA1UdDgQWBBT7hLoZ/03rqwcslIc2
0k0z2R+vNTCCAdwGA1UdEQSCAdMwggHPggtvdXRsb29rLmNvbYIWKi5jbG8uZm9v
dHByaW50ZG5zLmNvbYIWKi5ucmIuZm9vdHByaW50ZG5zLmNvbYIgYXR0YWNobWVu
dC5vdXRsb29rLm9mZmljZXBwZS5uZXSCG2F0dGFjaG1lbnQub3V0bG9vay5saXZl
Lm5ldIIdYXR0YWNobWVudC5vdXRsb29rLm9mZmljZS5uZXSCHWNjcy5sb2dpbi5t
aWNyb3NvZnRvbmxpbmUuY29tgiFjY3Mtc2RmLmxvZ2luLm1pY3Jvc29mdG9ubGlu
ZS5jb22CC2hvdG1haWwuY29tgg0qLmhvdG1haWwuY29tggoqLmxpdmUuY29tghZt
YWlsLnNlcnZpY2VzLmxpdmUuY29tgg1vZmZpY2UzNjUuY29tgg8qLm9mZmljZTM2
NS5jb22CFyoub3V0bG9vay5vZmZpY2UzNjUuY29tgg0qLm91dGxvb2suY29tghYq
LmludGVybmFsLm91dGxvb2suY29tggwqLm9mZmljZS5jb22CEm91dGxvb2sub2Zm
aWNlLmNvbYIUc3Vic3RyYXRlLm9mZmljZS5jb22CGHN1YnN0cmF0ZS1zZGYub2Zm
aWNlLmNvbTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG
AQUFBwMCMGsGA1UdHwRkMGIwL6AtoCuGKWh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNv
bS9zc2NhLXNoYTItZzEuY3JsMC+gLaArhilodHRwOi8vY3JsNC5kaWdpY2VydC5j
b20vc3NjYS1zaGEyLWcxLmNybDBMBgNVHSAERTBDMDcGCWCGSAGG/WwBATAqMCgG
CCsGAQUFBwIBFhxodHRwczovL3d3dy5kaWdpY2VydC5jb20vQ1BTMAgGBmeBDAEC
AjB8BggrBgEFBQcBAQRwMG4wJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLmRpZ2lj
ZXJ0LmNvbTBGBggrBgEFBQcwAoY6aHR0cDovL2NhY2VydHMuZGlnaWNlcnQuY29t
L0RpZ2lDZXJ0U0hBMlNlY3VyZVNlcnZlckNBLmNydDAMBgNVHRMBAf8EAjAAMA0G
CSqGSIb3DQEBCwUAA4IBAQA3zjN7I6jTeL+08nhG5eAY0q4pLY40bCQHqONBLSI3
uRmQFUfrQOPYBqLC1QU+J2Z2HcX7YiqE3WAR3ODS9g2BAVXkKOQKNBnr2hKwueOz
qPwyvTyzcIQYUw+SrTX+bfJwYMTmZvtP9S7/pB1jPhrV7YGsD55AI9bGa9cmH7VQ
OiL1p5Qovg5KRsldoZeC04OF/UQIR1fv47VGptsHHGypvSo1JinJFQMXylqLIrUW
lV66p3Ui7pFABGc/Lv7nOyANXfLugBO8MyzydGA4NRGiS2MbGpswPCg154pWausU
M0qaEPsM2o3CSTfxSJQQIyEe+izV3UQqYSyWkNqCCFPN
-----END CERTIFICATE-----
OK, great. So I want to validate this against any public CA. I believe this is a valid certificate, and the chain is correctly verified using this checking service:
Array
(
[name] => /C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=outlook.com
[subject] => Array
(
[C] => US
[ST] => Washington
[L] => Redmond
[O] => Microsoft Corporation
[CN] => outlook.com
)
[hash] => a3c08ece
[issuer] => Array
(
[C] => US
[O] => DigiCert Inc
[CN] => DigiCert SHA2 Secure Server CA
)
[version] => 2
[serialNumber] => 3740952067977374966703603448215281638
[serialNumberHex] => 02D07B2D5B110A681BC968A2C3B49FE6
[validFrom] => 170913000000Z
[validTo] => 190913120000Z
[validFrom_time_t] => 1505260800
[validTo_time_t] => 1568376000
[signatureTypeSN] => RSA-SHA256
[signatureTypeLN] => sha256WithRSAEncryption
[signatureTypeNID] => 668
[purposes] => Array
(
[1] => Array
(
[0] => 1
[1] =>
[2] => sslclient
)
[2] => Array
(
[0] => 1
[1] =>
[2] => sslserver
)
[3] => Array
(
[0] => 1
[1] =>
[2] => nssslserver
)
[4] => Array
(
[0] =>
[1] =>
[2] => smimesign
)
[5] => Array
(
[0] =>
[1] =>
[2] => smimeencrypt
)
[6] => Array
(
[0] =>
[1] =>
[2] => crlsign
)
[7] => Array
(
[0] => 1
[1] => 1
[2] => any
)
[8] => Array
(
[0] => 1
[1] =>
[2] => ocsphelper
)
[9] => Array
(
[0] =>
[1] =>
[2] => timestampsign
)
)
[extensions] => Array
(
[authorityKeyIdentifier] => keyid:0F:80:61:1C:82:31:61:D5:2F:28:E7:8D:46:38:B4:2C:E1:C6:D9:E2
[subjectKeyIdentifier] => FB:84:BA:19:FF:4D:EB:AB:07:2C:94:87:36:D2:4D:33:D9:1F:AF:35
[subjectAltName] => DNS:outlook.com, DNS:*.clo.footprintdns.com, DNS:*.nrb.footprintdns.com, DNS:attachment.outlook.officeppe.net, DNS:attachment.outlook.live.net, DNS:attachment.outlook.office.net, DNS:ccs.login.microsoftonline.com, DNS:ccs-sdf.login.microsoftonline.com, DNS:hotmail.com, DNS:*.hotmail.com, DNS:*.live.com, DNS:mail.services.live.com, DNS:office365.com, DNS:*.office365.com, DNS:*.outlook.office365.com, DNS:*.outlook.com, DNS:*.internal.outlook.com, DNS:*.office.com, DNS:outlook.office.com, DNS:substrate.office.com, DNS:substrate-sdf.office.com
[keyUsage] => Digital Signature, Key Encipherment
[extendedKeyUsage] => TLS Web Server Authentication, TLS Web Client Authentication
[crlDistributionPoints] =>
Full Name:
URI:http://crl3.digicert.com/ssca-sha2-g1.crl
Full Name:
URI:http://crl4.digicert.com/ssca-sha2-g1.crl
[certificatePolicies] => Policy: 2.16.840.1.114412.1.1
CPS: https://www.digicert.com/CPS
Policy: 2.23.140.1.2.2
[authorityInfoAccess] => OCSP - URI:http://ocsp.digicert.com
CA Issuers - URI:http://cacerts.digicert.com/DigiCertSHA2SecureServerCA.crt
[basicConstraints] => CA:FALSE
)
)
Here is how I am trying to validate the sig in phpseclib:
$x509 = new \phpseclib\File\X509();
// From the Mozilla bundle (getPublicCaCerts splits them with a regex)
$splitCerts = getPublicCaCerts(file_get_contents('cacert.pem'));
// Load the certs separately
$caStatus = true;
foreach ($splitCerts as $caCert)
{
$caStatus = $caStatus && $x509->loadCA($caCert);
}
// $caStatus is now true, so all good here
$certData = $x509->loadX509($pem_encoded); // From the TLS server
$valid = $x509->validateSignature();
// $valid is now false
This returns false
, which is not what I expect. I wonder if I have got the input formats correct? The loading of the CAs and the cert under test seem to return good values. Unfortunately, the phpseclib docs are a bit light on examples, and I've not found much elsewhere on the web.
Aside: I have a vague suspicion that this library could help me, assuming it has the feature to verify a certificate. However, I think it is trying to do to much for my case - I want my software to run on shared hosting, and auto-downloading feels like another moving part that might fail. I would rather deploy my own package, supply the public CA information as a (large) parameter, and run the validation test in situ. phpseclib is probably perfect for that, as long as I can figure out the input formats!
Possible reason: phpseclib can't find a matching cert to test
I have narrowed the problem down to a search loop in phpseclib's validator. On L2156, we have this code:
case !defined('FILE_X509_IGNORE_TYPE') && $this->currentCert['tbsCertificate']['issuer'] === $ca['tbsCertificate']['subject']:
The constant is indeed undefined, so the test really is whether a CA can match on the right cert particulars. The cert has this meta-data:
id-at-countryName = US
id-at-organizationName = DigiCert Inc
id-at-organizationalUnitName = www.digicert.com
id-at-commonName = DigiCert SHA2 High Assurance Server CA
And for all the current certs that would otherwise match, I have only these values in the latest cert bundle (i.e. all of the below would match if it was not for the common name DigiCert SHA2 High Assurance Server CA
not being found):
id-at-commonName = DigiCert Assured ID Root CA
id-at-commonName = DigiCert High Assurance EV Root CA
id-at-commonName = DigiCert Assured ID Root G2
id-at-commonName = DigiCert Assured ID Root G3
id-at-commonName = DigiCert Global Root G2
id-at-commonName = DigiCert Global Root G3
id-at-commonName = DigiCert Trusted Root G4
Thus, the system does not even get at far as a digital signature check, since it cannot find the CA corresponding to this cert. What am I missing? This simple task should be a lot easier than this!
Possible reason: the Mozilla bundle is web server certs only
I have speculated that mail server certificates are not in the Mozilla bundle because a web browser would have no need for them. I would assume though that the certs on my GNU/Linux Mint install would be up-to-date and suitable for the purpose, since an operating system should be able to verify certs used in mail servers.
I therefore tried this code, which loads all the system certs into phpseclib:
$certLocations = openssl_get_cert_locations();
$dir = $certLocations['default_cert_dir'];
$glob = $dir . '/*';
echo "Finding certs: " . $dir . "\n";
$x509 = new \phpseclib\File\X509();
foreach (glob($glob) as $certPath)
{
// Change this so it is recursive?
if (is_file($certPath))
{
$ok = $x509->loadCA(file_get_contents($certPath));
if (!$ok)
{
echo sprintf("CA cert `%s` is invalid\n", $certPath);
}
}
}
// The 'getCertToTest' func just gets the live.com cert as a string
$data = $x509->loadX509(getCertToTest());
if (!$data)
{
echo "Cert is invalid\n";
exit();
}
$valid = $x509->validateSignature();
echo sprintf("Validation: %s\n", $valid ? 'Yes' : 'No');
Unfortunately this fails as well.
Confirm that my system certs are OK using openssl
I have issued this command on my system, and the remote TLS cert is verified OK. I don't know the phpseclib code well, but it doesn't look like it is doing any chaining, which is evidently necessary.
openssl s_client -connect smtp.live.com:25 -starttls smtp
CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert Cloud Services CA-1
verify return:1
depth=0 C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, CN = outlook.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=outlook.com
i:/C=US/O=DigiCert Inc/CN=DigiCert Cloud Services CA-1
1 s:/C=US/O=DigiCert Inc/CN=DigiCert Cloud Services CA-1
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIG/jCCBeagAwIBAgIQDs2Q7J6KkeHe1d6ecU8P9DANBgkqhkiG9w0BAQsFADBL
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMSUwIwYDVQQDExxE
aWdpQ2VydCBDbG91ZCBTZXJ2aWNlcyBDQS0xMB4XDTE3MDkxMzAwMDAwMFoXDTE4
MDkxMzEyMDAwMFowajELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24x
EDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlv
(snipped, see other code block)
nGhseM2tJfwa2HMwUpuuo5029u4Dd40qvD0cMz33cOvBLRGkTPbXCFw24ZBdQrkt
SC5TAWzHFyT2tLC17LeSb7d0g+fuj41L6y4a9och8cPiv9IAP4sftzYupO99h4qg
7UXP7o3AOOGqrPS3INhO4068Z63indstanIHYM0IUHa3A2xrcz7ZbEuw1HiGH/Ba
HMz/gTSd2c0BXNiPeM7gdOK3
-----END CERTIFICATE-----
subject=/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=outlook.com
issuer=/C=US/O=DigiCert Inc/CN=DigiCert Cloud Services CA-1
---
No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA1:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA1:DSA+SHA1:RSA+SHA512:ECDSA+SHA512
Shared Requested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA1:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA1:DSA+SHA1:RSA+SHA512:ECDSA+SHA512
Peer signing digest: SHA1
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3831 bytes and written 478 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: C11A0000050CD144CB5C49DD873D2C911F7CDDECFE18001F70FE0427C88B52F7
Session-ID-ctx:
Master-Key: 5F4EC0B1198CF0A16D19F758E6A0961ED227FCEBD7EF96D4D6A7470E3F9B0453A2A06AC0C1691C31A1CA4B73209B38DE
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1519322480
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
250 SMTPUTF8
I may drop phpseclib in favour of the binary command, but I would be relying on system
/exec
etc, which may not be available. Still, working sometimes is better than not working always!
Summary
Despite extensive work, I have reached a dead end on this. I will summarise here what I am wanting to do.
I want to use PHP to verify mail server SSL certificates against known public CAs. I don't know if the Mozilla certificates are appropriate to use for this, or whether I need to obtain them from elsewhere. I have found that my Linux Mint development machine has certificates that will verify the example mail server above.
The trivial strategy here is to use PHP 5.6+ and ensure all verification options are enabled in the stream context (though ideally, I wish to support 5.5 also). However, I want to do the proof myself, either using openssl_
functions or a library such as phpseclib, so I can see why a given cert is valid (or not). The openssl
binary does this (as shown above) and it does so presumably using something very similar to PHP's openssl calls, but I don't know how it does so. For example, does the openssl binary use cert chain information to do this?
Another approach would be to read some information from a valid SSL session, but I cannot find anything in the manual to do that either.
$x509->loadCA($certs)
is returningfalse
, which means my public CA certs are indeed not recognised. I have a sneaking suspicion I have to parse the Mozilla extract and supply eachBEGIN ... END
to this method separately. Answers/comments still welcome, but I will update this in due course! – Mckenneytrue
for when each one is loaded. However, the validation method is still coming back false. There are a lot of clauses in that, so I may have to do some digging! – Mckenneyid-at-commonName
between the cert under test and my CA list. Given that I am testing a mail server cert and the list of CAs comes from Mozilla, I wonder if my CA list is in fact "web server only"? – Mckenneyopenssl
binary viaexec()
, but in my little collection of four real-world shared hosting test accounts, two had the system functions disabled. So I am hesitant to implement this withexec
when using the openssl module would be more resilient (and that module is a system requirement anyway). If it tempts you into doing some investigation, I am happy to put a juicy bounty on this! – Mckenneyopenssl
command output does the cert chaining. Perhaps one approach is to do twoopenssl_
PHP calls: one with verification off, to obtain the cert for cases where it is self-signed (and other invalid cases), and then once with verification on, to test whether it is verifiable. Perhaps in the latter case, there is some SSL session info available that can be retrieved, which describes the chain? – MckenneyloadCA()
should be seen more asloadTrustedCerts()
. CA certs are auto-trusted as are certs signed by the CA certs. If you have an intermediate cert that's signing your end cert and that intermediate cert is signed by a CA then that intermediate cert should be loaded withloadCA()
. – Mellott:o)
. Essentially, I can tell that the certs on my machine are enough to trust thelive.com
certificate (as confirmed by theopenssl
binary) but I am struggling to find out how to do this using theopenssl
PHP extension. Whether this necessitates chaining can be part of my question! – Mckenneyverify_peer
,verify_peer_name
, etc) and I am on PHP 5.6+ then I "know" the remote cert is trusted by the server (since the connection will be aborted if the connection is unverified). However, if I am on PHP 5.5 then I believe the status is "unchecked", and I'd be more comfortable if I can verify this properly using the extension, rather than hoping it is done for me. – Mckenney