Using Boost-Beast (Asio) http client with SSL (HTTPS)

Question

I am new to SSL and the Boost-Beast library, and also C++, but that's a different story. I have a question regarding using HTTPS (SSL) with Beast. I am going to use the library to connect to a REST service and post JSON to a server where I do not control the certificates, the API etc.

And it kind of works. The thing is that I do not clearly understand HOW it works. In the example that comes with Beast it is referenced to a file example/common/root_certificates.hpp where it is two certificates in base64 format or something like that.

When I tried the example, I got it to work with POST a JSON string to a server, let's call it "exampleserver.com". Connected to both port 80 and 443. Even though I commented out the line where it calls the root certificate function, load_root_certificates(ctx);. No errors from handshaking or anything else showed up. And the server responded correctly.

So, my questions are then:

1) Did the Beast library get the certificate from exampleserver.com or did I already have it installed and that's why it worked? If so what happens when it expires? Do I have to reinstall a new one on every client that uses my application? I would prefer to not have any hardcoded certificates in my code to check it upon. The certificate says DigiCert Global Root CA -> DigiCert SHA2 Secure Server CA is that a standard certificate maybe that comes with the browser?

2) Did it just use plain HTTP over port 443 (no SSL)? Don know if this is possible at all...

Heavenward

You need to build Boost Asio with the OpenSSL, so that asio will support TLS protocol, as well as HTTPS – Wolfsbane

See Asio build instructions – Wolfsbane

Yes, already done that. And it works. I tried to explain what I am asking for in the question. Thanks – Heavenward

If unsure about encryption simply use wireshark to intercept and then look at what is going on. No need to over-analyze the source code at first because you'll see TLS handshakes and a encrypted stream if it was correctly encrtyped. Once this is checked, I'd give code analysis a go. – Kershner

Answer 1

Even though I commented out the line where it calls the root certificate funtion, "load_root_certificates(ctx);"

In that case, openssl uses the systemwide default certificate store (e.g. on linux /etc/ssl/certs), so the "usual" authorities will be trusted (just like e.g. your browser does).

1) Did the Beast library get the certificate from "exampleserver.com" or did I already have it installed and thats why it worked?

Yes.

If so what happens when it expires?

It will fail to verify. Test it, if you want: https://expired.badssl.com/

That site has many excellent SSL tests (https://badssl.com)

The certificate says "DigiCert Global Root CA -> DigiCert SHA2 Secure Server CA" is that a standard certificate maybe that comes with the browser?

The browser's trusted certificates aren't relevant (you're not using the browser). However, you can see openssl's (see above), or you can test using something like

openssl s_client  -connect exampleserver.com:443 -verify -showcerts 

Which prints something similar to

verify depth is 0
CONNECTED(00000003)
depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify return:1
depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
verify return:1
depth=1 C = US, ST = TX, L = Houston, O = "cPanel, Inc.", CN = "cPanel, Inc. Certification Authority"
verify return:1
depth=0 CN = tradingfleet.com
verify return:1
---
Certificate chain
 0 s:/CN=tradingfleet.com
   i:/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
 1 s:/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
   i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
 2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFUjCCBDqgAwIBAgIQPI9I0oyjgNMrudesOYyqgDANBgkqhkiG9w0BAQsFADBy
MQswCQYDVQQGEwJVUzELMAkGA1UECBMCVFgxEDAOBgNVBAcTB0hvdXN0b24xFTAT
BgNVBAoTDGNQYW5lbCwgSW5jLjEtMCsGA1UEAxMkY1BhbmVsLCBJbmMuIENlcnRp
ZmljYXRpb24gQXV0aG9yaXR5MB4XDTE4MDIxODAwMDAwMFoXDTE4MDUxOTIzNTk1
OVowGzEZMBcGA1UEAxMQdHJhZGluZ2ZsZWV0LmNvbTCCASIwDQYJKoZIhvcNAQEB
BQADggEPADCCAQoCggEBANe5zu81biDwwIloBMFHWc2OvoiGTNBr2aya8auWrzRm
rmbOfugZOaIAms79jnINCQ7jy0Qk2xwblgCifCg7y/UfSXvv7IWUWcEDywsAoyz/
sUc9myvQbot+kD1DaxVoyN85LnDehaYF5+myDznJISQe1ei01n/aIF8gwOz4k3Gn
R07Zh0sDRBjIiRsAL6ZljrPRk47cul2+8pD0qNJHHN0QX6hz/KPOugTiivI1+ymo
onSeeN29oh5oTtCHP2yj9+RNsCNcPAnbDawy0RAgFi2W5GyHiIo/NkUxBXN8tQxH
2xrPnY+MQJHUcKXJd//DTX6tWoQqo4xisN6Q9iZ3+R8CAwEAAaOCAjkwggI1MB8G
A1UdIwQYMBaAFH4DWmVBa6d+CuG4nQjqHY4dasdlMB0GA1UdDgQWBBQKTFmhmBNx
pS9uBbXjqE1ZjCOiFjAOBgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNV
HSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwTwYDVR0gBEgwRjA6BgsrBgEEAbIx
AQICNDArMCkGCCsGAQUFBwIBFh1odHRwczovL3NlY3VyZS5jb21vZG8uY29tL0NQ
UzAIBgZngQwBAgEwTAYDVR0fBEUwQzBBoD+gPYY7aHR0cDovL2NybC5jb21vZG9j
YS5jb20vY1BhbmVsSW5jQ2VydGlmaWNhdGlvbkF1dGhvcml0eS5jcmwwfQYIKwYB
BQUHAQEEcTBvMEcGCCsGAQUFBzAChjtodHRwOi8vY3J0LmNvbW9kb2NhLmNvbS9j
UGFuZWxJbmNDZXJ0aWZpY2F0aW9uQXV0aG9yaXR5LmNydDAkBggrBgEFBQcwAYYY
aHR0cDovL29jc3AuY29tb2RvY2EuY29tMIGXBgNVHREEgY8wgYyCEHRyYWRpbmdm
bGVldC5jb22CF2NwYW5lbC50cmFkaW5nZmxlZXQuY29tghVtYWlsLnRyYWRpbmdm
bGVldC5jb22CGHdlYmRpc2sudHJhZGluZ2ZsZWV0LmNvbYIYd2VibWFpbC50cmFk
aW5nZmxlZXQuY29tghR3d3cudHJhZGluZ2ZsZWV0LmNvbTANBgkqhkiG9w0BAQsF
AAOCAQEAPFIZv1oHXm79+uoLnP9Sya2qEghOn/uPpNtappgUSrh2Pb0MueX84C0P
4HRS4yHRO1TD9ZOfCuPsguzXhl+RUB7Asl2iAhwthoZGMLhv6uaUnAUHZbpdkJY3
r/quuWHXDGNoe2quAOxGLPDO7WMvrDh1hFi7x7AGshkRSZ4DREBnCS7iprKzKL6H
BaNqtAlWgoXcSSg1RpnbU2o4bWIv8mZG0ATr7Cc8VSf04SjBLZnLTNeqo6Z+ALQ3
yrFsAytim6857FB231V5NEvLh+iZjSOuBG9xv+4Nw46bVz9z8QxB3czAodrDGXbB
lgH1s5f486lRq45dRn/hGY+DZjJXgg==
-----END CERTIFICATE-----
subject=/CN=tradingfleet.com
issuer=/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 4988 bytes and written 431 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 24CB439538212A23E0391887F856E369858AB6864B25DA5F1FD618550C41EB92
    Session-ID-ctx: 
    Master-Key: 1B8A3028923478527196B429D10F3584C5FA5DE4175C834CBBEF9EB19013FBFE58E7668CED9C0877E15F4F214A61F80C
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - ca 83 5a 76 c2 51 7b c7-68 15 12 a7 cb c9 f5 35   ..Zv.Q{.h......5
    0010 - 0a dc c1 2a 90 fd 61 69-0a d9 89 09 f0 c4 b3 40   ...*..ai.......@
    0020 - 79 dc 97 8a c5 0d a1 67-85 5e b4 25 47 94 ed 23   y......g.^.%G..#
    0030 - 42 df b2 99 25 ec b1 fa-d7 3e 3e 24 37 ef 67 ef   B...%....>>$7.g.
    0040 - 56 f4 d2 57 cd 47 48 bd-d7 86 b1 2f b5 76 d6 db   V..W.GH..../.v..
    0050 - 12 9d 7a d3 94 b0 58 bf-c5 c4 3e 7d 05 98 75 1d   ..z...X...>}..u.
    0060 - 31 bc 9b 23 4f a7 ce 37-af 77 8a 96 89 20 20 64   1..#O..7.w...  d
    0070 - 3d bf de 25 b2 09 02 20-49 09 b5 57 a1 c3 75 ed   =..%... I..W..u.
    0080 - 97 ec 51 d2 46 f7 c6 b7-4a d8 b2 db 95 eb ac d6   ..Q.F...J.......
    0090 - be 76 14 80 ca 08 dc b7-b6 cb e9 c9 cc 8b 45 bd   .v............E.
    00a0 - d7 1d a7 88 9b a4 91 33-aa 23 fe 23 65 b8 e1 d9   .......3.#.#e...
    00b0 - 98 f6 55 1e 25 32 97 b5-22 ac d0 58 01 a6 42 60   ..U.%2.."..X..B`

    Start Time: 1522150150
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
^C

2) Did it just use plain http over port 443 (no SSL)? Don know if this is possible at all...

No it didn't.

It's possible only given a whacky server config, but usually if your server connects https from a browser, it will be impossible to connect plain:

• https://askubuntu.com/questions/860511/is-it-possible-to-set-same-port-work-with-http-and-https
• on whacky configs: Nodejs HTTP and HTTPS over same port

Tannenberg

I do not believe OpenSSL consults the Windows certificate store. It only consults the "OpenSSL" certificate store. Which is to say, a folder that you have to configure and store certificates in manually. Which no one ever does, except maybe on Linux at certain workplaces. Certificate verification is hard to get right and Beast just barely slides in with the minimum level of support needed to get an example working. root_certificates.hpp is a hack. We will be working on a new library that does only certificate verification (using Asio) but this is a ways off. – Ximenez

Thanks a lot sehe for your excellent answer. I already feel a lot wiser, but I am still uncertain how I can proceed with this for the users of my application. @Vinnie Falco do I have I have to install the certificate manually for all users and make a hack (like the root_certificates.hpp example) to make sure it works as intended? Do I need to do anything else with my application or is the level of support in the library good enough? I am happy with a bare minimum as long as I can trust that the application is working and it the connection is SSL :) Thanks! – Heavenward

@VinnieFalco on linux there is very much a default, and you can find it like this. On debian-likes it is symlinked to /etc/ssl/certs and usually filled (for me ~450 files, in ~298 unique certs). I didn't see OP saying he's using Windows. It would be /interesting/ to have support for the Windows trusted store(s) indeed. – Tannenberg

@Heavenward I don't think that the Beast examples can be trusted 100% with respect to certificate verification. The verification of certificates is outside the scope of Beast, it is the caller's responsibility to interact with Asio to do this. – Ximenez

If you're concerned about whether "the connection is SSL :)" - the code is fine because the handshake will simply fail otherwise. If you want certificate validation in the mix, it's best to (a) have tests (b) program defensively. E.g. make sure you log/store the server certificate for later use. You can't store a certificate if you didn't have one. One "simple" way is a whitelist of the only certificate(s) that you accept for a given server and accept. This will obviously still have some overhead in case you need to migrate servers/change certs – Tannenberg

Answer 2

As suggested by Vinnie Falco, in the file example/common/root_certificates.hpp, use the following header only library https://github.com/djarek/certify

In your code add the following includes

#include <boost/certify/extensions.hpp>
#include <boost/certify/https_verification.hpp>

And replace the initial code:

    // This holds the root certificate used for verification
    load_root_certificates(ctx);

    // Verify the remote server's certificate
    ctx.set_verify_mode(ssl::verify_peer);

by this:

    ctx.set_verify_mode(ssl::context::verify_peer );
    boost::certify::enable_native_https_server_verification(ctx);

Tested quickly with site badssl.com and sites with good ssl certificate. And it's work like a charm.

Linguist

Answer 3

Possibly, you already figured it out by now.

I tried the same beast sample (Boost libraries 1.70) and had to make the following change to the ctor of session (I did it there it may be possible to make the change at some other place in code as well):

    ws_.next_layer().set_verify_mode(boost::asio::ssl::verify_peer);
    ws_.next_layer().set_verify_callback(std::bind(&session::verify_certificate, this, _1, _2));

and added a method (that I copied as is from the Asio client sample):

bool verify_certificate(bool pverified_ok, ssl::verify_context& ctx)
{
    char subject_name[256];
    X509 *cert = X509_STORE_CTX_get_current_cert(ctx.native_handle());
    X509_NAME_oneline(X509_get_subject_name(cert), subject_name, 256);
    std::cout << "Verifying " << subject_name << std::endl;

    return pverified_ok;
}

This change causes the verification to fail (I had removed the hard-coded certificates since I did not want to use those). The callback assists in logging that the server certificate was actually being verified.

Just like the Asio sample adding a CA certificate to the ssl::context e.g.

    boost::asio::ssl::context ctx(boost::asio::ssl::context::sslv23);
    ctx.load_verify_file("ca.pem"); // CA certificate

Causes the verification to pass.

You need to create the self-signed CA certificate and server certificate signed by it and place it in the server code (again from Asio sample) e.g.

    context_.use_certificate_chain_file("..\\sample-server1.pem");
    context_.use_private_key_file("..\\sample-server1-key.pem", boost::asio::ssl::context::pem);
    context_.use_tmp_dh_file("..\\dh2048.pem");

Constantia