How does an SSL certificate chain bundle work?

Question

I've created a chain hierarchy like this.

root-ca ==> signing-ca ==> subordinate-ca ==> server

It is mentioned to create chain bundle, the lowest should go first.

$ cat server.crt subordinate-ca.crt signing-ca.crt > server.pem

But verification fails.

$ openssl verify -CAfile root-ca.crt server.pem
error 20 at 0 depth lookup:unable to get local issuer certificate

However, if I change the order it seems to work.

$ cat signing-ca.crt subordinate-ca.crt server.crt > server.pem
$ openssl verify -CAfile root-ca.crt server.pem
server.pem: OK

So what would be the error here?

The chain after "cat" looks like below.

-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

More info: According to "http://www.herongyang.com/crypto/openssl_verify_2.html", I perform the following test which works.

$ cat signing-ca.crt subordinate-ca.crt > inter.crt
$ openssl verify -CAfile root-ca.crt -untrusted inter.crt server.crt
server.crt: OK

Does that mean all the links are good?

OK, I finally discover that this cannot be done through OpenSSL command line (or at least easily). http://openssl.6102.n7.nabble.com/check-certificate-chain-in-a-pem-file-td43871.html

Hierocracy

Hard to tell for sure, but your chain indeed seems broken somehow. As far as I can tell, the openssl verify in the first case will check the chain and fail, while the second only will check the chain from the signing-ca.crt to the root (not needing the other certs, so just ignoring them) – Rhamnaceous

@JoachimIsaksson, OK, got it. How to debug this level by level? If I do "openssl verify -CAfile subordinate-ca.crt server.crt", should it fail or succeed? – Hierocracy

As far as I can tell, it should succeed if the server cert is a direct "child" of the subordinate-ca cert. – Rhamnaceous

@JoachimIsaksson, This is not true, verifying part of the chain without including the root will fail. Check the links I add to my question. But thanks anyway, your comments are helpful. – Hierocracy

Also ensure that you dos2unix the files first to sanitise them with LF endings before concatenation, this caught me out and manifests as "140017985005208:error:0906D066:PEM routines:PEM_read_bio:bad end line:pem_lib.c:809:" style error. – Bernina

Answer 1

The original order is in fact backwards. Certs should be followed by the issuing cert until the last cert is issued by a known root per IETF's RFC 5246 Section 7.4.2

This is a sequence (chain) of certificates. The sender's certificate MUST come first in the list. Each following certificate MUST directly certify the one preceding it.

See also SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch for troubleshooting techniques.

But I still don't know why they wrote the spec so that the order matters.

Galvanometer

For certificate bundles for use with Nginx, the order of the certificates in the bundle will need to be reversed, with the peer certificate first followed by the chain ending at the root CA. Consistency sure would be nice! nginx.org/en/docs/http/configuring_https_servers.html – Killer

actually, i believe my answer here is just wrong, and you are right; my understanding at the time was flawed. – Galvanometer

nginx and the standard match each other The RFC says says "The sender's certificate MUST come first in the list." and nginx says "The server certificate must appear before the chained certificates in the combined file: " – Granddaddy

It seems to be backwards, but in fact, it's in processing order. In order to check a signature, you first have to have the object which to check and which also holds the signature. That would be the leaf. – Oleomargarine

Answer 2

The current top answer is wrong on multiple levels.

First, it misses the main issue, which is simply that 'verify' does not verify chains of certificates in a single file in this way. Proof:

$ cp /etc/ssl/certs/Certigna_Root_CA.pem /tmp/poc.pem
$ echo "unverifiable garbage" >> /tmp/poc.pem
$ openssl verify /tmp/poc.pem 
/tmp/poc.pem: OK

Thus, of the 2 provided orderings in the original question, only the one which has a root certificate at the beginning will report "OK", because that invocation simply checks the root certificate alone, and that is already trusted by the system.

On the subject of ordering certificates, the order as specified in the standard ("The sender's certificate MUST come first in the list.") and that of nginx ("The server certificate must appear before the chained certificates in the combined file") match each other. This corresponds to the OP's first attempt using cat.

The actually correct answer is in the email-thread which was linked by the OP. I'll quote it here:

on the command line, things are unnecessarily difficult.

Here is what you need to do:

1. Split the chain file into one file per certificate, noting the order

2. For each certificate starting with the one above root:

2.1 Concatenate all the previous certificates and the root certificate to one temporary file (This example is for when you are checking the third certifate from the bottom, having already checked cert1.pem and cert2.pem

Unix:    cat cert2.pem cert1.pem root.pem > cert2-chain.pem
Windows: copy /A cert1.pem+cert1.pem+root.pem cert2-chain.pem /A

2.2 Run this command

         openssl verify -CAfile cert2-chain.pem cert3.pem

2.3 If this is OK, proceed to the next one (cert4.pem in this case)

Thus for the first round through the commands would be

Unix: cat root.pem > root-chain.pem Windows: copy /A root.pem root-chain.pem Both: openssl verify -CAfile root-chain.pem cert1.pem

And the second round would be

Unix: cat cert1.pem root.pem > cert1-chain.pem Windows: copy /A cert1.pem+root.pem cert1-chain.pem Both: openssl verify -CAfile cert1-chain.pem cert2.pem

Etc.

Further notes:

1. Step 1 (splitting the file) may be automated like so:

csplit -f cert- $file '/-----BEGIN CERTIFICATE-----/' '{*}'

2. openssl verify takes information about trust from your system (e.g. /etc/ssl/certs/) also, so if you really want to make sure that you're verifying correctly your invocation should be something like openssl verify -verbose -x509_strict -CAfile upto-cert-02 -CAPath nosuchdir cert-01 (where nosuchdir is a non-existing path, and upto-cert-02 is the concatenation of files cert-nn to certt02)

Jung

-CApath nosuchdir gives error: "verify: Not a directory: nosuchdir". nosuchdir should be an existing directory for this to work – Skirl

You get the "not a directory" error because newer version of openssl validate the argument and older ones didn't. However, newer versions also include a -no-CApath option that is a better choice. – Integrated