Amazon S3: using DNS alias to bucket + HTTPS at the same time

Question

I want to create an S3 bucket in the Frankfurt zone, and make the files accessible with the URL: https://files.stample.co/filename

So I want both HTTPS, and a custom DNS alias (CNAME) at the same time.

According to what I understand, Amazon has a wildcard certificate for URL https://*.s3.amazonaws.com.

HTTPS / SSL

So this wildcard will:

• Work for https://stample-files.s3.amazonaws.com
• Not work for https://files.stample.co.s3.amazonaws.com

So what I understand and confirmed by other StackOverflow posts is that if I want SSL to work I have to use a bucket name with no dot otherwise the Amazon certificat with wildcard won't match the bucket domain.

Using DNS alias / CNAME

On this S3 documentation, under Customizing Amazon S3 URLs with CNAMEs section:

Depending on your needs, you might not want "s3.amazonaws.com" to appear on your website or service. For example, if you host your website images on Amazon S3, you might prefer http://images.johnsmith.net/ instead of http://johnsmith-images.s3.amazonaws.com/.

The bucket name must be the same as the CNAME. So http://images.johnsmith.net/filename would be the same as http://images.johnsmith.net.s3.amazonaws.com/filename if a CNAME were created to map images.johnsmith.net to images.johnsmith.net.s3.amazonaws.com.

This seems to be for technical reasons because otherwise Amazon can't know the bucket we try to target:

Because Amazon S3 sees only the original host name www.example.com and is unaware of the CNAME mapping used to resolve the request, the CNAME and the bucket name must be the same.

So what I understand here is that for CNAME to work, we have to use dots in the bucketname.

Both together ?

If I use dots in bucket name:

• SSL won't work
• CNAME will work

If I don't use dots in bucket name:

• SSL will work
• CNAME won't work

I've tested both cases and could not make SSL and CNAME work fine together.

What can I do to make both work? It seems to me that what I want to achieve is not very fancy...

Hartsock

Answer 1

It seems it is currently not possible to do by using S3 only, but it is possible with CloudFront as it supports custom certificates.

CloudFront is not very expensive and can even be cheaper than S3 in some cases. It support custom certificates for free when using SNI (however it's not supported by older browsers like < IE7, < Chrome6, < Firefox 2.0)

HOWTO with CloudFront

I'll take as example that you want to use https://files.mydomain.com to point to an S3 bucket called mydomain-files (the bucket name does not mater and can contain dots).

Custom certificate is required

According to "Michael - sqlbot" anwser, it is required to use a custom certificate. My initial assumption was that using a CNAME will permit to use Amazon S3 wildcard certificate while using my custom domain but this was false: a custom certificate is absolutly required, and is possible to setup with CloudFront only, not S3.

Get free certificate

You can use whatever certificate provider you want but here I take StartSSL (StartCom) which provide free SSL certificates (limited to one subdomain and 1 year however).

• Validate your ownership of domain mydomain.com
• Create certificate with domain files.mydomain.com
• Download the certificate (files.crt) and private key (files.key, encrypted with your custom password): they are in PEM format
• Decrypt the private key: openssl rsa -in files.key -out files.key
• Generate certificate chain with StartSSL files from here: cat sub.class1.server.ca.pem ca.pem >> chain.crt

Upload certificate to AWS

• Install AWS CLI to upload the certificate (note the key must be unencrypted, the certificate be in PEM format, and the certificate chain is required for CloudFront). You have to choose a name and a path (choose what you want but the path should start with /cloudfront/
• Upload your certificate to AWS for Cloudfront usage, like documented here: aws iam upload-server-certificate --server-certificate-name CUSTOM_CERTIFICATE_NAME --certificate-body file://files.crt --private-key file://files.key --certificate-chain file://chain.crt --path /cloudfront/CUSTOM_PATH/

Configure CloudFront

• Create a new Web Distribution
• Use CNAME: files.mydomain.com
• Select the "Custom SSL certificate" radio button and select your certificate (CUSTOM_CERTIFICATE_NAME you choose when uploading)
• Select your S3 bucket as CloudFront distribution origin
• Validate and wait for deployment to complete: you should access your bucket files with url like https://xyzxyzxyz.cloudfront.net/file

Configure DNS

• Open your mydomain.com DNS configuration
• Add the CNAME: files IN CNAME xyzxyzxyz.cloudfront.net
• Wait for DNS to propagate (see DNS TTL) (can be fast if new DNS entry)

End

You should now be able to access your files with https://files.mydomain.com/file. The certificate will be your custom certificate generated for files.mydomain.com so everything will work fine.

Hartsock

I suspect this is for political reasons and may permit Amazon to charge more the users that need this feature. I believe you are being pessimistic on that point. Enabling support for custom SSL certs adds substantial technical complexity and is arguably outside S3's core competency. In some configurations, you will pay less with CloudFront. In us-east-1 and us-west-2, downloads from S3 directly are $0.09/GB, but from CloudFront are $0.085/GB (and the transfer from S3 to CloudFront is free). – Charlatan

thanks for these details Michael, I've edited my anwser and I'm sure it will be helpful to many – Hartsock

Answer 2

You can potentially do both, depending on what you mean by that, but it requires a change in your approach and an understanding of some S3 internals, or an understanding of why what you are trying can't possibly be expected to work.

First, to clarify, when I say you can do both, I assume you do not expect that you could create a bucket called "example.com" and then be able to access the bucket as https://example.com.

If that's what you want or expect, you are missing out on some fundamentals of SSL.

For a web server to offer SSL, it must have an SSL certificate signed by a trusted Certificate Authority. This certificate not only contains a public key used for encryption, it also contains the hostname for which it is valid, and will not work for a different hostname. SSL not only provides encryption, it also provides assurance that the web site you reached is indeed the web site you believe it to be. For s3, the hostname in the certificate is *.s3.amazonaws.com and some regional variants, which I'll explain below. The rules for hostname matching require that the * not match anything with a dot in it, so *.s3.amazonaws.com does match example-bucket.s3.amazonaws.com (no dot in the hostname component that aligns with the *) but it does not match example.com.s3.amazonaws.com because * does not match example.com since it contains a dot. These rules are imposed by browsers, and this isn't a limitation in S3.

Now, if you are trying to use https://example.com to access your bucket, S3 alone won't enable this, because example.com by itself isn't even remotely similar to the hostname on S3's SSL certificate, which is *.s3.amazonaws.com. You not only need to purchase an SSL certificate from a Certificate Authority that has been signed for use with "example.com," but you also need to install it on the web server... which, in this case, is S3... and S3 doesn't support this.

Amazon CloudFront, however, does. You can configure a CloudFront distribution as a front-end to your bucket, and install your own SSL certificate on CloudFront, accessing the bucket over HTTPS that way. In this configuration, it doesn't matter what your bucket name is -- it doesn't even have to match your domain name, because you can configure CloudFront to use whatever bucket you want. The browser fetches content through CloudFront, which in turn pulls the content from the bucket.

Now... assuming that isn't what you're asking... assuming that you don't expect your own domain to work in SSL with Amazon's certificate... you can configure a bucket with dotted name, allowing you to create a CNAME for hosting your "example.com" content... and still access the bucket with HTTPS. However, you can't access it with SSL using the CNAME, for reasons explained above... because your hostname doesn't match the one on S3's SSL cert.

However, if you want to access a bucket with dots in the name over SSL, you can do this using the regional endpoint where your bucket was provisioned.

The "example.com" bucket in the us-west-2 region would be addressed by putting the bucket name in the first part if the path instead if the beginning of the hostname... like this:

https://s3-us-west-2.amazonaws.com/example.com/path/to/file.jpg

Each S3 region has at least one regional endpoint that works this way. For the "US-Standard" region, the endpoints are "s3.amazonaws.com" which geographically routes to the main site in Virginia or the mirror site in Oregon, "s3-external-1.amazonaws.com" which routes only to Virginia, or "s3-external-2.amazonaws.com" which is largely undocumented but routes to the mirror in Oregon.

So, it is possible to access resources in a bucket with dots in the name, over HTTPS, but you have to know the bucket's region... though you cannot simply access it with https://your-cname.example.com, because of the way SSL works, in general.

Charlatan

thanks Michael. I've added my own answer explaining how to do it with CloudFront for documentation purpose. Actually I was aware of all you said, except I was thinking that the Amazon S3 wildcard certificate may match my custom domain in case the CNAME redirects to S3, but it is not the case – Hartsock

It doesn't match, specifically because the hostname in the browser's address bar -- not intermediate hostnames that may be used in DNS to resolve the destination -- is what the browser uses to validate the certificate presented by the server. CloudFront uses Server Name Identification (SNI), where the browser provides the hostname it expects to see in the cert, and that's how CloudFront can offer up the right cert for any given request, assuming it has such a cert to offer. – Charlatan