How do you set a default root object for subdirectories for a statically hosted website on Cloudfront?

Question

How do you set a default root object for subdirectories on a statically hosted website on Cloudfront? Specifically, I'd like www.example.com/subdir/index.html to be served whenever the user asks for www.example.com/subdir. Note, this is for delivering a static website held in an S3 bucket. In addition, I would like to use an origin access identity to restrict access to the S3 bucket to only Cloudfront.

Now, I am aware that Cloudfront works differently than S3 and amazon states specifically:

The behavior of CloudFront default root objects is different from the behavior of Amazon S3 index documents. When you configure an Amazon S3 bucket as a website and specify the index document, Amazon S3 returns the index document even if a user requests a subdirectory in the bucket. (A copy of the index document must appear in every subdirectory.) For more information about configuring Amazon S3 buckets as websites and about index documents, see the Hosting Websites on Amazon S3 chapter in the Amazon Simple Storage Service Developer Guide.

As such, even though Cloudfront allows us to specify a default root object, this only works for www.example.com and not for www.example.com/subdir. In order to get around this difficulty, we can change the origin domain name to point to the website endpoint given by S3. This works great and allows the root objects to be specified uniformly. Unfortunately, this doesn't appear to be compatable with origin access identities. Specifically, the above links states:

Change to edit mode:

Web distributions – Click the Origins tab, click the origin that you want to edit, and click Edit. You can only create an origin access identity for origins for which Origin Type is S3 Origin.

Basically, in order to set the correct default root object, we use the S3 website endpoint and not the website bucket itself. This is not compatible with using origin access identity. As such, my questions boils down to either

1. Is it possible to specify a default root object for all subdirectories for a statically hosted website on Cloudfront?

2. Is it possible to setup an origin access identity for content served from Cloudfront where the origin is an S3 website endpoint and not an S3 bucket?

Daryl

I think this is now doable with Lambda@edge, using a function that redirects all URLs ending in / to /index.html I'll try it out on my website and report back the results and post the detailed configuration as an answer. – Modlin

January '22 - @ktutnik's answer here https://mcmap.net/q/157432/-how-do-you-set-a-default-root-object-for-subdirectories-for-a-statically-hosted-website-on-cloudfront seems to be the cleanest and easiest way to do this - still supports OAIs and can leave the bucket fully locked down. – Hautbois

Answer 1

UPDATE: It looks like I was incorrect! See JBaczuk's answer, which should be the accepted answer on this thread.

Unfortunately, the answer to both your questions is no.

1. Is it possible to specify a default root object for all subdirectories for a statically hosted website on Cloudfront?

No. As stated in the AWS CloudFront docs...

... If you define a default root object, an end-user request for a subdirectory of your distribution does not return the default root object. For example, suppose index.html is your default root object and that CloudFront receives an end-user request for the install directory under your CloudFront distribution:

http://d111111abcdef8.cloudfront.net/install/

CloudFront will not return the default root object even if a copy of index.html appears in the install directory.

...

The behavior of CloudFront default root objects is different from the behavior of Amazon S3 index documents. When you configure an Amazon S3 bucket as a website and specify the index document, Amazon S3 returns the index document even if a user requests a subdirectory in the bucket. (A copy of the index document must appear in every subdirectory.)

2. Is it possible to setup an origin access identity for content served from Cloudfront where the origin is an S3 website endpoint and not an S3 bucket?

Not directly. Your options for origins with CloudFront are S3 buckets or your own server.

It's that second option that does open up some interesting possibilities, though. This probably defeats the purpose of what you're trying to do, but you could setup your own server whose sole job is to be a CloudFront origin server.

When a request comes in for http://d111111abcdef8.cloudfront.net/install/, CloudFront will forward this request to your origin server, asking for /install. You can configure your origin server however you want, including to serve index.html in this case.

Or you could write a little web app that just takes this call and gets it directly from S3 anyway.

But I realize that setting up your own server and worrying about scaling it may defeat the purpose of what you're trying to do in the first place.

Letsou

The one problem I have with this is that do get this to work means you would have two (2) URLs capable of accessing your web site on s3. Your cloud front URL and your s3 url (bucket_name.s3-website-us-east-1.amazonaws.com) – Eruptive

Answer 2

There IS a way to do this. Instead of pointing it to your bucket by selecting it in the dropdown (www.example.com.s3.amazonaws.com), point it to the static domain of your bucket (eg. www.example.com.s3-website-us-west-2.amazonaws.com):

Thanks to This AWS Forum thread

Kaliningrad

@schickling is right. This should be the accepted answer. – Korykorzybski

Anyone know if this charges differently when having an s3 origin vs a web origin? – Wetmore

the problem with this is if you have an origin path other than / and the user forgets the last / like /blog vs /blog/ then S3 will redirect to [origin path]/blog/ – Undercover

Does this work fine if I want to serve my whole website and files over HTTPS only? – Tantivy

Does it mean that the S3 has to be enabled as a web server? – Jolty

Do you need to have a /install folder inside this custom S3 bucket? or will the files be served from the root? How do you specifically set up the subfolder path? – Unassailable

It works fine if you want to serve over HTTPS only—just make sure to set up Viewer Protocol Policy to redirect HTTP to HTTPS. – Oldworld

BTW s3 will remove the query params for origin paths other than root, if someone has a solution for that this will be great :) – Elicia

@JBaczuk: I think www.example.com.s3-website-us-west-2.amazonaws.com should be www.example.com.s3-website.us-west-2.amazonaws.com, notice the dash to dot change after the s3-website. – Est

Yup, awesome! Also, do not specify the CloudFront's 'Default Root Object'. That will only make the root case work with the intuitive setup, but it will interfere with this awesome trick from working for all folders. – Brassy

Note: This doesn't work if you want to keep your S3 private and use "Restricted access" – Udelle

OP explicitly stated this approach wont work for him: "In order to get around this difficulty, we can change the origin domain name to point to the website endpoint given by S3. This works great and allows the root objects to be specified uniformly. Unfortunately, this doesn't appear to be compatable with origin access identities". AWS themselves seem to be recommending lamda@edge for this - aws.amazon.com/blogs/compute/… – Agrostology

If you use Terraform with the AWS provider, the attribute you want to use in the CloudFront settings is: website_endpoint (C.F. terraform.io/docs/providers/aws/d/s3_bucket.html). WARNING: you will also need to update the origin settings in your CloudFront distribution (C.F. terraform.io/docs/providers/aws/r/…): indeed, by default if nothing is specified it is an S3 Origin, but you now need a Custom Origin. – Unaccountedfor

This is not compatible Cloud Front - Origin Access Identity. You won't be able to restrict access to your S3 bucket this way. – Sway

this is a workaround. The correct solutions which protect your s3 bucket are https://mcmap.net/q/157432/-how-do-you-set-a-default-root-object-for-subdirectories-for-a-statically-hosted-website-on-cloudfront or https://mcmap.net/q/157432/-how-do-you-set-a-default-root-object-for-subdirectories-for-a-statically-hosted-website-on-cloudfront – Boneblack

you saved me from having to write some goofy lambda function to do something that every other static web server in the world has been doing for years. thank you. – Colleen

@rocketspacer there is a solution using aws s3api copy-object which works in S3 bucket with restricted access, please see my answer below. – Josselyn

For the detailed comparison of this solution vs the others, as well as the challenges here, see my answer below: stackoverflow.com/a/66977399 – Neeley

Answer 3

(New Feature May 2021) CloudFront Function

Create a simple JavaScript function below

function handler(event) {
    var request = event.request;
    var uri = request.uri;
    
    // Check whether the URI is missing a file name.
    if (uri.endsWith('/')) {
        request.uri += 'index.html';
    } 
    // Check whether the URI is missing a file extension.
    else if (!uri.includes('.')) {
        request.uri += '/index.html';
    }

    return request;
}

Read here for more info

Thumbprint

This works perfectly, and much cheaper than using Lambda. Here's an example of how to setup CF functions inside Serverless Framework deploy script (just replace the function code with the code/link in above answer). – Nigrify

This worked flawlessly - simply created a Function in the relevant section (left menu) of CF, then associated it with the default behaviour -> Viewer Request of my distribution. Hugo site now working as intended! – Hautbois

I have tried this solution but no joy. Can you see what I am doing wrong?#70717668 – Earplug

You good sir, YOU are the real MVP. I don't know why this isn't with the default AWS documentation. – Mammilla

@Mammilla - it is: docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/… – Patriciate

This seems to be causing a page refresh on a SPA due to manipulation of the request. Am I the only one experiencing this behaviour? – Trescott

Answer 4

Activating S3 hosting means you have to open the bucket to the world. In my case, I needed to keep the bucket private and use the origin access identity functionality to restrict access to Cloudfront only. Like @Juissi suggested, a Lambda function can fix the redirects:

'use strict';

/**
 * Redirects URLs to default document. Examples:
 *
 * /blog            -> /blog/index.html
 * /blog/july/      -> /blog/july/index.html
 * /blog/header.png -> /blog/header.png
 *
 */

let defaultDocument = 'index.html';

exports.handler = (event, context, callback) => {
    const request = event.Records[0].cf.request;

    if(request.uri != "/") {
        let paths = request.uri.split('/');
        let lastPath = paths[paths.length - 1];
        let isFile = lastPath.split('.').length > 1;

        if(!isFile) {
            if(lastPath != "") {
                request.uri += "/";
            }

            request.uri += defaultDocument;
        }

        console.log(request.uri);
    }

    callback(null, request);
};

After you publish your function, go to your cloudfront distribution in the AWS console. Go to Behaviors, then chooseOrigin Request under Lambda Function Associations, and finally paste the ARN to your new function.

Vrablik

There is a ready to deploy lambda function similar to that one: serverlessrepo.aws.amazon.com/applications/… – Scilicet

The problem here is that this function needs to be deployed to us-east-1, so if you have a company under strict GDPR regulation that doesn't allow a single bit outside Germany then this is not for you. – Ahab

Answer 5

I totally agree that it's a ridiculous problem! The fact that CloudFront knows about serving index.html as Default Root Object AND STILL they say it doesn't work for subdirectories (source) is totally strange!

The behavior of CloudFront default root objects is different from the behavior of Amazon S3 index documents. When you configure an Amazon S3 bucket as a website and specify the index document, Amazon S3 returns the index document even if a user requests a subdirectory in the bucket.

I, personally, believe that AWS has made it this way so CloudFront becomes a CDN only (loading assets, with no logic in it whatsoever) and every request to a path in your website should be served from a "Server" (e.g. EC2 Node/Php server, or a Lambda function.)

Whether this limitation exists to enhance security, or keep things apart (i.e. logic and storage separated), or make more money (to enforce people to have a dedicated server, even for static content) is up to debate.

---

Anyhow, I'm summarizing the possible solutions workarounds here, with their pros and cons.

1) S3 can be Public - Use Custom Origin.

It's the easiest one, originally posted by @JBaczuk answer as well as in this github gist. Since S3 already supports serving index.html in subdirectories via Static Website Hosting, all you need to do is:

1. Go to S3, enable Static Website Hosting
2. Grab the URL in the form of http://<bucket-name>.s3-website-us-west-2.amazonaws.com
3. Create a new Origin in CloudFront and enter this as a Custom Origin (and NOT S3 ORIGIN), so CloudFront treats this as an external website when getting the content.

Pros:

1. Very easy to set up.
2. It supports /about/, /about, and /about/index.html and redirect the last two to the first one, properly.

Cons:

1. If your files in the S3 bucket are not in the root of S3 (say in /artifacts/* then going to www.domain.com/about (without the trailing /) will redirect you to www.domain.com/artifacts/about which is something you don't want at all! Basically the /about to /about/ redirect in S3 breaks if you serve from CloudFront and the path to files (from the root) don't match.

2. Security and Functionality: You cannot make S3 Private. It's because CloudFront's Origin Access Identity is not going to be supported, clearly, because CloudFront is instructed to take this Origin as a random website. It means that users can potentially get the files from S3 directly, which might not be what you ever what due to security/WAF concerns, as well as the website actually working if you have JS/html that relies on the path being your domain only.

3. [maybe an issue] The communication between CloudFront and S3 is not the way it's recommended to optimize stuff.

4. [maybe?] someone has complained that it doesn't work smoothly for more than one Origin in the Distribution (i.e. wanting /blog to go somewhere)

5. [maybe?] someone has complained that it doesn't preserve the original query params as expected.

2) Official solution - Use a Lambda Function.

It's the official solution (though the doc is from 2017). There is a ready-to-launch 3rd-party Application (JavaScript source in github) and example Python Lambda function (this answer) for it, too.

Technically, by doing this, you create a mini-server (they call it serverless!) that only serves CloudFront's Origin Requests to S3 (so, it basically sits between CloudFront and S3.)

Pros:

1. Hey, it's the official solution, so probably lasts longer and is the most optimized one.
2. You can customize the Lambda Function if you want and have control over it. You can support further redirect in it.
3. If implemented correctly, (like the 3rd party JS one, and I don't think the official one) it supports /about/ and /about both (with a redirect from the latter without trailing / to the former).

Cons:

1. It's one more thing to set up.
2. It's one more thing to have an eye, so it doesn't break.
3. It's one more thing to check when something breaks.
4. It's one more thing to maintain -- e.g. the third-party one here has open PRs since Jan 2021 (it's April 2021 now.)
5. The 3rd party JS solution doesn't preserve the query params. So /about?foo=bar is 301 redirected to /about/ and NOT /about/?foo=bar. You need to make changes to that lambda function to make it work.
6. The 3rd party JS solution keeps /about/ as the canonical version. If you want /about to be the canonical version (i.e. other formats get redirected to it via 301), you have to make changes to the script.
7. [minor] It only works in us-east-1 (open issue in Github since 2020, still open and an actual problem in April 2021).
8. [minor] It has its own cost, although given CloudFront's caching, shouldn't be significant.

3) Create fake "Folder File"s in S3 - Use a manual Script.

It's a solution between the first two -- It supports OAI (private S3) and it doesn't require a server. It's a bit nasty though!

What you do here is, you run a script that for each subdirectory of /about/index.html it creates an object in S3 named (has key of) /about and copy that HTML file (the content and the content-type) into this object.

Example scripts can be found in this Reddit answer and this answer using AWS CLI.

Pros:

1. Secure: Supports S3 Private and CloudFront OAI.
2. No additional live piece: The script runs pre-upload to S3 (or one-time) and then the system remains intact with the two pieces of S3 and CF only.

Cons:

1. [Needs Confirmation] It supports /about but not /about/ with trailing / I believe.
2. Technically you have two different files being stored. Might look confusing and make your deploys expensive if there are tons of HTML files.
3. Your script has to manually find all the subdirectories and create a dummy object out of them in S3. That has the potential to break in the future.

PS. Other Tricks)

Dirty trick using Javascript on Custom Error

While it doesn't look like a real thing, this answer deserves some credit, IMO!

You let the Access Denied (404s turning into 403) go through, then catch them, and manually, via a JS, redirect them to the right place.

Pros

1. Again, easy to set up.

Cons

1. It relies on JavaScript in Client-Side.
2. It messes up with SEO -- especially if the crawler doesn't run JS.
3. It messes up with the user's browser history. (i.e. back button) and possibly could be improved (and get more complicated!) via HTML5 history.replace.

Neeley

Another thing I don't like about solution 1 is that I can't force the communication between CloudFront and S3 to be secure -- as I understand it, it'll have to be over HTTP. Maybe the traffic will be routed purely through Amazon's own network, in which case this wouldn't matter so much, but I'm not sure. – Santoyo

Answer 6

There is an "official" guide published on AWS blog that recommends setting up a Lambda@Edge function triggered by your CloudFront distribution:

Of course, it is a bad user experience to expect users to always type index.html at the end of every URL (or even know that it should be there). Until now, there has not been an easy way to provide these simpler URLs (equivalent to the DirectoryIndex Directive in an Apache Web Server configuration) to users through CloudFront. Not if you still want to be able to restrict access to the S3 origin using an OAI. However, with the release of Lambda@Edge, you can use a JavaScript function running on the CloudFront edge nodes to look for these patterns and request the appropriate object key from the S3 origin.

Solution

In this example, you use the compute power at the CloudFront edge to inspect the request as it’s coming in from the client. Then re-write the request so that CloudFront requests a default index object (index.html in this case) for any request URI that ends in ‘/’.

When a request is made against a web server, the client specifies the object to obtain in the request. You can use this URI and apply a regular expression to it so that these URIs get resolved to a default index object before CloudFront requests the object from the origin. Use the following code:

'use strict';
exports.handler = (event, context, callback) => {

    // Extract the request from the CloudFront event that is sent to Lambda@Edge
    var request = event.Records[0].cf.request;

    // Extract the URI from the request
    var olduri = request.uri;

    // Match any '/' that occurs at the end of a URI. Replace it with a default index
    var newuri = olduri.replace(/\/$/, '\/index.html');

    // Log the URI as received by CloudFront and the new URI to be used to fetch from origin
    console.log("Old URI: " + olduri);
    console.log("New URI: " + newuri);

    // Replace the received URI with the URI that includes the index page
    request.uri = newuri;

    // Return to CloudFront
    return callback(null, request);

};

Follow the guide linked above to see all steps required to set this up, including S3 bucket, CloudFront distribution and Lambda@Edge function creation.

Nuno

Answer 7

There is one other way to get a default file served in a subdirectory, like example.com/subdir/. You can actually (programatically) store a file with the key subdir/ in the bucket. This file will not show up in the S3 management console, but it actually exists, and CloudFront will serve it.

Moke

S3 converst subdir/ to subdir; when you try to upload the HTML. Also, when you try to access example.com/subdir/ it fails, and if you try to access example.com/subdir; it downloads the HTML file instead of rendering it. – Dignitary

This is actually one of the best approaches I found, as it works on all S3 distributions, without specific configuration, and does need to use lambda@edge (that generates extra redirect and slow down serving the pages while the lambda is executed). I disagree with @Dignitary comment. It works perfectly well, when use programatically. I did a small lambda function triggered by a S3 event. See my reply below. – Farrah

Thanks Johan and @Farrah this approach works! you can do this with awscli as well. – Josselyn

Ahh, I missed the connection that this will only work programatically. I had tested this solution via the web interface which had the behavior I indicated. I'll keep this in my back pocket next time I encounter this kind of issue. – Dignitary

Answer 8

aws CLI one command solution

If you don't want to use labmda, javascript or additional s3 configs... As johan-gorter and jeremie indicated index.html can be stored as an object with key subdir/. I validated this approach works and appears to be the simplest way to do this with aws CLI's s3api copy-object

aws s3api copy-object --copy-source bucket_name/subdir/index.html --key subdir/ --bucket bucket_name

Josselyn

Answer 9

Workaround for the issue is to utilize lambda@edge for rewriting the requests. One just needs to setup the lambda for the CloudFront distribution's viewer request event and to rewrite everything that ends with '/' AND is not equal to '/' with default root document e.g. index.html.

Sheets

More details regarding this approach here: aws.amazon.com/blogs/compute/… – Reni

unfortunately Lambda@Edge only works on us-east-1 region, source: github.com/awslabs/serverless-application-model/issues/635 – Ella

the Lambda@Edge function is only deployed on us-east-1, the function is replicated and runs at Edge locations worldwide and where it runs depends on the closest Edge location to the user. – Thetos

Answer 10

Another alternative to using lambda@edge is to use CloudFront's error pages. Set up a Custom Error Response to send all 403's to a specific file. Then add javascript to that file to append index.html to urls that end in a /. Sample code:

if ((window.location.href.endsWith("/") && !window.location.href.endsWith(".com/"))) {
    window.location.href = window.location.href + "index.html";
}
else {
    document.write("<Your 403 error message here>");
}

Culbertson

Answer 11

One can use newly released cloudfront functions and here is sample code.

Note: If you are using static website hosting, then you do not need any function!

Excitability

Answer 12

I know this is an old question, but I just struggled through this myself. Ultimately my goal was less to set a default file in a directory, and more to have the the end result of a file that was served without .html at the end of it

I ended up removing .html from the filename and programatically/manually set the mime type to text/html. It is not the traditional way, but it does seem to work, and satisfies my requirements for the pretty urls without sacrificing the benefits of cloudformation. Setting the mime type is annoying, but a small price to pay for the benefits in my opinion

Braswell

Answer 13

@johan-gorter indicated above that CloudFront serves file with keys ending by / After investigation, it appears that this option works, and that one can create this type of files in S3 programatically. Therefore, I wrote a small lambda that is triggered when a file is created on S3, with a suffix index.html or index.htm

What it does is copying an object dir/subdir/index.html into an object dir/subdir/

import json
import boto3

s3_client = boto3.client("s3")

def lambda_handler(event, context):

    for f in event['Records']:

        bucket_name = f['s3']['bucket']['name']
        key_name = f['s3']['object']['key']
        source_object = {'Bucket': bucket_name, 'Key': key_name}

        file_key_name = False

        if key_name[-10:].lower() == "index.html" and key_name.lower() != "index.html":
            file_key_name = key_name[0:-10]
        elif key_name[-9:].lower() == "index.htm" and key_name.lower() != "index.htm":
            file_key_name = key_name[0:-9]
        
        if file_key_name:
            s3_client.copy_object(CopySource=source_object, Bucket=bucket_name, Key=file_key_name)

Farrah

Answer 14

Further to the CloudFront Function solution posted above by ktutnik ... when the request was https://www.example.com/some-folder, I found that a particular type of single page app did not generate the paths to its resources properly.

The paths to images, CSS and JS were:

• https://www.example.com/lib/icomoon.css
• https://www.example.com/lib/lzwcompress.js

Rather than

• https://www.example.com/some-folder/lib/icomoon.css
• https://www.example.com/some-folder/lib/lzwcompress.js

What worked for me was a slight modification to add a trailing slash and use a redirect.

/*
  CloudFront function based on 
  https://github.com/aws-samples/amazon-cloudfront-functions/tree/main/url-rewrite-single-page-apps
  
  When URI is /folder-name/ 
  * URL rewrite to append index.html to the end of URLs that end in a slash
  * /folder-name/index.html
  
  When URI is /folder-name
  * 302 Redirect to URI with slash added to the end
  * /folder-name --- 302 ---> /folder-name/
*/
function handler(event) {
    var request = event.request;
    var uri = request.uri;
    var host = request.headers.host.value;

    // Check whether the URI is missing a file name.
    if (uri.endsWith('/')) {
        request.uri += 'index.html';
    }
    // Check whether the URI is missing a file extension.
    else if (!uri.includes('.')) {
        // Add slash to the end of the uri and redirect to that
        var newurl = `https://${host}${uri}/`
        var response = {
            statusCode: 302,
            statusDescription: 'Found',
            headers:
                { "location": { "value": newurl } }
            }

        return response;
    }

    return request;
}

Screwdriver