Amazon s3 static web hosting caching

Question

I'm using Amazon S3 webhosting for my static html,js,css (etc..) files. After replacing my index.html file, I still get the old version when consuming via the browser. I would like to set a default ttl to the bucket (and not to specific objects in it). I found this link: http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-values-specify.html#DownloadDistValuesDefaultTTL

but can't find the "Object Caching" setting in the dashboard. can someone point out where it is?

Soriano

The link you cited is for CloudFront, not S3. If you are not using CloudFront, this information is not relevant. If you are using CloudFront, this should be mentioned in the question. – Allude

Related: #22501965 – Fullfaced

Answer 1

Here's an Amazon S3 link that answers your question. According to Amazon, you can't set the cache control header for the entire bucket unless you use a third party tool (that page links to a few). One tool I ran across describes how to set the cache directives for objects submitted using a PUT request (or in bulk using their tool). See BucketExplorer for more info.

Here is a cut-paste of Amazon's instructions (since S.O. doesn't like to rely on external links that might change or disappear):

To add a Cache-Control or Expires header field to Amazon S3 objects using the Amazon S3 console

1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3.
2. In the Amazon S3 console, in the buckets pane, click the name of the bucket that contains the files.
3. In the list of objects, select the first object to which you want to add a header field.
4. Click Actions and click Properties.
5. In the right pane, expand Metadata.
6. Click Add More Metadata.
7. In the Key list, click Cache-Control or Expires, as applicable.
8. In the Value field, enter the applicable value:
9. For a Cache-Control field, enter: max-age=number of seconds that you want objects to stay in a CloudFront edge cache
10. For an Expires field, enter a date and time in HTML format.
11. Click Save.

If you want to add a header field to additional objects, click the name of the next object, and repeat steps 5 through 9.

Saccharose

Notice that you can manage this behavior at a folder level too using the same procedure. – Underbelly

Something that added clarification for me when reviewing the Amazon S3 link. If you want Cloudfront to cache but not the browser, tl;dr make sure the s3 objects have max-age of zero but that Cloudfront has a minimum TTL set – Awhirl

this is working fine but when we are pushing new build and file getting replaced its removing cache headers is there any way it can auto set the headers of those files – Statue

Answer 2

2022 AWS CLI V2 Method

The simplest way to achieve this is to use AWS CLI (S3). This can also be automated entirely for free using a tool like GitHub actions.

A static site should not have the cache-control set to a long-lived value on HTML files because the users will not see the updated version until the browser cache expires or manually busts their cache.

---

Due to AWS CLI restrictions, you have to do the following to set the cache for the whole bucket.

---

Generic Example

Upload the content and --delete old S3 content, and set cache-control on all content.

aws s3 sync [YOUR_LOCAL_SOURCE_CODE_PATH] s3://[BUCKET_NAME] --delete --cache-control max-age=31536000

Recursively remove cache-control headers from all HTML files and set the file back to type HTML.

aws s3 cp s3://[BUCKET_NAME] s3://[TO_BUCKET_NAME] --recursive --exclude "*" --include "*.html" --metadata-directive REPLACE --cache-control max-age:no-cache --content-type text/html

---

Notes

• TO_BUCKET_NAME is almost always the same as BUCKET_NAME
• If you modify an HTML file's metadata in AWS S3, you must also set the content-type, or it will automatically be set to a generic type causing the browser to download the file instead of rendering it in the browser.

---

Example

// delete old files and upload files from the local directory to the s3 bucket, and set the cache-control header on every file.
aws s3 sync ./out s3://www.test.com --delete --cache-control max-age=31536000

// copy all files and remove cache control header from only HTML files and set back to html content type
aws s3 cp s3://www.test.com s3://www.test.com --recursive --exclude "*" --include "*.html" --metadata-directive REPLACE --cache-control max-age:no-cache --content-type text/html

// bonus - if using CloudFront - small site can invalidate all cache (/*)
aws cloudfront create-invalidation --distribution-id=123ABCDEFG --paths "/*"

Apologetics

Hi Sean. Just a quick explanation about the edit I made that you then reverted. If you insist that you intend to comment on another answer, then you should comment on that answer when you have earned the privilege. Don't use answers to respond to answers - this isn't a threaded discussion forum. I figured the information you had was helpful to the OP and others who may come along with the same problem, so it was worthwhile tweaking your post to sound like an answer, and avoid getting deleted. – Shechem

Answer 3

If you use AWS CLI you can add --cache-control or --metadata directive per synced object

Here are my npm scripts:

 "scripts": {
    "prod": "NODE_ENV=production webpack --config ./webpack.config.prod.js",
    "sync": "aws s3 sync public/ s3://[BUCKET_NAME] --delete --cache-control max-age=31536000",
    "invalidate": "aws cloudfront create-invalidation --distribution-id [DISTRIBUTION_ID] --paths \"/*\"",
    "deploy": "npm run prod && npm run sync && npm run invalidate"
  },

Fowling

Answer 4

D. Woods's answer is good, but you need to set Cache control for every file separately. If you have thousands of files and different cache policy for specific extension, Good luck with that.

IMO better way if to configure it on CloudFront level, you can set up it using file patterns (e.g. different per extension)

1. Go to specific CloudFront distribution

2. Behaviors tab

3. Edit/Add behavior with pattern or Default - here you can configure patter per html/css etc. files.

4. Response headers policy

5. Create new

6. Create proper security headers, and focus on Custom headers

7. Add "Cache-Control" =

   max-age=86400 to configure browser cache

   no-cache, no-store if you want to instruct the browser not to use the cache.

If you do not configure this header, the browser will use default policy (different per browser).

Maida

Answer 5

Yes. In case you are using the CloudFront Distribution - you just need to check your invalidation and why your object invalidation didn't work properly. What is TTL value.

Do you use CDN?

Berners

Setting S3 Cache-Control is different to setting Cloudfront TTL which just controls how long your file stays in the Cloudfront cache. Nothing to do with Browser Caching. – Maggee