"Request has expired" when using S3 with Active Storage

Question

I'm using ActiveStorage for the first time. Everything works fine in development but in production (Heroku) my images disappear without a reason.

They were showing ok the first time, but now no image is displayed. In the console I can see this error:

GET https://XXX.s3.amazonaws.com/variants/Q7MZrLyoKKmQFFwMMw9tQhPW/XXX 403 (Forbidden)

If I try to visit that URL directly I get an XML

<Error>
  <Code>AccessDenied</Code>
  <Message>Request has expired</Message>
  <X-Amz-Expires>300</X-Amz-Expires>
  <Expires>2018-07-24T13:48:25Z</Expires>
  <ServerTime>2018-07-24T15:25:37Z</ServerTime>
  <RequestId>291D41FAC6708334</RequestId>      
  <HostId>lEVGuwA6Hvlm/i40PeXaje9SEBYks9+uk6DvBs=</HostId>
</Error>

This is what I have in the view

<div class="cover" style="background-image: url('<%= rails_representation_path(experience.thumbnail) %>')"></div>

This is what I have in the model

def thumbnail
  self.cover.variant(resize: "300x300").processed
end

In simple words, I don't want images to expire but to be always there.

Thanks

Haemoglobin

Answer 1

ActiveStorage does not support non-expiring link. It uses expiring links (private), and support uploading files only as private on your service.

It was a problem for me too, and did 2 patches (caution) for S3 only, one simple ~30lines that override ActiveStorage to work only with non-expiring (public) links, and another that add an acl option to has_one_attached and has_many_attached methods.

Hope it helps.

Bosomed

Hi Dinatih, that looks great but I'm tryng to get my head around this: "ActiveStorage does not support non-expiring link". What does that mean? What I need is a super simple: images are always there when people load the page. They don't expire, nothing fancy happens. People visit the website and they see images. Why can't I upload public images using ActiveStorage? – Haemoglobin

Looks like ActiveStorage was not built with something like that in mind. Same for performance - links to assets generate lots of redirects, latency, and make using a CDN very difficult. – Waves

Answer 2

Your question doesn't say so, but it's common to use a CDN like AWS CloudFront with a Rails app. Especially on Heroku you probably want to conserve compute power.

Here is what happens in that scenario. You render a page as usual, and all the images are requested from the asset host, which is the CDN, because that's how it is configured to integrate. Its setup to fetch anything it doesn't find in cache from origin, which is your application again.

First all image requests are passed through. The ActiveStorage controller creates signed URLs for them, and the CDN passes them on, but also caches them.

Now comes the problem. The signed URL expires in 5 minutes by default, but the CDN caches usually much longer. This is because usually you use digest assets, meaning they are invalidated not by time but by name, on any change.

The solution is simple. Increase the expiry of the signed URL to be longer than the cache's TTL. Now the cache drops the cached signed URL before it becomes invalid.

Set the URL expiry using ActiveStorage::Service.url_expires_in in 5.2 or directly in Rails.application.config.active_storage.service_urls_expire_in in an initializer see this answer for details.

To set cache TTL in CloudFront: open the AWS console, pick the distribution, open the Behavior tab, scroll down to these fields:

Then optionally issue an invalidation to force re-caching of all contents.

Keep in mind there is a security trade-off. If the image contents are private, then they don't belong into a CDN most likely, and shouldn't have long lasting temp URLs either. In that case choose a solution that exempts attachments from CDN altogether. Your application will have to handle the additional load of signing all attached assets' URLs on top of rendering the relevant page.

Further keep in mind, that this isn't necessarily a good solution, but more of a workaround. With the above setup you will cache redirects, and the heavier requests will hit your storage bucket directly. The usual scenario for CDNs is large media, not lightweight redirects. You do relieve the app of handling a lot of requests though. How much that is a valid optimization should be looked into.

Reluctant

Answer 3

I had this same issue, but after I corrected the time on my computer, the problem was resolved. It was a server time difference, that the aws servers did not recognize.

Velvetvelveteen

Answer 4

In production.rb change:

config.active_storage.service = :local

To:

config.active_storage.service = :amazon

Should match AWS/Amazon whatever you defined it as in storage.yml

Opium

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Owner