How to store data in S3 and allow user access in a secure way with rails API / iOS client?

Question

I am new to writing Rails and APIs. I need some help with S3 storage solution. Here's my problem.

I am writing an API for an iOS app where the users login with the Facebook API on iOS. The server validates the user against the token Facebook issues to the iOS user and issues a temporary Session token. From this point the user needs to download content that is stored in S3. This content only belongs to the user and a subset of his friends. This user can add more content to S3 which can be accessed by the same bunch of people. I guess it is similar to attaching a file to a Facebook group...

There are 2 ways a user can interact with S3... leave it to the server or get the server to issue a temporary S3 token (not sure of the possibilities here) and the user can hit up on the content URLs directly to S3. I found this question talking about the approaches, however, it is really dated (2 yrs ago): Architectural and design question about uploading photos from iPhone app and S3

So the questions:

• Is there a way to limit a user to access only some content on S3 when a temporary token is issued? How can I do this? Assume there's... say 100,000 or more users.
• Is it a good idea to let the iOS device pull this content out directly?
• Or should let the server control all content passing (this solves security of course)? Does this mean I have to download all content to server before handing it down to the connected users?
• If you know rails... can I use paperclip and aws-sdk gems to achieve this kinda setup?

Apologies for multiple questions and I appreciate any insight into the problem. Thanks :)

Tirzah

found this and thought i would comment for others looking docs.aws.amazon.com/AmazonS3/latest/dev/… – Wholesale

Answer 1

Using the aws-sdk gem, you can get a temporary signed url for any S3 object by calling url_for:

s3 = AWS::S3.new(
  :access_key_id => 1234,
  :secret_access_key => abcd
)
object = s3.buckets['bucket'].objects['path/to/object']
object.url_for(:get, { :expires => 20.minutes.from_now, :secure => true }).to_s

This will give you a signed, temporary use URL for only that object in S3. It expires after 20 minutes (in this example), and it's only good for that one object.

If you have lots of objects the client needs, you'll need to issue lots of signed URLs.

Or should let the server control all content passing (this solves security of course)? Does this mean I have to download all content to server before handing it down to the connected users?

Note that this doesn't mean the server needs to download each object, it only needs to authenticate and authorize specific clients to access specific objects in S3.

API docs from Amazon: https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#RESTAuthenticationQueryStringAuth

Endorsement

Thanks for that @ejdyksen. The solution I've devised used exactly that (I haven't updated the question with my answer)! So my solution was to do authenticated URLs for GET requests. However, when a user contributes content, he creates resources to a specific /bucket/user/objectname location using federated IAM token (temporary credentials that expire) with the policy attached to allow for /bucket/user/* write access. so no user in the system can do harm to other users' content. Seems to work okay. Appreciate your answer. – Tirzah

It's just a temporary url, not one time url. – Eachern

@okwap you're right. "One time" is not very accurate, as it can be used an unlimited number of times before the expiration. I updated the answer to reflect that. – Endorsement

If you're using v2 of the aws-sdk-ruby, note that the methods are somewhat different : docs.aws.amazon.com/sdkforruby/api/Aws/S3/… – Pintsize

Isn't it a risk that the user can see my access key? There is also the secret key (converted) – Posting

Users will be able to see your AWS credentials by inspecting the link. Even if they are not technical, they can easily the authenticated URL on an S3 error page when something goes wrong. You can get around this by streaming/downloading the file to your server and then serving it to the client. – Phosphoprotein

@Phosphoprotein the whole point is that the file doesn't have to hit your server. The link does not contain your AWS credentials. It might contain the ACCESS_KEY_ID (I don't remember off the top of my head), but that is not intended to be a secret. – Endorsement

@Endorsement you're right, I just verified that the URL only contains the AWS_ACCESS_KEY_ID. I originally thought that AWS_SECRET_ACCESS_KEY was also displayed but it isn't. – Phosphoprotein

thanks @Endorsement for this solution. I am working on a rails application and working on integrating s3 with rails application I have multiple module like images, profile_pics etc. all these object dont need to be protected but I have upload folder which container the files which I like to protect do you have idean how I can do this via s3? – Chrome

Answer 2

The above answers use the old aws-sdk-v1 gem rather than the new aws-sdk-resources version 2.

The new way is:

aws_resource = Aws::S3::Resource::new
aws_resource.bucket('your_bucket').object('your_object_key').presigned_url(:get, expires_in: 1*20.minutes)

where your_object_key is the path to your file. If you need to look that up, you would use something like:

s3 = Aws::S3::Client::new
keys = []
s3.list_objects(bucket: 'your_bucket', prefix: 'your_path').contents.each { |e| 
  keys << e.key
}

That information was startlingly difficult to dig up, and I almost just gave up and used the older gem.

Reference

http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Object.html#presigned_url-instance_method

Greenway

expires_in expects the data in the form of seconds, just make sure to convert that first. – Piccolo

"ArgumentError (expected :expires_in to be a number of seconds)" – Hefty