Use Fog with Ruby to generate a Pre-signed URL to PUT a file in Amazon S3

Question

I am using the Fog gem to generate presigned urls. I can do this successfully to get read access to the file. Here's what I do:

    fog_s3 = Fog::Storage.new({
          :provider                 => 'AWS',
          :aws_access_key_id        => key,
          :aws_secret_access_key    => secret
    })
    object_path = 'foo.wav' 
    expiry = Date.new(2014,2,1).to_time.to_i
    url = fog_s3.directories.new(:key => bucket).files.new(:key => object_path).url(expiry,path_style: true)

But this doesn't work when I try to upload the file. Is there a way to specify the http verb so it would be a PUT and not a GET?

EDIT I see a method: put_object_url which might help. I don't know how access it.

Thanks

EDIT based upon your suggestion:

It helped - it got me a PUT - not GET. However, I'm still having issues. I added content type:

    headers = { "Content-Type" => "audio/wav" }
    options = { path_style: true }
    object_path = 'foo.wav' 
    expiry = Date.new(2014,2,1).to_time.to_i  
    url = fog_s3.put_object_url(bucket,object_path, expiry, headers, options)

but the url does not contain Content-Type in it. When done from Javascript in HTML I get the Content-Type in the url and that seems to work. Is this an issue with Fog? or is my header incorrect?

Blinkers

Answer 1

I think put_object_url is indeed what you want. If you follow the url method back to where it is defined, you can see it uses a similar method underlying it called get_object_url here (https://github.com/fog/fog/blob/dc7c5e285a1a252031d3d1570cbf2289f7137ed0/lib/fog/aws/models/storage/files.rb#L83). You should be able to do something similar and can do so by calling this method from the fog_s3 object you already created above. It should end up just looking like this:

headers = {}
options = { path_style: true }
url = fog_s3.put_object_url(bucket, object_path, expires, headers, options)

Note that unlike get_object_url there is an extra headers option snuck in there (which you can use to do stuff like set Content-Type I believe).

Hope that sorts it for you, but just let me know if you have further questions. Thanks!

Addendum

Hmm, seems there may be a bug related to this after all (I'm wondering now how much this portion of the code has been exercised). I think you should be able to work around it though (but I'm not certain). I suspect you can just duplicate the value in the options as a query param also. Could you try something like this?

headers = query = { 'Content-Type' => 'audio/wav' }
options = { path_style: true, query: query }
url = fog_s3.put_object_url(bucket, object_path, expires, headers, options)

Hopefully that fills in the blanks for you (and if so we can think some more about fixing that behavior within fog if it makes sense to do so). Thanks!

Thundershower

May be a bug, honestly not sure how much/how well this code has been exercised (sorry about that). I'll update my answer with some more info/ideas. – Thundershower

geemus - you're a genius! the addendum did the trick. I needed to know how to add the headers in there. If you think its a bug, we could create an issue - or even a doc issue - for helping others? – Blinkers

I think it is probably a bug, though I'm not totally sure how I want to fix it (it is a little awkward because of reusing some of the code in more than one place). But the first step is definitely going to be an issue. If you can create one with some of these details that would be great (and we can discuss and sort out a fix over there). Thanks! – Thundershower

@Thundershower You're a damn wizard! I've been cocking around with getting this working for hours. I read through countless Amazon articles and blog posts on the matter, nothing was working. I thought to myself, "maybe Fog has already done this?" Found put_object_url and eventually found my way to this question. I can verify that the above works for generating signed URLs for PUT uploads with CORS from the browser. Thanks! – Sweetening

Answer 2

Instead of using the *put_object_url* might I suggest that you try using the bucket.files.create action which take a Fog file Hash attributes and return a Fog::Storage::AWS::File.

I prefer to break it down in a bit more steps, here is an example:

fog_s3 = Fog::Storage.new({
   :provider                 => 'AWS',
   :aws_access_key_id        => key,
   :aws_secret_access_key    => secret
})

# Define the filename
ext = :wav
filename = "foo.#{ext.to_s}"

# Path to your audio file?
path ="/"

# Define your expiry in the amount of seconds
expiry = 1.day.to_i

#Initialize the bucket to store too
fog_bucket = connection.directories.get(bucket)

file = {
   :key => "#{filename}",
   :body => IO.read("#{path}#{filename}"),
   :content_type => Mime::Type.lookup_by_extension(ext),
   :cache_control => "public, max-age=#{expiry}",
   :expires => CGI.rfc1123_date(Time.now + expiry),
   :public => true
}

# Returns a Fog::Storage::AWS::File
file = fog_bucket.files.create( file )

# Now to retrieve the public_url
url = file.public_url

Note: For subdir's checkout the :prefix option for a AWS bucket.

Fog File Documentation: Optional attributes... bottom of the page, :) http://rubydoc.info/gems/fog/Fog/Storage/AWS/File

Hopefully the example will help explain the steps in creating a fog file... Cheers! :)

Similitude

Hi Kevin, that example works well if I have the file locally on my server. However, I need to generate the URL and hand it off to my browser based client app. The client app then uploads the file using this URL. My server doesn't act as a proxy to AWS, but rather helps control access to the private buckets. Thx for the suggestion as I can use it in another part of the app. :-) – Blinkers