Missing required arguments: aws_access_key_id, aws_secret_access_key in server

Question

I keep getting this error whenever I am trying to start the server validate_options': Missing required arguments: aws_access_key_id, aws_secret_access_key (ArgumentError) I can't seem to solve the issue! I am almost sure that the issue is caused Carrierwave and fog gems to use amazon s3. But I am not sure exactly in which folder the issue is!

Here's my initializer/carrierwave.rb:

CarrierWave.configure do |config|
  config.fog_credentials = {
    :provider => 'AWS',                        # required
    :aws_access_key_id  =>   ENV["S3_ACCESS_KEY"],                        # required
    :aws_secret_access_key => ENV["S3_SECRET_KEY"]                      # required
  }
  config.fog_directory  = ENV["S3_BUCKET"]                          # required
end

here's my log:

=> Booting WEBrick
=> Rails 4.2.5 application starting in development on http://0.0.0.0:8080
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Exiting
/usr/local/rvm/gems/ruby-2.3.0/gems/fog-core-1.43.0/lib/fog/core/service.rb:244:in `validate_options': Missing required arguments: aws_access_key_id, aws_secret_access_key (ArgumentError)
        from /usr/local/rvm/gems/ruby-2.3.0/gems/fog-core-1.43.0/lib/fog/core/service.rb:268:in `handle_settings'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/fog-core-1.43.0/lib/fog/core/service.rb:98:in `new'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/fog-core-1.43.0/lib/fog/core/services_mixin.rb:16:in `new'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/fog-core-1.43.0/lib/fog/storage.rb:27:in `new'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/carrierwave-1.0.0/lib/carrierwave/uploader/configuration.rb:123:in `eager_load_fog'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/carrierwave-1.0.0/lib/carrierwave/uploader/configuration.rb:136:in `fog_credentials='
        from /home/ubuntu/workspace/config/initializers/carrierwave.rb:2:in `block in <top (required)>'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/carrierwave-1.0.0/lib/carrierwave/uploader/configuration.rb:158:in `configure'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/carrierwave-1.0.0/lib/carrierwave.rb:14:in `configure'
        from /home/ubuntu/workspace/config/initializers/carrierwave.rb:1:in `<top (required)>'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:268:in `load'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:268:in `block in load'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:240:in `load_dependency'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:268:in `load'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/engine.rb:652:in `block in load_config_initializer'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.5/lib/active_support/notifications.rb:166:in `instrument'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/engine.rb:651:in `load_config_initializer'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/engine.rb:616:in `block (2 levels) in <class:Engine>'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/engine.rb:615:in `each'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/engine.rb:615:in `block in <class:Engine>'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/initializable.rb:30:in `instance_exec'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/initializable.rb:30:in `run'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/initializable.rb:55:in `block in run_initializers'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:228:in `block in tsort_each'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:422:in `block (2 levels) in each_strongly_connected_component_from'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:431:in `each_strongly_connected_component_from'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:421:in `block in each_strongly_connected_component_from'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/initializable.rb:44:in `each'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/initializable.rb:44:in `tsort_each_child'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:415:in `call'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:415:in `each_strongly_connected_component_from'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:349:in `block in each_strongly_connected_component'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:347:in `each'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:347:in `call'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:347:in `each_strongly_connected_component'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:226:in `tsort_each'
        from /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:205:in `tsort_each'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/initializable.rb:54:in `run_initializers'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/application.rb:352:in `initialize!'
        from /home/ubuntu/workspace/config/environment.rb:5:in `<top (required)>'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:274:in `require'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:274:in `block in require'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:240:in `load_dependency'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:274:in `require'
        from /home/ubuntu/workspace/config.ru:3:in `block in <main>'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/rack-1.6.5/lib/rack/builder.rb:55:in `instance_eval'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/rack-1.6.5/lib/rack/builder.rb:55:in `initialize'
        from /home/ubuntu/workspace/config.ru:in `new'
        from /home/ubuntu/workspace/config.ru:in `<main>'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/rack-1.6.5/lib/rack/builder.rb:49:in `eval'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/rack-1.6.5/lib/rack/builder.rb:49:in `new_from_string'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/rack-1.6.5/lib/rack/builder.rb:40:in `parse_file'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/rack-1.6.5/lib/rack/server.rb:299:in `build_app_and_options_from_config'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/rack-1.6.5/lib/rack/server.rb:208:in `app'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/commands/server.rb:61:in `app'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/rack-1.6.5/lib/rack/server.rb:336:in `wrapped_app'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/commands/server.rb:139:in `log_to_stdout'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/commands/server.rb:78:in `start'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:80:in `block in server'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:75:in `tap'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:75:in `server'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.5/lib/rails/commands.rb:17:in `<top (required)>'
        from /home/ubuntu/workspace/bin/rails:9:in `require'
        from /home/ubuntu/workspace/bin/rails:9:in `<top (required)>'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/spring-2.0.0/lib/spring/client/rails.rb:28:in `load'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/spring-2.0.0/lib/spring/client/rails.rb:28:in `call'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/spring-2.0.0/lib/spring/client/command.rb:7:in `call'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/spring-2.0.0/lib/spring/client.rb:30:in `run'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/spring-2.0.0/bin/spring:49:in `<top (required)>'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/spring-2.0.0/lib/spring/binstub.rb:31:in `load'
        from /usr/local/rvm/gems/ruby-2.3.0/gems/spring-2.0.0/lib/spring/binstub.rb:31:in `<top (required)>'
        from /home/ubuntu/workspace/bin/spring:13:in `require'
        from /home/ubuntu/workspace/bin/spring:13:in `<top (required)>'
        from bin/rails:3:in `load'
        from bin/rails:3:in `<main>'

Candor

Are you setting S3_ACCESS_KEY and S3_SECRET_KEY in your environment? If so, how? It looks like it isn't working. – Enthrall

Answer 1

My Bug

I had a the exact same error: validate_options': Missing required arguments: aws_access_key_id, aws_secret_access_key (ArgumentError).

Bucket Restrictions and Limitations - Amazon Simple Storage Service https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

---

Suggestion

Now, I personally avoid the secrets.yml approach. I suggest using fog for production and file storage for development. For example:

photo_uploader.rb

class PhotoUploader < CarrierWave::Uploader::Base
  include CarrierWave::MiniMagick

  process resize_to_fill: [100, 100]
  # Choose what kind of storage to use for this uploader:
  if Rails.env.production?
    storage :fog
  else
    storage :file
  end

  # This is a sensible default for uploaders that are meant to be mounted:
  def store_dir
    "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end

  # Add a white list of extensions which are allowed to be uploaded.
  def extension_whitelist
    %w[jpg jpeg gif png]
  end
end

Gemfile

group :production do
  gem 'fog', '1.42'
end

carrierwave.rb

if Rails.env.production?

  CarrierWave.configure do |config|
    config.fog_provider = 'fog/aws'                        
    config.fog_credentials = {
      provider:              'AWS',                        
      aws_access_key_id:     ENV["S3_ACCESS_KEY"],        
      aws_secret_access_key: ENV["S3_SECRET_KEY"],
    }
    config.fog_directory  = ENV["S3_BUCKET"]              
  end

end

---

Environment Variables

If you still want to manage your environmental variables, maybe this can come in handy:

Temporary:

export S3_ACCESS_KEY=[...]
export S3_SECRET_KEY=[...]
export S3_BUCKET=[...]

After setting them you can print them to check if they got set:

printenv

You can also unset the variables:

unset S3_ACCESS_KEY
unset S3_SECRET_KEY
unset S3_BUCKET

Permanent:

Edit the ~/.bashrc or ~/.bash_profile to set the environmental variables permanently.

Hope this helps, have a great day!

Oliver

Answer 2

UPDATED

Declare your AWS credentials in one of your yml file (secrets.yml for example) like this:

AWS_ACCESS_KEY_ID: XXXXXXXXXXXXXXXXXXXX
AWS_SECRET_ACCESS_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
ARTIFACTS_REGION: eu-central-1
ARTIFACTS_BUCKET: some_bucket

Don't forget to add the file with your credentials to .gitignore

Then use your credentials in the carrierwave config file (carrierwave.rb) like this:

config.fog_credentials = {
  provider:              'AWS',            
  aws_access_key_id:     ENV['AWS_ACCESS_KEY_ID'],          
  aws_secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],             
  region:                ENV['ARTIFACTS_REGION']            
}

UPDATED

To manage your environment variables easier try Figaro gem. Maybe it'll expose your actual problem and you'll solve it.

Methenamine

Without the ENV? And how about ARTIFACTS_REGION what do I replace it by? – Candor

@Candor You declare variables in one of your *.yml files (for example secrets.yml) like this: AWS_ACCESS_KEY_ID: XXXXXXXXXXXXXXXXXXXX. Then you use it in config.fog_credentials block like this: aws_access_key_id: ENV['AWS_ACCESS_KEY_ID'] – Methenamine

Oh, I see. I thought you named your AWS credentials variables with wrong names. But it seems you didn't declare them at all. So I update my answer – Methenamine

I added everything just as you said but nothing worked! for some reason still saying ``validate_options': Missing required arguments: aws_access_key_id, aws_secret_access_key (ArgumentError)` when running the server! – Candor

@hekmat, if you double-checked that you added everything and it doesn't work then the problem could be anything. Do you declare your credentials for development environment exactly or for every environments? You need to check how your environment variables evaluate. If they evaluate to nils somehow then it's natural Rails miss them. – Methenamine

I actually tried to declare only in development and it did not work and then tried to have it for every environment it also has not work! The thing is that I was using paperclip before and then i deleted it and used carrierwave gem to use fog. I think what I will do is that I will delete carrierwave since it caused many problem to my app! and configure paperclip to use s3, hopefully everything returns to normal – Candor

You can try Figaro gem to manage your env variables. It could expose your problem. – Methenamine

Answer 3

Usually this results from either the values not being set in env, or accidentally being mis-spelled.

There are a few possible ways to manage this, but in development the most direct way is with export, like this:

export S3_ACCESS_KEY=[...]

If you fill in the [...] portion with your value (and repeat for the three distinct variables), it should be able to find everything.

You can also check the current values of these by using echo like this:

echo $S3_ACCESS_KEY

If you have set the value correctly, it should print the value to your terminal. If the value is not set, it will just print an empty line (if there is an empty line for any of the values, it would result in the error you are seeing).

Hypsography