Storing Amazon S3 credentials in .bashrc environemental variables causes Rails app to fail

Question

I appologise in advance for the length of this post....

I'm developing a rais app that uses paperclip to store stuff on Amazon S3. The app is hosted on Heroku. I'm developing on Ubuntu Karmic.

The problem that I am about to describe occurs in development (on my localhost) and production (on Heroku).

---

The standard way of passing S3 creds to paperclip is by putting them in config/s3.yml like so:

access_key_id: 12345678
secret_access_key: 903490409fdf09fshsfdoif/43432

When I do this, everything works just fine. But this makes it difficult to share my code with others so Heroku suggest an alternative method - http://docs.heroku.com/config-vars.

They advise that you should put your S3_KEY and S3_SECRET into your .bashrc like so:

S3_KEY=12345678
export S3_KEY
S3_SECRET=903490409fdf09fshsfdoif/43432
export S3_SECRET

They then suggest that you create config/initializers/s3.yml (note the slightly different path) and put the following into that file:

AWS::S3::Base.establish_connection!(
  :access_key_id     => ENV['S3_KEY'],
  :secret_access_key => ENV['S3_SECRET']
)

BUT, When I do this, paperclip throws a wobbler and spits out the following error message:

undefined method `stringify_keys' for #<String:0xb6d6c3f4>

/vendor/plugins/paperclip/lib/paperclip/storage.rb:176:in `parse_credentials'
/vendor/plugins/paperclip/lib/paperclip/storage.rb:138:in `extended'
/vendor/plugins/paperclip/lib/paperclip/storage.rb:137:in `instance_eval'
/vendor/plugins/paperclip/lib/paperclip/storage.rb:137:in `extended'

.... other stuff 

So clearly it's all kicking off inside the storage.rb module. Stepping through the stack trace:

The parse_credentials method on Line 176 is flagged - here's the call as it appears in the code:

def parse_credentials creds
  creds = find_credentials(creds).stringify_keys
  (creds[RAILS_ENV] || creds).symbolize_keys
end

The parse_credentials method attempts to call another method, find_credentials, and this is where I believe the problem lies. Heres the code for find_credentials:

def find_credentials creds
    case creds
    when File
      YAML::load(ERB.new(File.read(creds.path)).result)
    when String
      YAML::load(ERB.new(File.read(creds)).result)
    when Hash
      creds
    else
      raise ArgumentError, "Credentials are not a path, file, or hash."
    end
end

I can't see how the find_credentials method is equipped to read values from my .bashrc file. It's got two cases where it can read from YAML and one where it's looking for a hash.

My model references the credentials like so:

  has_attached_file :photo,
                (some code removed)
                :s3_credentials => "#{RAILS_ROOT}/config/initializers/s3.yml",

If I remove the :s3_credentials hash from the model, the stringify_keys error goes away and the rails console throws the error message that appears at the end of the find_credentials method: i.e. "Credentials are not a path, file, or hash".

So I'm stumped. I realise that this is possibly a question for the guys at Heroku (who I am actually going to email this link to in the hope that they can answer it) and it's also possibly a question for the doods at thoughtbot, but I thought that StackOverflow might be the best place to ask this as it's proven to be quite a reliable forum for me in the past.

As I said at the beginning, my app works fine when I take the standard approach of sticking my key and secret into config/s3.yml, but I would prefer to use the method that Heroku suggest because it makes things WAY easier for me and it means I can store my repo on my public github page for others to use without having to write any customer merge drivers in Git to keep my api keys out of the public domain.

Once again, sorry for the lengthy post, and if you've made it this far, I applaud you. Anyone got any ideas?

I've tried sticking the ENV variables in etc/bash.bashrc as well as ~/.bashrc and after rebooting, I still have the same problem. The problems occur on development machine as well as on Heroku. I've made sure to push my config-vars to Heroku as well.

I've been at it for 8 hours straight!! I'm gonna go watch the football now.

Answer 1

Rename the file config/initializers/s3.yml to config/initializers/s3.rb and give it a try.

Answer 2

After much searching I found the answer here - http://tammersaleh.com/posts/managing-heroku-environment-variables-for-local-development

The trick is to remove the S3.rb file altogether and just refer to the ENV variables in the model like so:

has_attached_file :photo, 
                  #...
                  :storage        => :s3, 
                  :bucket         => ENV['S3_BUCKET'],
                  :s3_credentials => { :access_key_id     => ENV['S3_KEY'], 
                                       :secret_access_key => ENV['S3_SECRET'] }

Anyway, David, thanks for your suggestion. I don't know if you want to update the Heroku docs to say that some users have had to do it this way. Thanks again though.