Environment Variables or YAML config files

Question

The background:
Step 1 -> We have a box that runs unit and functional tests of an application by running it in test mode with a specific configuration.
Step 2 -> Upon success of Step 1, we run integration tests of an application by running it in test mode with different configuration set, in another box.
Step 3 -> Upon success of step 2, we run the performance tests of an application by running it in production mode, in the performance test box.
Step 4 -> Upon success of step 3, the build is considered stable and the UAT box is updated with that code base and the application is run in production mode, for the customer review and feedback. Step 5 -> With GO from the customer, the production box is updated with the code base.

Now, from above steps we observe that in steps 1 and 2, while the application runs in test mode it has different configuration. Similar is the case with steps 3,4 and 5.

In such situations, what is the recommended practice? We were having YAML configuration files, but personally I felt that maintaining numerous configuration files doesn't make sense. And so, I changed from the practise of
"Config file per environment"
to
"Config file per rails mode, externalizing the variables to linux environment".

Am I on right track? Doesn't my action, simplify things?

What are the pros and cons of these two approaches?

Detach

Answer 1

In my experience, environment variables are the configuration option of last-resort. They absolutely have their place, but applications should generally check some other more reliable and explicitly intentional configuration option first. I would highly recommend loading configuration from a YAML file and only use an environment variable as a fallback. Always assume that your environment variable is going to apply to everything system-wide and assume that it's going to accidentally get unset or set to the wrong value at some point. i.e., your app shouldn't commit seppuku because some configuration directory got set to / and it doesn't have permissions to it (or worse you wipe your drive because the app ran as root). Or more likely, something like your RAILS_ENV was set to test when it should've been production and no one realized and now users are writing data to the wrong place or to /dev/null on account of all the 500s.

Personally, I like to drop logger.warn messages anytime I fallback to an environment variable for a configuration value.

With your precise issue, honestly, I'd probably pass in the configuration value for which environment to start on the command line.

Valine

Thanks. It is based on your statement that environment variables for app config is the last resorted action, and your advice to use YAML file to load the app configs, I came up with the design that I have blogged in my post. Your reply doubled my passion to get to the solution. Again Thanks! – Detach

Yes, this, a million times. Config files get messed with less than the Environment. Plus, when you re-read a config file, you get the latest-saved value. I can't tell you how many times I've had to explain to folks that you have to explicitly change your particular shell's current environment variable or kill your shell after changing .profile and restart it to get an updated environment variable loaded. It confuses people. Just use a config file. – Heirship

Answer 2

At my company, we actually have both, in a way.

We have a Rails app that can point at one of many different installations of another piece of software and use the API from that installation. To specify an installation, about 5 variables need to be set.

We had each of these variables as separate environment variables, but setting all of them got old really fast and we inevitably forgot one.

So now we have one environment variable we're calling ENV_TOKEN and we have yaml files that contain entries corresponding to the valid ENV_TOKEN variables, and code in config/initializers that sets ENV[key]=value.

So let's say I have variables "FOO" and "BAR" that I want to set to "one" and "two", respectively. I could create a yaml file that contains:

carolclarinet:
  FOO: one
  BAR: two

and then I would set my environment variable ENV_TOKEN to carolclarinet and FOO and BAR get set to one and two.

I have no idea if this is the best way of doing this, but it works for us.

ETA: Note that this is for developing and testing only, the installer of our software takes care of setting all this up so our customers never change any environment variables.

Tinder

I'm not sure if it's the best way or not either, but that's definitely an improvement. – Valine

Answer 3

After a lot of Googling in vain, discussions with some Rails folks and some brainstorming, I have made changes to the code such that I have "A config file per rails mode, externalizing the application configurations to a YML file, which in my case remains outside of the Rails environment"

Follow through the self-explanatory code snippets to get an understanding of how I achieve it in simple way. The quick explanation is that the code snippet in the environment.rb file reads the YAML file from system to copy all key value pairs to Rails' ENV hash. This ENV hash is available everywhere while/on/after application load.

File: config/environment.rb
# Mechanism to load all application related configurations
$CONFIG_FILE = "/var/myapp/config/jsconfig.yml"
require 'yaml'
APP_CONFIG = YAML.load_file($CONFIG_FILE)
APP_CONFIG.each do |key, value|
  ENV[key] = value
end

#3rd Party Server's (that my application is using) Configurations here...
3RD_PARTY_SERVER_URL = ENV['3rd_party_webservice_endpoint_url']
3RD_PARTY_SERVER_CREDENTIALS = {:username => ENV['3rd_party_server_username'], :password=> ENV['3rd_party_server_password']}


File: /var/myapp/config/jsconfig.yml
3rd_party_webservice_endpoint_url: url
3rd_party_server_username: username
3rd_party_server_password: password
myapp_db_url: jdbc:oracle:thin:@localhost:1521:XE
myapp_db_username: kartz
myapp_db_password: rails_savvy


File: /var/myapp/config/database.yml
development:
  adapter: oracle_enhanced
  url: <%= ENV['myapp_db_url'] %>
  username: <%= ENV['myapp_db_username'] %>
  password: <%= ENV['myapp_db_password'] %>
  encoding: utf8

test:
  adapter: oracle_enhanced
  url: <%= ENV['myapp_db_url'] %>
  username: <%= ENV['myapp_db_username'] %>
  password: <%= ENV['myapp_db_password'] %>
  encoding: utf8

production:
  adapter: oracle_enhanced
  url: <%= ENV['myapp_db_url'] %>
  username: <%= ENV['myapp_db_username'] %>
  password: <%= ENV['myapp_db_password'] %>
  encoding: utf8

More details of it can be found in my blog post: https://blog.codonomics.com/2011/02/externalizing-all-application-specific.html

Detach