Using God to monitor Unicorn - Start exited with non-zero code = 1

Question

I am working on a God script to monitor my Unicorns. I started with GitHub's examples script and have been modifying it to match my server configuration. Once God is running, commands such as god stop unicorn and god restart unicorn work just fine.

However, god start unicorn results in WARN: unicorn start command exited with non-zero code = 1. The weird part is that if I copy the start script directly from the config file, it starts right up like a brand new mustang.

This is my start command:

/usr/local/bin/unicorn_rails -c /home/my-linux-user/my-rails-app/config/unicorn.rb -E production -D

I have declared all paths as absolute in the config file. Any ideas what might be preventing this script from working?

Bevus

Answer 1

I haven't used unicorn as an app server, but I've used god for monitoring before.

If I remember rightly when you start god and give your config file, it automatically starts whatever you've told it to watch. Unicorn is probably already running, which is why it's throwing the error.

Check this by running god status once you've started god. If that's not the case you can check on the command line what the comand's exit status is:

/usr/local/bin/unicorn_rails -c /home/my-linux-user/my-rails-app/config/unicorn.rb -E production -D; echo $?;

that echo will print the exit status of the last command. If it's zero, the last command reported no errors. Try starting unicorn twice in a row, I expect the second time it'll return 1, because it's already running.

EDIT:

including the actual solution from comments, as this seems to be a popular response:

You can set an explicit user and group if your process requires to be run as a specific user.

God.watch do |w|
  w.uid = 'root'
  w.gid = 'root'

  # remainder of config
end

Elodia

Interesting... this was a really good suggestion. I used the echo addition to read the output of the start and stop commands when executed outside of god. They both exited with a value of 0. However, when identical commands are executed by god, it exits with a value of 1. All processes are supposed to start as root... both god and the unicorns. I wonder if there is an overarching permissions error here. Does that seem like a plausible answer? – Bevus

yeah sounds like although god is running as root, it might be executing the command as another user? Looks like you can set the user god runs the commands as via: God.watch do |w| ... w.uid = 'root' w.gid = 'root' ... end – Elodia

Answer 2

My problem was that I never bundled as root. Here is what I did:

sudo bash
cd RAILS_ROOT
bundle

You get a warning telling you to never do this:

Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine.

But it was the only way I could get resque or unicorn to run with god. This was on an ec2 instance if that helps anyone.

Bacitracin

Answer 3

Add the log option has helped me greatly in debugging.

God.watch do |w|
  w.log = "#{RAILS_ROOT}/log/god.log"

  # remainder of config
end

In the end, my bug turned out to be the start_script in God was executed in development environment. I fixed this by appending the RAILS_ENV to the start script.

start_script = "RAILS_ENV=#{ENV['RACK_ENV']} bundle exec sidekiq -P #{pid_file} -C #{config_file} -L #{log_file} -d"

Anthropophagy