Did upstart or bash scripts change on Ubuntu 14.04? (Trying to boot sidekiq with upstart)

Question

I'm fairly new to bash scripting, but it feel like I'm really missing something basic here. I'm trying to a barely modified version of Mike Perham's upstart sidekiq script on an Ubuntu 14.04 machine, yet almost nothing is being evaluated as expected:

• export doesn't seem to be working
• source doesn't seem to evaluate my changed PATH variable in .bashrc or running the rbenv init command
• cd doesn't seem to change directories, unless the $(pwd) command isn't the correct way to evaluate it

Here's my modified script:

# /etc/init/sidekiq.conf - Sidekiq config

# This example config should work with Ubuntu 12.04+.  It
# allows you to manage multiple Sidekiq instances with
# Upstart, Ubuntu's native service management tool.

# change to match your deployment user
setuid deploy
setgid deploy

stop on (stopping workers or runlevel [06])

respawn
respawn limit 3 30

instance $index

script
# this script runs in /bin/sh by default
# respawn as bash so we can source in rbenv
exec /bin/bash <<EOT
  # use syslog for logging
  # exec &> /dev/kmsg

  # pull in system rbenv
  export HOME=/home/deploy
  echo "home is $HOME"
  source /home/deploy/.bashrc
  echo "path is $PATH"

  cd /home/deploy/domain_freek/current
  echo "user is $(whoami) and pwd is $(pwd) and rbenv is located at $(which rbenv)"
  exec bundle exec sidekiq -i ${index} -e production
EOT
end script

Here's the output I get in the upstart log file:

home is
path is /usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
user is deploy and pwd is / and rbenv is located at 
/bin/bash: line 12: exec: bundle: not found

Gemini

Try exec /bin/bash <<'EOT' (hard quotes around the marker). Output should make more sense. – Cumquat

Alright! Thanks, hard quotes helped. Now HOME is exported and cd actually changes the directory... Now I just gotta figure out how to properly load the .bashrc file, or initialize rbenv another way – Gemini

Answer 1

2 Changes made all the difference:

1) Add hard quotes to EOT in exec /bin/bash << 'EOT' (credit to Mat, thanks!)

2) Instead of loading .bashrc using source, add the rbenv lines from .bashrc directly to the upstart script. Replace source /home/deploy/.bashrc with:

export PATH="$HOME/.rbenv/bin:$PATH"
eval "$(rbenv init -)"
export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"

I have no idea why those two changes made the difference, and if that's related to a newer version of ubuntu, upstart, or bash. If anyone can explain, please chime in.

I've included my full working script for anyone just looking for the answer:

# /etc/init/sidekiq.conf - Sidekiq config

# This example config should work with Ubuntu 12.04+.  It
# allows you to manage multiple Sidekiq instances with
# Upstart, Ubuntu's native service management tool.
#
# See workers.conf for how to manage all Sidekiq instances at once.
#
# Save this config as /etc/init/sidekiq.conf then mange sidekiq with:
#   sudo start sidekiq index=0
#   sudo stop sidekiq index=0
#   sudo status sidekiq index=0
#
# or use the service command:
#   sudo service sidekiq {start,stop,restart,status}
#

description "Sidekiq Background Worker"

# no "start on", we don't want to automatically start
stop on (stopping workers or runlevel [06])

# change to match your deployment user
setuid deploy
setgid deploy

respawn
respawn limit 3 30

# TERM is sent by sidekiqctl when stopping sidekiq.  Without declaring these as normal exit codes, it just respawns.
normal exit 0 TERM

instance $index

script
# this script runs in /bin/sh by default
# respawn as bash so we can source in rbenv
exec /bin/bash << 'EOT'
  # use syslog for logging
  # exec &> /dev/kmsg

  # pull in system rbenv
  export HOME=/home/deploy
  echo "$HOME"
  #source /home/deploy/.bashrc
  export PATH="$HOME/.rbenv/bin:$PATH"
  eval "$(rbenv init -)"
  export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"
  echo "$PATH"

  cd /home/deploy/domain_freek/current
  echo "user is $(whoami) and pwd is $(pwd) and rbenv is located at $(which rbenv)"  
  exec bundle exec sidekiq -i ${index} -e production
EOT
end script

Gemini

Please submit a PR so we can get this fixed upstream! – Noddy

I would use Upstart's ENV support directly: upstart.ubuntu.com/cookbook/#env env HOME=/home/deploy – Noddy

I wish it was possible to source .bashrc from inside the Upstart script. It's incredibly annoying to have to put env vars in two places when provisioning apps. – Kerosene

Just added a issue in sidekiq to update the script to fix this issue : github.com/mperham/sidekiq/issues/2361 – Fusilier

Answer 2

The default .bashrc file on Ubuntu 14.04 has a couple lines to return immediately if the shell is running in non-interactive mode. When you remove those lines from your bashrc then source will work as expected in upstart.

~/.bashrc (Lines to remove)

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *) return;;
esac

Fullblown

I'm running 14.04.4 and the line I commented out of the .bashrc was [ -z "$PS1" ] && return. This allowed me to simply source the .bashrc in the upstart script. Thanks for the tip! – Blackfellow

Answer 3

Josh's solution didn't work for me on ubuntu 14.04 using rbenv. This however did work:

exec /bin/bash <<EOF
  export RBENV_ROOT=/home/ubuntu/.rbenv
  export RBENV_VERSION=2.2.2
  cd /var/www/app/current
  exec /home/ubuntu/.rbenv/bin/rbenv exec bundle exec sidekiq -i ${index} -e production
EOF

Trifling