Crontab in Amazon Elastic Beanstalk

Question

I am doing a cron tab in AWS - Elastic Beanstalk with Ruby on Rails 3, but I don't know what is wrong.

I have this code in my .ebextensions/default.config

container_commands:
  01remove_old_cron_jobs:
    command: "crontab -r || exit 0"
  02send_test_email:
    command: crontab */2 * * * * rake send_email:test
    leader_only: true

I receive this error:

Failed on instance with return code: 1 Output: Error occurred during build: Command 02send_test_email failed .

UPDATE 1

I tried next:

crontab.txt

*/2 * * * * rake send_email:test > /dev/null 2>&1

default.config

02_crontab:
  command: "cat .ebextensions/crontab.txt | crontab"
  leader_only: true

RESULT: No errors, but it does not work.

UPDATE 2

crontab.sh

crontab -l > /tmp/cronjob
#CRONJOB RULES
echo "*/2 * * * * /usr/bin/wget http://localhost/crontabs/send_test_email > /dev/null 2>&1" >> /tmp/cronjob
#echo "*/2 * * * * rake send_email:test > /dev/null 2>&1" >> /tmp/cronjob

crontab /tmp/cronjob
rm /tmp/cronjob
echo 'Script successful executed, crontab updated.'

default.config

02_crontab:
  command: "/bin/bash .ebextensions/crontab.sh"
  leader_only: true

RESULT: Works with url, but not with rake task.

Disinfectant

Answer 1

Updated for 2018

In order to get this to work on the latest version of Elastic Beanstalk, I added the following to my .ebextensions:

.ebextensions/0005_cron.config

files:
  "/etc/cron.d/mycron":
    mode: "000644"
    owner: root
    group: root
    content: |
      56 11 * * * root . /opt/elasticbeanstalk/support/envvars && cd /var/app/current && /opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test >> /var/app/current/log/cron.log 2>&1

commands:
  remove_old_cron:
    command: "rm -f /etc/cron.d/*.bak"

How I got there:

There are four main issues to confront when trying to cron a rake task in AWS EB:

1. The first hurdle is making sure all of your EB and Rails environment variables are loaded. I beat my head against the wall a while on this one, but then I discovered this AWS forum post (login may be required). Running . /opt/elasticbeanstalk/support/envvars loads all of your environment variables.

2. Then we need to make sure we cd into the current app directory using cd /var/app/current.

3. Next we need to know where to find the bundle and rake executables. They are not installed in the normal bin directories, but are located in a directory specific to your ruby version. To find out where your executables are located, ssh into your EB server (eb ssh) and then type the following:

   $ cd /var/app/current
   $ which bundle
   /opt/rubies/ruby-2.3.4/bin/bundle
   $ which rake
   /opt/rubies/ruby-2.3.4/bin/rake
   

   You could probably guess the directory based on your ruby version, but the above commands will let you know for sure. Based on the above, your can build your rake command as:

   /opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test
   

   NOTE: If you update your ruby version, you will likely need to update your cron config as well. This is a little brittle. I'd recommend making a note in your README on this. Trust me, six months from now, you will forget.

4. The fourth thing to consider is logging. I'd recommend logging to the same location as your other rails logs. We do this by tacking on >> /var/app/current/log/cron.log 2>&1 to the end of our command string.

Putting all of this together leads to a cron command string of:

. /opt/elasticbeanstalk/support/envvars && cd /var/app/current && /opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test >> /var/app/current/log/cron.log 2>&1

Finally, I referenced the latest AWS documentation to build an .ebextensions config file for my cron command. The result was the .ebextensions/0005_cron.config file displayed at the top of this answer.

Janik

Since you're loading in the environment variables, I think you should be able to use $RUBY_ROOT in place of the actual path (e.g. /opt/rubies/ruby-2.3.4) to avoid the eventual pain of "why hasn't this task been running?" when the Ruby version inevitably updates. – Minded

@Matt, Excellent idea. I have not had time to test it yet, but if you are confident, feel free to edit this answer. – Janik

/opt/rubies/ruby-current is linked to the current ruby version on recent Beanstalk ruby machines, so you can use that instead of a path for a specific ruby version. – Counterclaim

A more verbose command string but without hardwired paths: . $(/opt/elasticbeanstalk/bin/get-config container -k support_dir)/envvars && cd $(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir) && . $(/opt/elasticbeanstalk/bin/get-config container -k script_dir)/use-app-ruby.sh && bundle exec rake -T – Jedda

If it helps anyone, my Node.js Amazon Linux 2 environment didn't have the envvars file. get-config is an alternative, but it is not yet on Amazon Linux 2 (as of June 2020). A workaround that helped me is available here: github.com/awsdocs/elastic-beanstalk-samples/issues/… – Genova

Answer 2

I am having the same issue. Though I figured out that the reason that rake task doesn't run correctly on eb is because of RACK_ENV, RAILS_ENV and BUNDLE_WITHOUT
Defaults of eb:

RACK_ENV: production
RAILS_ENV: production
BUNDLE_WITHOUT: test:development

When the cron runs rake task, it runs in development mode, and gives gem not found error, as gems grouped in development are not installed.

you can see this by changing your cron a bit
from:

*/2 * * * * rake send_email:test > /dev/null 2>&1

to:

*/2 * * * * cd /var/app/current/ && /usr/bin/bundle exec /usr/bin/rake send_email:test > /tmp/cron_log 2>&1

and then checking the /tmp/cron_log file
To know the location of bundle and rake, run

which bundle
which rake

I tried setting RAILS_ENV in command in cron, but that didn't work aswell
One quick fix is to set

BUNDLE_WITHOUT to null

EDIT:

Finally I got it to work,

.ebextensions/.config

files:
  "/tmp/cron_jobs" :
    mode: "000777"
    content: |
      1 10 * * * cd /var/app/current/ && RACK_ENV=production rake some:task >> /var/app/current/log/cron_log 2>&1
    encoding: plain
container_commands:
  01_delete_cron_jobs:
    command: "crontab -r -u webapp || exit 0"
  02_add_cron_jobs:
    command: "crontab /tmp/cron_jobs -u webapp"
    leader_only: true
option_settings:
  - option_name: RAILS_ENV
    value: production
  - option_name: RACK_ENV
    value: production

Notice the '-u webapp' when removing and adding cron, this will run this cron under user webapp. The above will also run in production mode. And the output will be dumped in log/cron_log file.

If the above wont work then adding 'Bundle exec' before 'rake some:task' might work.

Crossfertilize

Answer 3

I've seen these used with separate files in .ebextensions, such as:

  02send_test_email:
    command: "cat .ebextensions/crontab | crontab"
    leader_only: true

I haven't gotten around to it yet, but I took note of this along the way. Let us know if this works.

This stackoverflow post has much more information

After Update 1/2:

Cron doesn't know where rake is. Your application runs from /var/app/current, and you need to be running bundle exec rake from that directory.

Elastic beanstalk is horrible with logging errors, to get this right, ssh to the machine and experiment until you have the commands right, then put this back into your cron script. You can even try and re-run some of the eb scripts as found in the logs, then reverse that into your ebextensions files.

Troxell

No works for me. i have a crontab.txt with */2 * * * * rake send_email:test > /dev/null 2>&1 and nothing. No errors, but no work. – Disinfectant

See Update 1 and Update 2. – Disinfectant

any example to running the bundle exec from /var/app/current? – Disinfectant