A cron job for rails: best practices?

Question

What's the best way to run scheduled tasks in a Rails environment? Script/runner? Rake? I would like to run the task every few minutes.

Aged

For those coming here from Google, look beyond the accepted answer for better approaches. – Chicory

The whenever answer seems more reasonable than the accepted answer, which is an old hack. – Apricot

Please also beware that at least one answer assumes you have a certain gem installed. – Darrickdarrill

A couple of (what I found out to be) good practices are summarized here wisecashhq.com/blog/writing-reliable-cron-jobs – Gainor

In many cases cron jobs are a bad smell. Better write scheduler through sidekiq/resque (or other background worker), or write a daemon (less functional and monitorable). Cron jobs have at least few bad things: 1) locking for the one instance is a pain; 2) monitoring cannot be done easily; 3) exceptions handling should be written manually again; 4) not easy to restart; 5) all above issues easily solving by background workers. – Wingding

Rake = Ruby make. This is as extremely powerful tool. I recommend viewing Ryan Bates' Railscast on it- railscasts.com/episodes/66-custom-rake-tasks – Hematite

Answer 1

I'm using the rake approach (as supported by heroku)

With a file called lib/tasks/cron.rake ..

task :cron => :environment do
  puts "Pulling new requests..."
  EdiListener.process_new_messages
  puts "done."
end

To execute from the command line, this is just "rake cron". This command can then be put on the operating system cron/task scheduler as desired.

Update this is quite an old question and answer! Some new info:

• the heroku cron service I referenced has since been replaced by Heroku Scheduler
• for frequent tasks (esp. where you want to avoid the Rails environment startup cost) my preferred approach is to use system cron to call a script that will either (a) poke a secure/private webhook API to invoke the required task in the background or (b) directly enqueue a task on your queuing system of choice

Perplexed

What should the cron entry be for this case, so the OS knows the correct path to the rake task? – Chicory

NB: these days I'm using whenever (see Jim Garvin's answer), but a raw cron entry to run rake task would be something like: 30 4 * * * /bin/bash -l -c 'cd /opt/railsapp && RAILS_ENV=production rake cron --silent' – Perplexed

How do you call this from the console? I did load "#{Rails.root}/lib/tasks/cron.rake" and rake cron, but got NameError: undefined local variable or method `cron' for main:Object – Shroyer

The problem with this approach is the :environment dependency. We have a very heavy Rails application that takes long to start up, our Rake is called every minute and consume more resources starting up the Rails environment that executing the task. I'd love to have an already started up Rails environment to be called through the cron, have to be something between the controller approach and the rake environment one. – Cigarillo

What is the duration of this task? I'm using an if condition. I want to know how regularly this is run. I can't find any information about this in heroku website. – Befriend

@ShubhamChaudhary the cron I referenced was replaced by heroku scheduler quite some time back. see devcenter.heroku.com/articles/scheduler – Perplexed

im just really curious what the performance difference is compared to background gems like delayed job or sidekiq. i definitely see the benefits when you want to kind of organize big loads of background tasks though – Diella

@Diella running rake via cron is quite wasteful as it needs to load up the rails environment each time. dj or sidekiq keep an instance(s) running. So my preferred use of cron is to poke a webhook that will enqueue the required job. That's lightweight and you get the benefit of using cron as a scheduler and dj/sidekiq as a job queue. – Perplexed

@Perplexed ok but you may configure the rake tasks without loading the entire environment. aside from that, it would be pretty much the same? unfortunately, i havent looked at sidekiq or know what cron is doing internally so i can only make these guesses so far – Diella

Answer 2

I've used the extremely popular Whenever on projects that rely heavily on scheduled tasks, and it's great. It gives you a nice DSL to define your scheduled tasks instead of having to deal with crontab format. From the README:

Whenever is a Ruby gem that provides a clear syntax for writing and deploying cron jobs.

Example from the README:

every 3.hours do
  runner "MyModel.some_process"       
  rake "my:rake:task"                 
  command "/usr/bin/my_great_command"
end

every 1.day, :at => '4:30 am' do 
  runner "MyModel.task_to_run_at_four_thirty_in_the_morning"
end

Sukey

If it is run every minute the environment will be restarted everytime, which can be costly. It seems that github.com/ssoroka/scheduler_daemon avoids this. – Bloodthirsty

+1 for keeping the cron configuration in with your version control system – Skees

I think this is the best solution. If you're using rails, i think is better to write everything in rails. With this approach you can also forget about the cron task when changing servers, it moves with the app. – Deform

There is a great Railscast about Whenever that is really helpful (an older free version is up as well). – Glairy

@Tony, Whenever is basically a domain specific language for writing cron jobs. It compiles into regular cron syntax on your rails server and cron is what executes the jobs you specify (usually through rails runner). – Ave

Answer 3

In our project we first used whenever gem, but confronted some problems.

We then switched to RUFUS SCHEDULER gem, which turned out to be very easy and reliable for scheduling tasks in Rails.

We have used it for sending weekly & daily mails, and even for running some periodic rake tasks or any method.

The code used in this is like:

    require 'rufus-scheduler'

    scheduler = Rufus::Scheduler.new

    scheduler.in '10d' do
      # do something in 10 days
    end

    scheduler.at '2030/12/12 23:30:00' do
      # do something at a given point in time
    end

    scheduler.every '3h' do
      # do something every 3 hours
    end

    scheduler.cron '5 0 * * *' do
      # do something every day, five minutes after midnight
      # (see "man 5 crontab" in your terminal)
    end

To learn more: https://github.com/jmettraux/rufus-scheduler

Furculum

Up for rufus, as I've used it for both simple ruby projects or full rails apps. – Seltzer

Could you be a little more specific about the issues you ran into with Whenever? – Veolaver

the most ever great answer – Halloran

Answer 4

Assuming your tasks don't take too long to complete, just create a new controller with an action for each task. Implement the logic of the task as controller code, Then set up a cronjob at the OS level that uses wget to invoke the URL of this controller and action at the appropriate time intervals. The advantages of this method are you:

1. Have full access to all your Rails objects just as in a normal controller.
2. Can develop and test just as you do normal actions.
3. Can also invoke your tasks adhoc from a simple web page.
4. Don't consume any more memory by firing up additional ruby/rails processes.

Hyland

How to prevent others from access this task ? If the task taking cpu and called it frequently will cause problems. – Jdavie

Same way you would for any other controller. Make sure you implement some sort of security - AuthLogic or restful_authentication, then add role management on top of that with acl9 or something. You can then you your rails app to serve all tasks via restful interface with security and stuff, just like any other web service. This gives things like: you can use your Domain to authenticate and let "server" user run tasks that are scheduled in cron or Scheduled Tasks if you use windows. Also this allows you to access same things throgh script/runner if you need to. Continued in the next comment. – Wildermuth

Ran out of room above. The main advantage for me is that I'm the only admin level user at my work and I don't wont non-programmers/non-system admins with root access to kick of certain jobs. This setup lets you create a page in your rails app, where an authorized user (such as Office Manager or Inventory Manager) can log in and kick of certain jobs, without giving them access to script/runner or a cron job. And again if you go this route you can later use some other rails plugin or gem such as workling or background ( don't remeber exact names, see ruby-toolbox.com) – Wildermuth

I know this was a while ago, but this is definitely not the best way to do cron jobs anymore. Why go through the web interface, violating what the interface really represents, when there are plenty of other ways to access the Rails environment? – Lely

One of the problems with this is that it blocks an instance until that job finishes. If you have 3 instances running, and you kick off 3 jobs at the same time, your 3 instances will be busy and no other requests can be served. – Phlegmatic

The qualification "assuming your tasks don't take too long to complete" seems like a HUGE one. Wouldn't it be better to use an approach that is more generally useful, and not only in those cases where tasks are very quick? That way you're not constantly reevaluating whether this or that task needs to be rewritten using a different approach. – Dragonnade

This old question is the top google result for "rails cron". This answer is far from the best approach. Please see the other responses for more sane suggestions. – Sukey

Not the best way. You have many other ways to access Rails env through a cron job without calling a REST service. Rake approach is certainly better – Delanie

I've been doing this for a while and at a certain point is becomes a real liability. Now rewriting the tasks to just use rake/cron; having sinatra involved just unnecessarily complicated matters. – Waddell

Everybody please vote down this answer, so that it has a negative value! Then people will know it's a bad method :D – Barajas

six years later, I've decided to accept a different answer. – Aged

Long running processing should never be done in an HTTP request. This works only for very quick things. – Gainor

While I agree that this is not the best approach, I am not as radically against it as most people seem to be. As long as you keep some things in mind (keep execution time low, make sure the action cannot triggered by everyone) it's still a great method to quickly get a periodic task up and running. For further reading, I have written an article about it. – Pebrook

I like it, because it is very easy to set up (just had the same idea) and you don't need to learn the usage of a gem just cron + wget and ready. But I have to admit, that it is for small intranet purpose only, so no security risks and no timing problems. – Complexity

Answer 5

The problem with whenever (and cron) is that it reloads the rails environment every time it's executed, which is a real problem when your tasks are frequent or have a lot of initialization work to do. I have had issues in production because of this and must warn you.

Rufus scheduler does it for me ( https://github.com/jmettraux/rufus-scheduler )

When I have long jobs to run, I use it with delayed_job ( https://github.com/collectiveidea/delayed_job )

I hope this helps!

Mohair

Answer 6

script/runner and rake tasks are perfectly fine to run as cron jobs.

Here's one very important thing you must remember when running cron jobs. They probably won't be called from the root directory of your app. This means all your requires for files (as opposed to libraries) should be done with the explicit path: e.g. File.dirname(__FILE__) + "/other_file". This also means you have to know how to explicitly call them from another directory :-)

Check if your code supports being run from another directory with

# from ~
/path/to/ruby /path/to/app/script/runner -e development "MyClass.class_method"
/path/to/ruby /path/to/rake -f /path/to/app/Rakefile rake:task RAILS_ENV=development

Also, cron jobs probably don't run as you, so don't depend on any shortcut you put in .bashrc. But that's just a standard cron tip ;-)

Creighton

You can run the job as any user (just set the crontab entry for the user you want) but your are correct that the profile and login scripts won't run and you won't start in your home directory. So it's common to start the command with a "cd" as shown in @luke-franci's comment – Bolitho

Answer 7

I'm a big fan of resque/resque scheduler. You can not only run repeating cron-like tasks but also tasks at specific times. The downside is, it requires a Redis server.

Magnus

Answer 8

That is interesting no one mentioned the Sidetiq. It is nice addition if you already using Sidekiq.

Sidetiq provides a simple API for defining recurring workers for Sidekiq.

Job will look like this:

class MyWorker
  include Sidekiq::Worker
  include Sidetiq::Schedulable

  recurrence { hourly.minute_of_hour(15, 45) }

  def perform
    # do stuff ...
  end
end

Jobey

Answer 9

Both will work fine. I usually use script/runner.

Here's an example:

0 6 * * * cd /var/www/apps/your_app/current; ./script/runner --environment production 'EmailSubscription.send_email_subscriptions' >> /var/www/apps/your_app/shared/log/send_email_subscriptions.log 2>&1

You can also write a pure-Ruby script to do this if you load the right config files to connect to your database.

One thing to keep in mind if memory is precious is that script/runner (or a Rake task that depends on 'environment') will load the entire Rails environment. If you only need to insert some records into the database, this will use memory you don't really have to. If you write your own script, you can avoid this. I haven't actually needed to do this yet, but I am considering it.

Forwent

Answer 10

Use Craken (rake centric cron jobs)

Mournful

writing cron jobs is so hard, better download a gem for that – Eckblad

it's not hard - but having them stored in git and always up to date on deploy is a big plus when one work in a team. – Gainor

Answer 11

I use backgroundrb.

http://backgroundrb.rubyforge.org/

I use it to run scheduled tasks as well as tasks that take too long for the normal client/server relationship.

Glucoside

Answer 12

Using something Sidekiq or Resque is a far more robust solution. They both support retrying jobs, exclusivity with a REDIS lock, monitoring, and scheduling.

Keep in mind that Resque is a dead project (not actively maintained), so Sidekiq is a way better alternative. It also is more performant: Sidekiq runs several workers on a single, multithread process while Resque runs each worker in a separate process.

Platonism

That's a correct answer. Many can forget about the nice features, that sidekiq or resque are providing, like web interface for monitoring what is happening: number of jobs running, failed or scheduled, restart them easily, lock for unique workers, throttling and limiting, etc. – Wingding

Answer 13

Here's how I have setup my cron tasks. I have one to make daily backups of SQL database (using rake) and another to expire cache once a month. Any output is logged in a file log/cron_log. My crontab looks like this:

crontab -l # command to print all cron tasks
crontab -e # command to edit/add cron tasks

# Contents of crontab
0 1 * * * cd /home/lenart/izziv. whiskas.si/current; /bin/sh cron_tasks >> log/cron_log 2>&1
0 0 1 * * cd /home/lenart/izziv.whiskas.si/current; /usr/bin/env /usr/local/bin/ruby script/runner -e production lib/monthly_cron.rb >> log/cron_log 2>&1

The first cron task makes daily db backups. The contents of cron_tasks are the following:

/usr/local/bin/rake db:backup RAILS_ENV=production; date; echo "END OF OUTPUT ----";

The second task was setup later and uses script/runner to expire cache once a month (lib/monthly_cron.rb):

#!/usr/local/bin/ruby
# Expire challenge cache
Challenge.force_expire_cache
puts "Expired cache for Challenges (Challenge.force_expire_cache) #{Time.now}"

I guess I could backup database some other way but so far it works for me :)

The paths to rake and ruby can vary on different servers. You can see where they are by using:

whereis ruby # -> ruby: /usr/local/bin/ruby
whereis rake # -> rake: /usr/local/bin/rake

Tb

Answer 14

you can use resque and resque-schedular gem for creating cron, this is very easy to do.

https://github.com/resque/resque

https://github.com/resque/resque-scheduler

Ardeha

Answer 15

I have recently created some cron jobs for the projects I have been working on.

I found that the gem Clockwork very useful.

require 'clockwork'

module Clockwork
  every(10.seconds, 'frequent.job')
end

You can even schedule your background job using this gem. For documentation and further help refer https://github.com/Rykian/clockwork

Boy

Answer 16

Once I had to make the same decision and I'm really happy with that decision today. Use resque scheduler because not only a seperate redis will take out the load from your db, you will also have access to many plugins like resque-web which provides a great user interface. As your system develops you will have more and more tasks to schedule so you will be able to control them from a single place.

Eberhart

Answer 17

Probably the best way to do it is using rake to write the tasks you need and the just execute it via command line.

You can see a very helpful video at railscasts

Also take a look at this other resources:

• Rails Rake Tutorial

Atheism

I tried unsuccessfully to use the syntax in this tutorial. Task was not executed. – Darrickdarrill

Answer 18

I used clockwork gem and it works pretty well for me. There is also clockworkd gem that allows a script to run as a daemon.

Forgave

Answer 19

I'm not really sure, I guess it depends on the task: how often to run, how much complicated and how much direct communication with the rails project is needed etc. I guess if there was just "One Best Way" to do something, there wouldn't be so many different ways to do it.

At my last job in a Rails project, we needed to make a batch invitation mailer (survey invitations, not spamming) which should send the planned mails whenever the server had time. I think we were going to use daemon tools to run the rake tasks I had created.

Unfortunately, our company had some money problems and was "bought" by the main rival so the project was never completed, so I don't know what we would eventually have used.

Spiegelman

Answer 20

I Use script to run cron, that is the best way to run a cron. Here is some example for cron,

Open CronTab —> sudo crontab -e

And Paste Bellow lines:

00 00 * * * wget https://your_host/some_API_end_point

Here is some cron format, will help you

::CRON FORMAT::

Examples Of crontab Entries
15 6 2 1 * /home/melissa/backup.sh
Run the shell script /home/melissa/backup.sh on January 2 at 6:15 A.M.

15 06 02 Jan * /home/melissa/backup.sh
Same as the above entry. Zeroes can be added at the beginning of a number for legibility, without changing their value.

0 9-18 * * * /home/carl/hourly-archive.sh
Run /home/carl/hourly-archive.sh every hour, on the hour, from 9 A.M. through 6 P.M., every day.

0 9,18 * * Mon /home/wendy/script.sh
Run /home/wendy/script.sh every Monday, at 9 A.M. and 6 P.M.

30 22 * * Mon,Tue,Wed,Thu,Fri /usr/local/bin/backup
Run /usr/local/bin/backup at 10:30 P.M., every weekday. 

Hope this will help you :)

Solve

Answer 21

good_job is gaining more traction as an alternative to the other ActiveJob backends and it also supports cron-style recurring jobs. Here is how it's done basically:

# Configure cron with a hash that has a unique key for each recurring job
config.good_job.cron = {
  # Every 15 minutes, enqueue `ExampleJob.set(priority: -10).perform_later(42, name: "Alice")`
  frequent_task: { # each recurring job must have a unique key
    cron: "*/15 * * * *", # cron-style scheduling format by fugit gem
    class: "ExampleJob", # reference the Job class with a string
    args: [42, "life"], # positional arguments to pass; can also be a proc e.g. `-> { [Time.now] }`
    kwargs: { name: "Alice" }, # keyword arguments to pass; can also be a proc e.g. `-> { { name: NAMES.sample } }`
    set: { priority: -10 }, # additional ActiveJob properties; can also be a lambda/proc e.g. `-> { { priority: [1,2].sample } }`
    description: "Something helpful", # optional description that appears in Dashboard
  },
  another_task: {
    cron: "0 0,12 * * *",
    class: "AnotherJob",
  },
  # etc.
}

You can reade more about it here.

Gilolo