How to correctly implement recurring tasks with time/frequency set by user

Question

Users subscribe to emails containing the last videos, but they also set when to get those emails.

Subscription(user_id, frequency, day, time, time_zone)

user_id  |  frequency  |  day    |  time   |  time_zone
1        |  daily      |  null   |  16:00  |  GMT
2        |  weekly     |  friday |  11:00  |  UTC
3        |  weekly     |  monday |  18:00  |  EST

How can we send the emails at the exact time and frequency chosen by users in their time zone without screwing up (like sending double emails or missing time)

The only frequencies are daily and weekly, if daily then the day is null.

I use redis as a database for this, let me know how to do this the right way!

Punke

if you were using delayed_job - github.com/collectiveidea/delayed_job - you can set the run_at on each job to anything you want (match your Subscription time, one job per schedule) then when the job is completing have it enqueue itself to run again based on your Scheduler setting – Reconstitute

How exact do you have to be? If its of by a minute, is that acceptable? – Prickly

Answer 1

I'm going to expand on the answer of fmendez using the resque-scheduler gem.

First, let's create the worker that sends the emails

class SubscriptionWorker
  def self.perform(subscription_id)
    subscription = Subscription.find subscription_id

    # ....
    # handle sending emails here
    # ....

    # Make sure that you don't have duplicate workers
    Resque.remove_delayed(SubscriptionWorker, subscription_id)        

    # this actually calls this same worker but sets it up to work on the next
    # sending time of the subscription.  next_sending_time is a method that
    # we implement in the subscription model.
    Resque.enqueue_at(subscription.next_sending_time, SubscriptionWorker, subscription_id)
  end
end

In your subscription model, add a next_sending_time method to calculate the next time an email should be sent.

# subscription.rb
def next_sending_time
  parsed_time = Time.parse("#{time} #{time_zone}") + 1.day

  if frequency == 'daily'
    parsed_time
  else
    # this adds a day until the date matches the day in the subscription
    while parsed_time.strftime("%A").downcase != day.downcase
      parsed_time += 1.day
    end
  end
end

Helot

Answer 2

I have used delayed_job for similar tasks in the past. Probably you can use the same technique with resque. Essentially, you have to schedule the next job at the end of the current job.

class Subscription < ActiveRecord::Base
  after_create :send_email        
  def send_email 
    # do stuff and then schedule the next run
  ensure
    send_email
  end
  handle_asynchronously :send_email, :run_at => Proc.new{|s| s.deliver_at }

  def daily? (frequency == "daily");end
  def max_attempts 1;end

  def time_sec
    hour,min=time.split(":").map(&:to_i)
    hour.hours + min.minutes
  end

  def days_sec
    day.nil? ? 0 : Time::DAYS_INTO_WEEK[day.to_sym].days
  end

  def interval_start_time
    time = Time.now.in_time_zone(time_zone)
    daily? ?  time.beginning_of_day : time.beginning_of_week
  end

  def deliver_at
    run_at = interval_start_time + days_sec + time_sec
    if time.past?
      run_at = daily? ? run_at.tomorrow : 1.week.from_now(run_at)
    end
    run_at
  end        
end

Rescheduling caveats

Update the code to handle cycle termination. You can handle this by adding a boolean column called active (set it to true by default). To disable the subscription, set the column to false.

  def send_email
    return unless active?
    # do stuff and then schedule the next run
  ensure
    send_email if active?
  end

Set the max_attempts for the job to 1. Otherwise you will flood the queue. In the solution above, the jobs for send_email will be attempted once.

Ancohuma

Updated the answer to handle max_attempts elegantly. – Ancohuma

Answer 3

This is more of a system level problem than being specific to ruby.

First off, store all your time internally as GMT. Timezones are not real, other than being a preference setting (in the user's table) that offsets the time in the view. The coordinate system in which the math is done should be consistent.

Then each frequency corresponds to a cronjob setting: daily, monday, tuesday, etc. Really, from the data in table, you don't need two columns, unless you see this as changing.

Then, when the cron fires, use a scheduler (like linux AT) to handle when the email goes out. This is more of a system level scheduling problem, or at least, I'd trust the system more to handle this. It needs to handle the case where the system is restarted, including the services and app.

One caveat however, is that if you are sending a large volume of mail, you really can't guarantee that the mail will be sent according to preference. You may actually have to throttle it back to avoid getting dumped/blocked (to say, 1000/messages spread over an hour). Different networks will have different thresholds of use.

So basically:

cron -> at -> send mail

Lapboard

Answer 4

I think you can leverage this gem for this purpose: https://github.com/bvandenbos/resque-scheduler

Regards,

Dove

Not sure if it's the right solution, Seems like you can't have recurring tasks created dynamically (not using a yaml configuration file) – Punke

@Ryan In the task configuration states otherwise: # If you want to be able to dynamically change the schedule, # uncomment this line. A dynamic schedule can be updated via the # Resque::Scheduler.set_schedule (and remove_schedule) methods. # When dynamic is set to true, the scheduler process looks for # schedule changes and applies them on the fly. # Note: This feature is only available in >=2.0.0. #Resque::Scheduler.dynamic = true – Dove

Answer 5

You could use the DelayedJob or Resque Gems, which represent each instance of a scheduled task as a row in a database table. There are methods for triggering, scheduling, and withdrawing tasks from the calendar.

Tellez

Answer 6

I've just implemented this for my project, I've found a really easy way to do it is to use the Whenever Gem (found here https://github.com/javan/whenever).

To get started, go to your app and put

gem 'whenever'

then use:

wheneverize .

in the terminal. This will create a schedule.rb file in your config folder.

You put your rules in your schedule.rb (shown below) and let them call certain methods - for instance, mine calls the model method DataProviderUser.cron which will run whatever code I have there.

Once you've created this file, to start the cron job, on the command line use:

whenever
whenever -w

and

whenever -c

stops/clears the cron jobs.

The gems documentation on github is really useful but I recommend you set the output to your own log file (as I've done below). Hope that helps :)

in my schedule.rb I have:

set :output, 'log/cron.log'
every :hour do
runner "DataProviderUser.cron('hourly')"
end

every :day do
runner "DataProviderUser.cron('daily')"
end

every '0 0 * * 0' do
runner "DataProviderUser.cron('weekly')"
end

every 14.days do
runner "DataProviderUser.cron('fortnightly')"
end

every :month do
runner "DataProviderUser.cron('monthly')"
end

Hoff