cancelling a sheduled Sidekiq job in Rails

Question

Some Sidekiq jobs in my app are scheduled to change the state of a resource to cancelled unless a user responds within a certain timeframe. There is a lot of information about how to best accomplish this task, but none of it actually cancels the job.

To cancel a job, the code in the wiki says:

class MyWorker
 include Sidekiq::Worker

 def perform(thing_id)
  return if cancelled?
  thing = Thing.find thing_id
  thing.renege!
 end

 def cancelled?
  Sidekiq.redis {|c| c.exists("cancelled-#{jid}") }
 end

 def self.cancel!(jid)
  Sidekiq.redis {|c| c.setex("cancelled-#{jid}", 86400, 1) }
 end
end

Yet here it's suggested that I do something like

 def perform(thing_id)
  thing = Thing.find thing_id
  while !cancel?(thing)
   thing.ignore!
  end
 end

 def cancel?(thing_id)
  thing = Thing.find thing_id
  thing.matched? || thing.passed?
 end

What's confusing about this and similar code on the wiki is none of it actually cancels the job. The above example just performs an update on thing if cancelled? returns false (as it should), but doesn't cancel if and when it returns true in the future. It just fails with an aasm transition error message and gets sent to the RetrySet. Calling MyWorker.cancel! jid in model code throws an undefined variable error. How can I access that jid in the model? How can actually cancel or delete that specific job? Thanks!

Vega

Ignore that SO answer, it's garbage. The wiki is correct. – Heteromerous

the wiki answer checks for a cancelled job, and then uses setex to change the ttl and value. it looks like it requires that the job be set as cancelled elsewhere. in which case - how do i cancel a job? – Vega

The API is right there: MyWorker.cancel! jid – Heteromerous

i'm afraid that this does not quite work as i'd expect. the wiki example says return if cancelled?, which (presumably) cancels if true and executes if false. state changes initiated by a user should result in a false and then this worker cancelling itself, but this code throws the worker into a retry loop – Vega

and then what does the value 1 do in that setex? the api is clear on the use of the ttl value 86400 but not the 1 – Vega

Answer 1

Ok, so turns out I had one question already answered. One of the code sets I included was a functionally similar version of the code from the wiki. The solution to the other question ("how can I access that jid in the model?") seems really obvious if you're not still new to programming, but basically: store the jid in a database column and then retrieve/update it whenever it's needed! Duh!

Vega

Answer 2

# The wiki code
class MyWorker
 include Sidekiq::Worker

 def perform(thing_id)
  return if cancelled?

  # do actual work
 end

 def cancelled?
  Sidekiq.redis {|c| c.exists("cancelled-#{jid}") }
 end

 def self.cancel!(jid)
  Sidekiq.redis {|c| c.setex("cancelled-#{jid}", 86400, 1) }
 end
end

# create job
jid = MyWorker.perform_async("foo")

# cancel job
MyWorker.cancel!(jid)

Heteromerous

thanks for the help and your time, but i'm still very unsure of how to make this work. elsewhere it says that jobs should be responsible for checking and cancelling themselves when they're no longer needed, but this cancel! api request looks like it requires a user action. does it not? if a user pays before an expiration date, i want the worker to cancel itself. otherwise, i want the worker to change state to something like cancelled. – Vega

instead, if a user pays on time, my worker sends the job into a retry queue and infinitely fails with an error like Event 'cancel' cannot transition from 'paid. the return if cancelled? should save this from happening, but it does not. – Vega

yes, cancel! is called based on some user action/event. Your user payment action would call it to cancel the corresponding job, so when the job runs, it just immediately returns and effectively does nothing. – Heteromerous

got it, that makes sense. i'm creating the worker in one method with Worker.perform_in(72.hours, self.id) but using perform_async to get that previous jid and cancel it in another method. this setup creates one delayed worker and one worker that starts and stops shortly after (as i'd think it would). how can i get the jid of the first worker? – Vega

github.com/mperham/sidekiq/issues/1999 seems to suggest using self.class.perform_in(*args) from within the worker but using perform_in inside the worker's perform function seems to affect nothing about when the worker is run – Vega

one more thing i feel needs a bit more clarification: the API instructions state "You should not be scanning through queues/sets and deleting jobs unless something has gone wrong and you need to repair data manually." does this mean that it's a bad idea to search for and cancel an existing worker? – Vega

Perhaps refactor your transition process via a finite state machine on Thing. Use the statesman gem github.com/gocardless/statesman. Then the sidekiq job can attempt to transition the object in question to cancelled state with a guarded transition. If it transistions, it will just finish whereas an unsuccessful guarded transition operation could raise an error which would force the sidekiq object to retry with exponential backoff until the guard condition is eventually satisfied. Statesman also give you insight into the states of your Things via querying by their state. – Dalesman

Answer 3

You can do this but it won't be efficient. It's a linear scan for find a scheduled job by JID.

require 'sidekiq/api' Sidekiq::ScheduledSet.new.find_job(jid).try(:delete) Alternatively your job can look to see if it's still relevant when it runs.

Phalange