rspec rails testing: how can I force ActiveJob job's to run inline for certain tests?

Question

I would like my background jobs to run inline for certain marked tests. I can do it by wrapping the test with perform_enqueued do but I'd like to just be able to tag them with metadata and it happens automatically, if possible.

I've tried the following:

it "does everything in the job too", perform_enqueued: true do
end

config.around(:each) do |example|
  if example.metadata[:perform_enqueued]
    perform_enqueued_jobs do
      example.run
    end
  end
end

but it results in an error:

undefined method `perform_enqueued_jobs=' for ActiveJob::QueueAdapters::InlineAdapter:Class

Neocolonialism

Answer 1

In your spec/rails_helper.rb:

RSpec.configure do |config|
  # ...
  config.include ActiveJob::TestHelper
end

Or in your test:

context "when jobs are executed" do
  include ActiveJob::TestHelper

  # ...
end

Then in your tests:

perform_enqueued_jobs do
  example.run
end

Meldon

Answer 2

You need to set your adapter for tests to ActiveJob::QueueAdapters::TestAdapter which responds to .perform_enqueued_jobs =. You can do that on your spec/rails_helper.rb file:

ActiveJob::Base.queue_adapter = :test

Alexandrite

Answer 3

How to use InlineAdapter in tests? — ActiveJob::TestHelper won't let you!

I see that you were trying to use InlineAdapter...

I had the same desire — to use the InlineAdapter in all my tests.

Unfortunately — at least for RSpec request tests (ActionDispatch integration tests) — ActiveJob::TestHelper gets automatically included into the test context, and it seems to force you to use ActiveJob::QueueAdapters::TestAdapter rather than InlineAdapter.

gems/activejob-7.0.4.3/lib/active_job/test_helper.rb has all this:

    ActiveJob::Base.include(TestQueueAdapter)

    def before_setup # :nodoc:
      test_adapter = queue_adapter_for_test

      queue_adapter_changed_jobs.each do |klass|
        klass.enable_test_adapter(test_adapter)
      end

      clear_enqueued_jobs
      clear_performed_jobs
      super
    end

    def after_teardown # :nodoc:
      super

      queue_adapter_changed_jobs.each { |klass| klass.disable_test_adapter }
    end

    # Specifies the queue adapter to use with all Active Job test helpers.
    #
    # Returns an instance of the queue adapter and defaults to
    # ActiveJob::QueueAdapters::TestAdapter.
    #
    # Note: The adapter provided by this method must provide some additional
    # methods from those expected of a standard ActiveJob::QueueAdapter
    # in order to be used with the active job test helpers. Refer to
    # ActiveJob::QueueAdapters::TestAdapter.
    def queue_adapter_for_test
      ActiveJob::QueueAdapters::TestAdapter.new
    end

which causes it to completely ignore any config.active_job.queue_adapter = :inline configuration you might have (because it overrides the queue_adapter class_attribute).

I even tried overriding queue_adapter_for_test:

    def queue_adapter_for_test
      ActiveJob::QueueAdapters::InlineAdapter.new
    end

but it still doesn't work, because InlineAdapter doesn't define enqueued_jobs and we get this:

     NoMethodError:
       undefined method `enqueued_jobs' for #<ActiveJob::QueueAdapters::InlineAdapter:0x00007f7efcce6580>
       Did you mean?  enqueue
     # gems/3.1.0/gems/activejob-7.0.4.3/lib/active_job/test_helper.rb:9:in `enqueued_jobs'
     # gems/3.1.0/gems/activejob-7.0.4.3/lib/active_job/test_helper.rb:641:in `clear_enqueued_jobs'
     # gems/3.1.0/gems/activejob-7.0.4.3/lib/active_job/test_helper.rb:46:in `before_setup'

In conclusion... There's no supported way to use InlineAdapter anywhere that ActiveJob::TestHelper is included (such as request tests). But there are always workarounds...

Workaround #1: Override enqueued_jobs, etc. to avoid getting errors

Assuming you're using RSpec, you can add a spec/support/active_job.rb that does this:

# Override some things from ActiveJob::TestHelper (which gets included automatically by RSpec in
# request tests) so that we can use InlineAdapter instead of TestAdapter and don't have to manually
# call perform_enqueued_jobs any time we have jobs that get enqueued — InlineAdapter automatically
# runs the job immediately (synchronously). See
# https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html.
module UseInlineQueueAdapter
  def queue_adapter_for_test
    ActiveJob::QueueAdapters::InlineAdapter.new
  end

  def enqueued_jobs
    if queue_adapter.respond_to?(__callee__)
      super
    else
      []
    end
  end

  def performed_jobs
    if queue_adapter.respond_to?(__callee__)
      super
    else
      []
    end
  end
end

RSpec.configure do |config|
  config.include UseInlineQueueAdapter
end

Your specific question — only for certain, marked tests

In my case, I was happy just using inline adapter for all tests...

But it sounds like you would like the ability to only use it for some tests, which you mark with metadata.

It seems like that shouldn't be much harder. It seems like all you would have to do is add a conditional to the queue_adapter_for_test method that you override:

  def queue_adapter_for_test
    if example.metadata[:inline_jobs]
      ActiveJob::QueueAdapters::InlineAdapter.new
    else
      ActiveJob::QueueAdapters::TestAdapter.new
    end
  end

Workaround #2: Just use TestAdapter like it wants you to

Since perform_enqueued_jobs causes any jobs enqueued during the duration of the block to be performed immediately ("inline"), what you had should have worked fine too — but only if you leave queue_adapter set to TestAdapter (:test):

config.around(:each) do |example|
  if example.metadata[:perform_enqueued]
    perform_enqueued_jobs do
      example.run
    end
  end
end

The other reason why it didn't work for you is because around(:each) gets run before before(:each). So even though ActiveJob::TestHelper automatically changes the queue adapter to TestAdapter — it does so from a before(:each) (technically a before_setup callback). So at the point where your around(:each) is calling perform_enqueued_jobs, the ActiveJob::Base.queue_adapter should still be however it was configured in your configuration.

Presumably you have something like config.active_job.queue_adapter = :inline in your config/environments/test.rb? As other answers have noted, you'll have to change that to :test instead if you want your approach to work.

Because, as the error points out, InlineAdapter has no concept of enqueued_jobs — and correspondingly, no method named perform_enqueued_jobs defined.

Here's what I came up with, which seems to work:

RSpec.configure do |config|
  config.include(Module.new do
    # Without this, the perform_enqueued_jobs block below has no effect, because it sets
    # perform_enqueued_jobs on ActiveJob::Base.queue_adapter, yet
    # ActiveJob::TestHelper#queue_adapter_for_test by default instantiates a _new_
    # ActiveJob::QueueAdapters::TestAdapter.new (this happens in a before(:example)), whose
    # perform_enqueued_jobs attribute would of course still have the default value of nil.
    def queue_adapter_for_test
      if ActiveJob::Base.queue_adapter.is_a?(ActiveJob::QueueAdapters::TestAdapter)
        ActiveJob::Base.queue_adapter
      else
        super
      end
    end
  end)

  config.around do |example|
    if example.metadata[:perform_enqueued_jobs]
      perform_enqueued_jobs do
        example.run
      end
    else
      example.run
    end
  end
end

Now you can go ahead and annotate any example groups or examples where you want enqueued jobs to be performed immediately!

it "performs the job immediately as soon as enqueued", :perform_enqueued_jobs do
  # ...
end

Having tried both of these workarounds, I would now recommend the second one, as it's more flexible, and still lets you do async enqueuing in any tests that need it, but lets you switch to inline performing when you can get away with that simpler option...

Changsha

Answer 4

In my case, I wanted to enable GoodJob in async mode for a specific test so that I could properly test that a job was being discarded when cancelled by the user. I found that ActiveJob::TestHelper was interfering with setting the queue adapter on ActiveJob::Base.

It turns out that module adds the following to ActiveJob::Base:

module ClassMethods
  def queue_adapter
    self._test_adapter.nil? ? super : self._test_adapter
  end

  def disable_test_adapter
    self._test_adapter = nil
  end

  def enable_test_adapter(test_adapter)
    self._test_adapter = test_adapter
  end
end

All I needed in the end was the following and it looks like ActiveJob::TestHelper handles the cleanup:

describe 'something that needs to queue real jobs' do
  before do
    ActiveJob::Base.enable_test_adapter ActiveJob::QueueAdapters::GoodJobAdapter.new(execution_mode: :async)
  end

  # Test stuff that has been queued via the GoodJob adapter
end

Ceilometer