Testing Rake in Rails: Multiple Error Raises Silenced In Test

Question

I have a rake task that guards against dangerous Rails rake rasks, based on the environment. It works fine. When I test each individual dangerous method in RSpec, the test passes. When I test multiple in a row, for multiple environments, the test fails after the first one. Even if I run the test multiple times for the same dangerous action, rake db:setup for example, it will only pass the first time. If I run the tests as individual it statements, one for each dangerous action, only the first two will pass (there are 4).

How can I get RSpec to behave correctly here, and pass all the tests when run in a suite?

The rake task

# guard_dangerous_tasks.rake
class InvalidTaskError < StandardError; end
task :guard_dangerous_tasks => :environment do
  unless Rails.env == 'development'
    raise InvalidTaskError
  end
end

%w[ db:setup db:reset ].each do |task|
  Rake::Task[task].enhance ['guard_dangerous_tasks']
end

The RSpec test

require 'spec_helper'
require 'rake'
load 'Rakefile'

describe 'dangerous_tasks' do
  context 'given a production environment' do
    it 'prevents dangerous tasks' do
      allow(Rails).to receive(:env).and_return('production')

      %w[ db:setup db:reset ].each do |task_name|
        expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
      end
    end
  end

  context 'given a test environment' do
    it 'prevents dangerous tasks' do
      allow(Rails).to receive(:env).and_return('test')

      %w[ db:setup db:reset ].each do |task_name|
        expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
      end
    end
  end
end

RSpec Output

# we know the guard task did its job,
# because the rake task didn't actually run.
Failure/Error: expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
   expected InvalidTaskError but nothing was raised

Martimartial

Are you using any environment preloader ? Spring, zeus ? Maybe they cause some problems. Additionally if there is no exception what is the content of Rails.env ? Could you add STDERR.puts "Env:#{Rails.env}" – Earthen

No 'spring'. No 'zeus'. This fails on CircleCI as well. When there is no exception, no rake tasks are called including the enhancement, so I can't find a way to trigger a puts. – Martimartial

@Martimartial I think I found solution of you problem. Check it in my answer. – Antefix

Answer 1

I can think about two solution of your problem.

But first we need to find out where is the root of the problem.

Root of the problem

Let's start with a line from your code:

Rake::Task[task].enhance ['guard_dangerous_tasks']

Comparing it with source code of Rake::Task

# File rake/task.rb, line 96
def enhance(deps=nil, &block)
  @prerequisites |= deps if deps
  @actions << block if block_given?
  self
end

you can see, that guard_dangerous_tasks should be added to @prerequisites array. It can be easily checked:

p Rake::Task['db:reset'].prerequisites # => ["environment", "load_config", "guard_dangerous_tasks"]

Continuing with you source code.

You use invoke to execute tasks. If we pay close attention to invoke's' documentation, it states:

Invoke the task if it is needed.

Once the task is executed, it could not be invoked again (unless we reenable it).

But why should this to be a problem? We are running different tasks, aren't we? But actually we don't!

We run guard_dangerous_tasks before all tasks in our tasks array! And it's being executed only once.

Solution #1 not the best one

As soon as we know where is our problem we can think about one (not the best solution).

Let's reenable guard_dangerous_tasks after each iteration:

dangerous_task = Rake::Task['guard_dangerous_tasks']
%w[ db:setup db:reset ].each do |task_name|
  expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
  dangerous_task.reenable
end

Solution #2 guard_dangerous_tasks is not a prerequisite

We get better solution of our problem if we realize, that guard_dangerous_tasks should not be a prerequisite! Prerequisite are supposed to "prepare" stage and be executed only once. But we should never blind our eyes to dangers!

This is why we should extend with guard_dangerous_tasks as an action, which will be executed each time the parent task is run.

According to the source code of Rake::Task (see above) we should pass our logic in a block if we want it to be added as an action.

%w[ db:setup db:reset ].each do |task|
  Rake::Task[task].enhance do
    Rake::Task['guard_dangerous_tasks'].execute
  end
end

We can leave our test unchanged now and it passes:

%w[ db:setup db:reset ].each do |task_name|
  expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
end

But leaving invoke is a ticket for new problems. It's better to be replaced with execute:

%w[ db:setup db:reset ].each do |task_name|
  expect { Rake::Task[task_name].execute }.to raise_error(InvalidTaskError)
end

Be careful with invoke!

We said above, that using invoke is a ticket for new problems. What kind of problems?

Let's try to test our code for both test and production environments. If we wrap our tests inside this loop:

['production','test'].each do |env_name|
  env = ActiveSupport::StringInquirer.new(env_name)
  allow(Rails).to receive(:env).and_return(env)

  %w[ db:setup db:reset ].each do |task_name|
    expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
  end
end

our test will fail with original reason. You can easily fix this by replacing the line

expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)

with

expect { Rake::Task[task_name].execute }.to raise_error(InvalidTaskError)

So what was the reason? You probably already guess it.

In failing test we invoked the same two tasks twice. First time they were executed. The second time they should be reenabled before invokation to execute. When we use execute, action is reenable automatically.

Note You can find working example of this project here: https://github.com/dimakura/stackoverflow-projects/tree/master/31821220-testing-rake

Antefix

Thank you for your thorough answer, but this does not solve the problem. Modify your test to test exactly the same thing in both production and test environments and I think you'll see the problem. – Martimartial

I ran your code with two sets of tests, one for production, one for test. It fails for both solutions. – Martimartial

It fails with Rake::Task[task_name].invoke !! It doesn't fail with Rake::Task[task_name].execute. Both solution #1 and #2 work. I will add a section explaining why invoke is bad in this settings (I actually already said it, but your example is a good illustration to it). – Antefix

@Martimartial I added a section which explains why this test failed with invoke executed on both environments. One should use execute instead. I think it resolves your problem. – Antefix

It works! Brilliant! Thank you for all your time and energy on this one. – Martimartial

Answer 2

Looks like two tasks cannot point to the same task for enhancement so maybe there is a conflict at runtime. So try the block method to handle the situation.

class InvalidTaskError < StandardError; end
%w[ db:setup db:reset ].each do |task|
  Rake::Task[task].enhance do
    unless Rails.env == 'development'
      raise InvalidTaskError
    end
  end
end

and in the spec file, the following modification would create two examples to track the specs properly.

# require 'rails_helper'
require 'spec_helper'
require 'rake'
load 'Rakefile'

describe 'dangerous_tasks' do
  context 'given a production environment' do
    %w[ db:setup db:reset ].each do |task_name|
      it "prevents dangerous tasks #{task_name}" do
        allow(Rails).to receive(:env).and_return('production')
        expect { Rake::Task[task_name].invoke }.to raise_error(InvalidTaskError)
      end
    end
  end
end

Marinmarina

I don't think this is the issue (and trying your suggestions did not fix it). The problem is not that multiple tasks point to the same enhancement task, but that I cannot call the same enhanced command twice in a row and get an error. – Martimartial

It's the same as in the original question, just an RSpec failure. – Martimartial

but using your code I also got the same rspec example failure and using this solution I was able to pass the examples – Marinmarina

I've used your code exactly as you've written it, but duplicated your tests to test both development and test environments. The second run test always fails. – Martimartial

Answer 3

Did you try passing in the specific error?:

expect { Rake::Task[task_name].invoke }.to raise_error(StandardError)

Pulverulent

I did, actually. I cut it out of the simplified question, but I've added it back in because I think you're right, it clarifies what the problem is not. – Martimartial