Testing entire Rspec suite fails

Question

I have an odd situation whereby if I run an individual rspec model spec file all examples are green, if I test my entire spec/models folder all my examples are green. If I test the controllers they all pass green. If I test the entire suite (via rspec spec) then I get failures - If I remove the controller tests entirely everything is green. Now I'm expecting this is entirely self inflicted but I just can't figure it out.

I've narrowed it down to specific examples in the controller tests - which cause the examples in model specs to fail.

eg. in a notes_controller_spec.rb if this line is present

 Note.any_instance.stubs(:valid?).returns(false)

it causes a failure in my models/account_spec.rb

Failure/Error: @account.all_notes.should have(2).notes
ArgumentError:
comparison of Note with Note failed
./app/models/account.rb:293:in `sort'

where line 293 is;

 (self.notes + self.transactions.map(&:notes).flatten).sort {|a,b| a.created_at <=> b.created_at }

I'm pretty sure this is going to be one of those face palm moments so be gentle with me!

Canaster

created_at may be nil for one of the Notes, which means your sort block would be comparing nil against a Fixnum, which would cause the block to return nil, which would make sort blow up with the kind of ArgumentError you're seeing. Are you sure all of your instances have been saved in the database when this line runs? Is it possible your database is not being emptied before each example? – Threegaited

I'm having a similar problem. I have a feeling it may be related to this rspec issue, though it claims to have been fixed. – Giulietta

sorry, I accidentally hit enter, more detail: I'm stubbing a method on any instance of a model - a different model from the one I'm testing in the particular set of tests. When I run a different set of tests that require that method, it isn't found. Taking out the stub in the tests in one file makes the other file pass. Really annoying. I've been experimenting with installing different versions of rspec. – Giulietta

here's a gist of my issue: gist.github.com/1111833 – Giulietta

Is that stubbing being done in a before :all block? That could cause the problem of the method staying stubbed. Just a guess. – Doro

I'm experiencing a similar problem with rails 4.1, rspec 3 and feature specs. In my case it is a date issue but I don't have a clue how to fix it. In a fixture I have a date column with a value of "1.day.ago.to_s(:db)". Then in my spec I do something like "expect(page).to have_content(1.day.ago.to_date)" and it fails because the "1.day.ago" of the spec is one day before the "1.day.ago" from the fixture. It fails but only when I run the full suite. And to make it worse it doesn't fail during certain parts of the day, making me think it's timezone related. I don't get it. – Drakensberg

Answer 1

Are you doing any date setup in a before :all block? These are not transactional and can cause test pollution issues.

Also, I think your syntax might be off here:

    Note.any_instance.stubs(:valid?).returns(false)

Should be:

    Note.any_instance.stub(:valid?).and_return(false)

Reger

Answer 2

I've experienced similar issues with RSpec 3 and Rails 4.1. Whenever I ran the problematic spec file on its own, it would pass, while running the full suite would make it fail.

In my case it was somehow related to time zones. I was explicitly setting the time zone in ApplicationController and for some reason my feature specs didn't like it. If I don't set the time zone when in test environment, everything passes again. e.g.

unless Rails.env.test?
  Time.zone = "some timezone value here"
end

Gantline

Answer 3

I was having a similar problem: Individual model specs passed. When running the whole model suite, I had about 30 failures. What I did was look at the file before all the failures happened. There I found that I was setting things inside threads and using default_scopes like in this railscast.

In a before clause, I printed the Company.current_id. As I thought, when running individually, Company.current_id was nil. When running the suite, Company.current_id was 2. This is what happens when you use default scopes. To fix it, I just set Company.current_id to nil in the before clause.

Before

describe Service, type: :model do
  before do
  end
end

After

describe Service, type: :model do
  before do
    Company.current_id = nil
  end
end

Lucillelucina