rspec test passes in isolation, but fails when run with other tests

Question

I have some specs, written in RSpec, that test various models. I use Factory Girl to generate object for testing.

Now the most peculiar thing happens:
When i run rspec spec/models/specific_model_spec.rb --- this passes all the tests in that spec

However, when I run rspec spec/models --- every test in this spec fails referring to an invalid association being created (via a factory)

The association created by the factory is obviously valid as running the test in isolation also shows.

What could be causing this behavior?

Update:
The error i get when running the spec together with other specs (the error is the same for each failure):

6) StreamItem adds a stream_item to a project and consultant when an engagement is added 
 Failure/Error: @project = Factory.create(:project, :name => 'bar' )
 Validation failed: Customer is invalid
 # ./spec/models/stream_item_spec.rb:44:in `block (2 levels) in <top (required)>'

The project factory is tested in another spec and passes fine...

Update 2: The relevant factory code used is a follows:

Factory.define :manager, :class => User do |f|
  f.sequence(:email) { |n| "bar#{n}@example.com" }
  f.password "pass12"
  f.sequence(:name) { |n| "Erwin#{n}" }
  f.roles_mask 4
end

Factory.define :customer do |f|
  f.sequence(:name) { |n| "foo customer#{n}" }
  f.association :last_actor, :factory => :manager
  f.account_id 1
end

Factory.define :project do |f|
  f.sequence(:name) { |n| "foo project#{n}" }
  f.association :manager, :factory => :manager
  f.association :customer, :factory => :customer
  f.start_date Date.today << 1
  f.finish_date Date.today >> 2
  f.status 1
  f.association :last_actor, :factory => :manager
  f.account_id 1
end

Physique

Thanks. And your factory code? – Schouten

Could you show your customer definition? – Happy

Answer 1

This usually indicates that your other specs leave some data in the DB that conflicts with later factory calls. I suspect if you look into why the factory create method failed, you'll see a validation for uniqueness fail, maybe on the customer's email.

Turn off transactional fixtures:

# spec_helper.rb
config.use_transactional_fixtures = false

and use database cleaner instead. This blog post might help as well.

Contemn

The problem ended up being some left over data. Not in the database but in a global variable of a plugin. – Physique

how did you find this global variable? – Grassplot

I don't like the solutions offered but the explanation is gold. I will attempt to find what is causing my conflict, thanks. – Purpose

I'm having the same issue, using Avdi's database cleaner setup. Running some of my feature tests alone work fine, but they'll fail for some reason when running along with other tests, because the records I'm creating that they depend on are deleted. Literally I'll get a bunch of failures, then I'll do rspec --only-failures and they'll all pass. Trying to figure out why. Is there a way to make Factory Girl fail with a reason? Or does anyone else have any insight as to what could make that happen? – Trantrance

The right file where to put that line is: rails_helper.rb and NOT spec_helper.rb – Spondee

@DiegoD in 2016, perhaps. In 2011 rails_helper.rb did not exist. – Contemn

Answer 2

RSpec now has a "bisect" feature designed specifically for finding this kind of issue.

Run the RSpec command that's causing the failure with the --bisect flag, and RSpec will automatically identify which combination of specs is causing that failure.

rspec spec/models --bisect

Alarice