Why do RSpec feature specs using Capybara, Poltergeist and Javascript fail inconsistently?

Question

Hi fellow software developers,

I have a Rails app with a usual testsuite set up (RSpec, Capybara, Poltergeist, jQuery, Rails). I use a little Javascript(jQuery) for remote requests. Basically it works and I don't understand my problem yet. So I may update the question with further knowledge.

Maybe you could give me some hints what to investigate before I post lots of irrelevant code and config.

The inconsistent and unexpected behaviour is in a feature spec that tests via Poltergeist in Capybara that the User interaction (clicking forms and links) hides and shows HTML-elements via AJAX with jQuery.

It seemed that it even works correctly if I test manually. Setting Capybara default time to wait changed only the time I had to wait until the spec run =). I found no other relevant configuration or usage yet.

So any help/idea is much appreciated. Thanks in advance.

My spec_helper:

require 'simplecov'
require 'rubygems'

ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'email_spec'
require 'rspec/autorun'
require 'rspec/rails'
require 'capybara/rspec'
require 'capybara/poltergeist'
require 'capybara/rails'

Capybara.javascript_driver = :poltergeist

Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}

RSpec.configure do |config|
  config.include(EmailSpec::Helpers)
  config.include(EmailSpec::Matchers)
  config.include Capybara::DSL
  #config.include Capybara::RSpecMatchers
  config.mock_with :rspec

  config.fixture_path = "#{::Rails.root}/spec/fixtures"

  # If you're not using ActiveRecord, or you'd prefer not to run each of your
  # examples within a transaction, remove the following line or assign false
  # instead of true.
  # set to false with Capybara + Database Cleaner
  # set to true with ActiveRecord::Base patch from Jose Valim and without Database Cleaner
  config.use_transactional_fixtures = false
  config.infer_base_class_for_anonymous_controllers = true
  config.order = "random"
  config.include FactoryGirl::Syntax::Methods

  config.after(:each) do
    # some model deletion
    model.delete_all
  end

  config.include FeatureHelpers
end

class ActiveRecord::Base
  mattr_accessor :shared_connection
  @@shared_connection = nil

  def self.connection
    @shared_connection || retrieve_connection
  end
end

# # Forces all threads to share the same connection. This works on Capybara
# # because it starts the web server in a thread
ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection

FactoryGirl.reload

Shrove

Which approach are you using to deal with the database contents and transactions? (See rubydoc.info/github/jnicklas/capybara/…) – Casabonne

I use a mixture of Jose Valim's approach and default. Basically I don't yet understand how every piece works under the hood. So I usually try to put everything together as recommended or in official documentation. – Shrove

@Lencho: could you explain to me why it has something to do with the database stuff in spec_helper or give me a link to read? – Shrove

It was a stretch, but I recently had a similar problem that I thought, "no way it has anything to do with the database in the other thread," and it did. Reading your comments on the answer, I don't think I can add much to what you're already pursuing. If you're using database cleaner, then don't use the shared connection. The shared connection is documented to sometimes lead to inconsistent behaviour in test cases (e.g. sometimes they work and sometimes they don't, even if the code hasn't changed). – Casabonne

Answer 1

This is a fresh setup of capybara-poltergeist for a rails app, could you check difference between this and your versions? (I'm too lazy ^_^ )

Gemfile

group :development, :test do
  gem 'rspec-rails'
end

group :test do
  gem 'capybara'
  gem 'database_cleaner'
  gem 'poltergeist'
  gem 'phantomjs', require: 'phantomjs/poltergeist'
end

spec/spec_helper.rb

ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'
require 'rspec/rails'
require 'capybara/rspec'
require 'capybara/rails'
require 'capybara/poltergeist'

Capybara.javascript_driver = :poltergeist

Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }

ActiveRecord::Migration.check_pending! if defined?(ActiveRecord::Migration)

RSpec.configure do |config|
  config.include FactoryGirl::Syntax::Methods
  config.order = "random"
  config.fixture_path = "#{::Rails.root}/spec/fixtures"
  config.use_transactional_fixtures = false
  config.infer_base_class_for_anonymous_controllers = false
end

spec/support/database_cleaner.rb

require "database_cleaner"

RSpec.configure do |config|
  config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
  end

  config.before(:each) do
    DatabaseCleaner.strategy = :transaction
  end

  config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.after(:each) do
    DatabaseCleaner.clean
  end
end

spec/features/posts_spec.rb

require "spec_helper"

describe "Posts pages" do
  let!(:post) { create :post } # Factory girl

  it "has post", js: true do
    visit posts_path
    expect(page).to have_content post.title
  end
end

Kaki

Thx for your quick answer. I will try this out and bring you up-to-date. – Shrove

Looks good so far. The suspicious spec runs for now. It boils down to the DatabaseCleaner stuff. The rest seems to be irrelevant. Can you give me an explanation or link for further reading, pls? I don't understand what clicking on a link requesting js has to do with DatabaseCleaner or my previous - I suppose - wrong spec_helper. – Shrove

Have I answered on your question? This is good article about full stack testing in rails. (also there's explanation about database_cleaner): robots.thoughtbot.com/how-we-test-rails-applications – Kaki

Well, I will have to look at my test suite at first if I can use the DatabaseCleaner-only-approach. Although I don't understand it yet I will accept your answer if my suite runs because you point into the right direction. Also many thanks for the link. – Shrove

Hm, same inconsistency again :( . Have to look into it tomorrow. Maybe my Javascript or its inclusion is wrong. – Shrove

No solution found so far :( . I accept your answer. It reminded me of this post from Avdi Grimm about DatabaseCleaner setup and you gave me a cleaner spec_helper template :) . thx! – Shrove