How do I prepare test database(s) for Rails rspec tests without running rake spec?

Question

After significant troubleshooting, I figured out that I needed to run rake spec once (I can abort with control-c) before I can run rspec directly (e.g. on a subset of our specs). We are running Rails 3.0.7 and RSpec 2.5.0.

Clearly, rake is running some important database setup tasks / code (we have custom code in the root level rails Rakefile and possibly other places).

How can I run the rake test database setup tasks / code without running rake spec?

In addition to being able to run rspec on a subset of files, I am using specjour to spread our specs across multiple cores (haven't had success with spreading them across the LAN yet), but I see the same behavior as for running rspec directly: I need to run rake spec on each test database (assuming two cores) before specjour works:

rake spec TEST_ENV_NUMBER=1
control-c (after tests start)
rake spec TEST_ENV_NUMBER=2
control-c (after tests start)
specjour

Note: my config/database.yml has this entry for test (as is common for the parallel testing gems):

test:
  adapter: postgresql
  encoding: unicode
  database: test<%=ENV['TEST_ENV_NUMBER']%>
  username: user
  password:

parallel_tests seems to set up its databases correctly, but many of our specs fail.

I should also mention that running specjour prepare causes Postgres to log errors that it can't find the databases, but it creates them (without tables). On a subsequent run, no errors are logged, but also no tables are created. It is possible that my whole issue is simply a bug in prepare, so I reported it on github.

I think that I can run arbitrary code on each specjour test database by setting Specjour::Configuration.prepare in .specjour/hooks.rb, so if there's any rake tasks or other code that I need to run, it may work there.

Politic

Answer 1

I had a similar problem setting up the CI system at work, so I gradually worked up a system to handle this. It may not be the best solution, but it works for me in my situation and I'm always on the lookout for better ways to do things.

I have a test database that I needed setup, but also needed seeded data loaded for our tests to work.

The basics of troubleshooting rake tasks is to run rake with the --trace option to see what is happening under the hood. When i did this, I found that running rake spec did a number of things that I could replicate (or modify as I saw fit) in a custom rake task.

Here's an example of what we do.

desc "Setup test database - drops, loads schema, migrates and seeds the test db"
task :test_db_setup => [:pre_reqs] do
  Rails.env = ENV['RAILS_ENV'] = 'test'
  Rake::Task['db:drop'].invoke
  Rake::Task['db:create'].invoke
  result = capture_stdout { Rake::Task['db:schema:load'].invoke }
  File.open(File.join(ENV['CC_BUILD_ARTIFACTS'] || 'log', 'schema-load.log'), 'w') { |f| f.write(result) }
  Rake::Task['db:seed:load'].invoke
  ActiveRecord::Base.establish_connection
  Rake::Task['db:migrate'].invoke
end

This is only an example, and specific to our situation, so you'll need to figure out what needs to be done to get your test db setup, but it is quite easy to determine using the --trace option of rake.

Additionally, if you find the test setup is taking too long (as it does in our case), you can also dump the database into .sql format and have the test database pipe it directly into mysql to load. We save several minutes off the test db setup that way. I don't show that here because it complicates things substantially -- it needs to be generated properly without getting stale, etc.

HTH

Conclusive

Yes, I have run rake spec with --trace and attempted to replicate some of its tasks in my prepare specjour hook, but it hasn't worked yet. I could potentially write a completely separate rake task to get things set up, but that's another step I was hoping to avoid. – Politic

Answer 2

I would recommend dropping your test database, then re-create it and migrate:

bundle exec rake db:drop RAILS_ENV=test
bundle exec rake db:create RAILS_ENV=test
bundle exec rake db:schema:load RAILS_ENV=test

After these steps you can run your specs:

bundle exec rspec spec

gerry3 noted that:

A simpler solution is to just run rake db:test:prepare

However, if you're using PostgreSQL this wont work because the rails environment gets loaded, which opens a database connection. This causes the prepare call to fail, because the DB cannot be dropped. Tricky thing.

Mur

A simpler solution is to just run rake db:test:prepare. – Politic

I have no problem running rake db:test:prepare with Postgres. You must be seeing your issue for some other reason. – Politic

If db:test:prepare doesn't work you can at least put the commands inline to save some typing: RAILS_ENV=test bundle exec rake db:drop db:create db:schema:load – Doctrinaire

Looks like rake db:test:prepare is deprecated in Rails 4. – Banal

You can chain rake tasks like this: bundle exec rake db:drop db:create db:schema:load RAILS_ENV=test – Trug

rake db:test:prepare still works in rails 5 (& with PostgreSQL 10.3) – Politic

Answer 3

The provided solutions all require to load the Rails environment, which is, in most cases, not the desired behaviour due to very large overhead and very low speed. DatabaseCleaner gem is also rather slow, and it adds another dependency to your app.

After months of chagrin and vexation thanks to reasons vide supra, I have finally found the following solution to be exactly what I need. It's nice, simple and fast. In spec_helper.rb:

config.after :all do
  ActiveRecord::Base.subclasses.each(&:delete_all)
end

The best part about this is: It will only clear those tables that you have effectively touched (untouched Models will not be loaded and thus not appear in subclasses, also the reason why this doesn't work before tests). Also, it executes after the tests, so the (hopefully) green dots will appear right away.

The only downside to this is that if you have a dirty database before running tests, it will not be cleaned. But I doubt that is a major issue, since the test database is usually not touched from outside tests.

Edit

Seeing as this answer has gained some popularity, I wanted to edit it for completeness: if you want to clear all tables, even the ones not touched, you should be able to do something like the "hacks" below.

Hack 1 - pre-loading all models for subclasses method

Evaluate this before calling subclasses:

Dir[Rails.root.join("app", "models", "**", "*.rb")].each(&method(:require))

Note that this method may take some time!

Hack 2 - manually truncating the tables

ActiveRecord::Base.connection.tables.keep_if{ |x| x != 'schema_migrations' }

will get you all table names, with those you can do something like:

case ActiveRecord::Base.configurations[Rails.env]["adapter"]
when /^mysql/, /^postgresql/
  ActiveRecord::Base.connection.execute("TRUNCATE #{table_name}")
when /^sqlite/
  ActiveRecord::Base.connection.execute("DELETE FROM #{table_name}")
  ActiveRecord::Base.connection.execute("DELETE FROM sqlite_sequence where name='#{table_name}'")
end

Dolores

Neat! This is quite useful. – Grace

Answer 4

It appears that in Rails 4.1+, the best solution is simply to add ActiveRecord::Migration.maintain_test_schema! in your rails_helper after require 'rspec/rails'.

i.e. you don't have to worry about having to prepare the database anymore.

https://relishapp.com/rspec/rspec-rails/docs/upgrade#pending-migration-checks

Wallet

Answer 5

In a spring-ified Rails 4 app, my bin/setup is usually augmented to contain

puts "\n== Preparing test database =="
system "RAILS_ENV=test bin/rake db:setup"

---

This is very similar to leviathan's answer, plus seeding the test DB, as

rake db:setup # Create the database, load the schema, and initialize with the seed data
(use db:reset to also drop the database first)

As the comment mentions, if we want to drop the DB first, rake db:reset does just that.

I also find that this provides more feedback when compared to rake db:test:prepare.

Festatus

Answer 6

I started by dropping my test database rake db:drop RAILS_ENV=test

when trying to create a new test database I ran into an issue because my user account was not the same as the account that owns the databases so I created the database in PostgreSQL instead.

type psql in the command prompt and then run the below to create a test database that uses an account other than your own. CREATE DATABASE your_database_name OWNER your_db_owner;

then run your migrations in the test environment. rake db:migrate RAILS_ENV=test

Hydnocarpate