Running integration/acceptance tests on the frontend. Need an API for the frontend to tell Rails which database state to set up for each test

Question

My frontend is an EmberJS-based app. It's totally async in nature, so testing it with Capybara is pain and misery. On the other hand, Ember provides a fantastic test suite out of the box which makes acceptance testing fun and effective.

Normally, either fixtures or backend mocks are used to acceptance-test Ember apps. But testing against mocks does not satisfy me at all:

• It will not reveal possible API inconsistencies between the backend and the frontend, race conditions, etc.
• It is impossible to test backend business logic this way. Such tests are not integration tests.
• Finally, acceptance tests require persistency, so you have to replicate backend behavior in a mock. It's very tedious and you effectively end up implementing two backends.

So I want to test against the real backend! It's trivial to set up Ember to use a local backend instance for testing. But the problem is that the backend will persist its state between individual tests and even test sessions.

That's why I'm thinking of implementing a special public API in Rails:

• The API is only available when Rails is run with a specific flag or an env var.
• Rails runs in a non-testing mode, serving normal API calls as it would in production.
• Before each test, the frontend calls the special API, telling Rails which database setup is needed for this specific test.
• When a call to the special API is received, Rails cleans the database and populates it with the requested data. For example, to test item deletion from the cart, the database should have three items in the cart.
• Rails finishes the API request, and the frontend starts the test.
• Frontend runs test steps, using the normal backend API as it would in production: log in, create posts, comment on them. It will also try doing some forbidden things, e. g. edit posts while not logged in, exceed text length constraints, etc and verify whether the backend rejects forbidden actions.
• When the frontend runs next test, it will call the special API again. Rails will discard the state produced by the previous test and set up a new one, for this specific test.

I'm a frontend dev with a sketchy knowledge of Rails. Factory Girl and Database Cleaner seem to be the right tools for the job, but there is absolutely no information how to use them outside Rails' normal test environment. I guess I need a controller or a Rails engine or something.

So the question is: how do I make an API in Rails that can be used by the frontend to tell Rails to set up a certain database state with a fixture factory, while Rails are running in a non-test mode i. e. serving REST API and not running through RSpec/Capybara/Cucumber?

Bonus feature: fixture factory properties should be defined on the frontend so that test code is stored in one place. Thus, the backend should be able to accept fixture factory properties via the special API. Defaults can still be defined in the backend's codebase.

I believe this could become an acceptance/integration testing best practice. If I manage to implement it, I promise to publish a gem.

Shabbygenteel

Perhaps not what you want to hear, but I haven't had terrible difficulty testing complex ember+rails with capybara (before ember's test suite). ember-cli-rails README says "end-to-end tests with frameworks like Cucumber should just work" (indicating I'm not alone). I do wish there was an easy way to ask ember "ready?" for various operations. Have you considered tackling it from that angle instead? If you want to do this, I agree, use FactoryGirl+DB Cleaner and perhaps construct a sinatra app that your tests POST to to control them. Make sure it only works in Test. – Divulge

I did a research on Capybara and discovered that it is a world of nastiest hacks and workarounds. Also, I do not want to keep tests in a repository different from the one they control. – Shabbygenteel

Interesting idea to use API for testing, I don't see exact question to answer. – Hygienic

The question is how to implement the API on the Rails side, how to make use of FactoryGirl and DatabaseCleaner outside of the RSpec environment. – Shabbygenteel

I think best option would be to start a server in a test environment rails s -e test -p3005 -P tmp/pids/server_test.pid. (e -> environment, P -> pidfile to not conflict with default development server, p -> port to listen to) Also you may need to require gems explicitly (if you have require: false in Gemfile) – Hygienic

The folks at dockyard do this, it may be worth reaching out to them. Also, I'd like mirage to eventually support this, and would love your help! github.com/samselikoff/ember-cli-mirage/issues/… – Realistic

Answer 1

May be something like this

config/routes.rb

 namespace 'test_api' do
   resource 'db_transaction', only: [:create, :destroy]
 end if Rails.env.test?

controllers/test_api/db_transactions_controller.rb

require 'database_cleaner'
def create
  DatabaseCleaner.start
end

def destroy
  DatabaseCleaner.clean
end

Hygienic