End-to-end testing a RESTful Web service (Rails)
Asked Answered
M

2

6

I'm trying to sift through the myriad of test solutions out there, and I'm not even sure if I'm headed the right direction. The story is: we're running a RESTful Web service, implemented as a Rails app, which backs our mobile clients. We're unit testing the Web service (of course), but that involves mocking out many parts of the application, for instance, the search stack (Apache SOLR).

Moreover, our tests don't (i.e. cannot!) cover critical routes such as the mobile sign-in/sign-on process, since that involves communication between the API application and the mobile website, where the user can enter credentials, e.g. for SSO (Janrain Engage). Hence, a standard Rails integration test won't do.

I realize that in theory, if the test suite is very well designed, where mocking only happens strictly at the those join points where the tests for the next layer start, then by unit or functional testing the service API and the mobile website separately, one could get the same test coverage. I find that in practice though, this is an illusion if you have several developers working on the test suite independently; I just admit that our unit tests are simply not that well designed. Particularly when exercising TDD I found that while the tests can drive the application code, the test code design is only tailored to the unit under test, resulting in a rather wildly grown test suite.

Another thing I found is that sometimes we didn't detect regressions purely using unit tests, where e.g. bad queries were sent to the SOLR server due to knock-on effects. That's why I thought the only true way to ensure that the entire stack works along the critical routes is to automatically end-to-end test it on a staging server before every deployment, i.e. having actual HTTP requests sent to the app.

My questions would be:

  • do you think this is a reasonable thing to do at all? I found very little information about end-to-end testing a live API on the Web, leaving me wondering whether I'm making any sense at all
  • which tools/setup would you suggest? We use Watir to run acceptance tests for our website, but it seems to be overkill for a Web service (no browser environment needed, no JS or anything UI-ish). Something as simple as a Ruby script even?
  • any general best practices or advice you can give me w.r.t. to designing such tests?
Moquette answered 15/6, 2011 at 9:0 Comment(1)
I was wondering if the combination of Capybara/WebRat with Mechanize as the driver would be a good choice?Moquette
S
1

This might be of interest to you: http://groups.google.com/group/ruby-capybara/browse_thread/thread/5c27bf866eb7fad3

What you might want to to try is combining Cucumber (or similar) with one of the tools mentioned in the link above, so you can do something like

Given I have 2 posts
When I send "DELETE" to post with id 1
Then I should have 1 post

This way you can test the API's full stack and check the result.

Sterrett answered 15/6, 2011 at 13:9 Comment(5)
actually, do you have any suggestions for "similar"? We don't want to do Cucumber, since API acceptance tests are read and written by developers, and I'd actually prefer a more low level test description. That's why I intially leaned towards Capybara+Mechanize. Any idea how I could integrate RestClient (I like that one, it seems to integrate well with OAuth) with anything that allows me to write tests that are more expressive than your average Test::Unit?Moquette
now contemplating rspec, shoulda, and matchy, or some combination of the three. too much choice!Moquette
Since the specs will only be read/written by developers, you can do without Cucumber, and stick to RSpec for individual/compartmentalized tests (RSpeck has its own mocking system and matchers). That said, if you want to exercise the full stack with integration scenarios that span several API requests (e.g. get these records from the API, then delete all their children using the API), Cucumber could still be really handy.Sterrett
I've now set up RSpec for our API project, and intend to combine it with RestClient. I've implemented a few specs and it looks like it's exactly what I had in mind. Thanks a lot!Moquette
No problem, glad I could help you find a solution !Sterrett
S
1

We've faced similar problems lately, like testing only on the unit level was fast but not enough to spot integration related anomalies.

I really like Cucumber's gherkin syntax, the value for me in it is that we can run the exercise once and then make several expectations after with separate titles, like if they would be separate RSpec it blocks.

There is a gem called Turnip, which makes possible to write RSpec tests with the gherkin syntax. We have been using it for a while and I'm quite happy with it, we even started having less integration level defects.

I collected my experience regarding in a blog post: https://blog.kalina.tech/2019/10/end-to-end-testing-for-api-only-rails-apps-with-cucumber-syntax.html

Seligmann answered 28/10, 2019 at 17:2 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.