Using url_for in Rails/Capybara/Poltergeist spec sends the driver to example.com instead of the app

Question

If I call url_for within a feature spec, it returns an absolute URL starting with http://www.example.com/. Capybara will happily attempt to load pages on that site, but that has nothing to do with my app. Here are minimal steps to reproduce the issue:

Start with this Gemfile:

source 'https://rubygems.org'

gem "sqlite3"
gem "jquery-rails"
gem "draper"
gem "rails", '4.1.0'
gem "therubyracer"
gem "uglifier"
gem "rspec-rails"
gem "capybara"
gem "poltergeist"
gem "launchy"

Run the following:

bundle
rails new myapp -O
cd myapp
rm Gemfile Gemfile.lock
rails generate controller Test test
rails generate rspec:install
mkdir spec/features

Comment out the lines in spec/spec_helper.rb that say they should be removed when not using ActiveRecord, and then create spec/features/feature_spec.rb with the following contents:

require 'capybara/poltergeist'

Capybara.configure do |config|
  config.javascript_driver = :poltergeist
end

require 'spec_helper'

describe "nothing", js: true do
  specify do
    visit(url_for(controller: :test, action: :test))
    save_and_open_page
  end
end

Finally, run rake spec and you will see the example.com page pop up in a browser. I have verified this behavior back to Rails 3.2.17.

Why is this happening, and is there a way to get URLs for the app being tested instead of example.com?

Edit: some things I have found looking into this more:

ActionDispatch::Routing::UrlFor.url_for is called from RSpec examples. It has only_path defaulting to false.

ActionView::RoutingUrlFor is the version you get in, say, a view. It has only_path defaulting to true, which works much better.

This commit to the rspec-rails gem probably caused the problem, by adding www.example.com as the default host. There is no explanation anywhere about why this host is an appropriate/useful choice.

Nealon

This Poltergeist GitHub issue is related: github.com/teampoltergeist/poltergeist/issues/201 – Nealon

On a related note, I also noticed this problem when using a dynamic url in a feature spec with javascript (e.g., visit thing_url). I found github.com/jnicklas/capybara/issues/306 where jeromelefeuvre says that thing_path should be used in place of thing_url. I changed to the _path version and it is working. – Arlinearlington

Answer 1

The problem manifests for the following reasons:

1. You are using Poltergeist, which uses PhantomJS, which is perfectly capable of opening any URL.
2. You are using the url_for helper which needs to know the domain it should be generating a url for. When used inside a Rails view or controller, Rails supplies the domain based on what was used to make the request. When outside of a view or a controller, like in an ActionMailer or Capybara test, the domain is unknown. Capybara defaults the unknown domain to example.com.

So everything is working the way it should. Now, it happens to not be the way that you want it to work. However, if you want it to work how you would like you should do one of the following things:

1. Use the path_only option in url_for to tell it not to use the host part.
2. Use the host option in url_for to specify the correct host.

Cremona

This is unsatisfying, because within a view or controller, only_path defaults to true, and everything works as intended. But within a spec, it is modified to default to false. Why is there this special behavior within specs that seems like it can only possibly break things? Yes, in this simple example I could certainly add only_path: true, but I would ultimately like to be able to call my own helper methods that in turn call url_for; what I am forced to do to implement your strategy is to write alternate versions of these methods just for use in testing. – Nealon

I am upvoting because this is the best solution I am aware of. But I am hoping for something that would resolve or at least explain the underlying issue, which is that Rails specs stick an apparently useless example.com into URLs. – Nealon

As I mentioned, only_path is false by default. It is not true by default. Within a view or a control this works because Rails understands the context of the request (domain, host, etc). When you are outside of the context of a real request, you must do something about that. It never hurts to be explicit with your options even if you are making a helper. I would also consider using url_for less in general. A well defined routes file and the standard path helpers should remove the need to use url_for. – Cremona

Please see the documentation for the options and defaults: apidock.com/rails/ActionDispatch/Routing/UrlFor/url_for – Cremona

Your link goes to the version of url_for called from specs. What I am saying is that that is different from the one normally used in the app, which has a true default: apidock.com/rails/ActionView/RoutingUrlFor/url_for – Nealon

I would like to get my named routes working better, but that's a different topic, and I think it might be impossible or at least very messy. For now, I would like to understand why the url_for used in specs appears to be broken, and if there is no reason for it, I am hoping that I can help fix it. – Nealon

What you linked to is part of ActionView. In a request spec you do not have a view or access to one or any kind of request context. You cannot use that version in you specs. It is a good analog to why you need to set your host/domain in ActionMailer. I believe the reason why it is that way is because inside ActionView, you can be assumed to be in a request context and just having a relative path would not only be possible but desired. Outside of ActionView when using the ActionDispatch version, you can't assume things like that so the default is false. – Cremona

Answer 2

This is how those gems work. The http://example.com is irrelevant to your app. In general, you should not have fully hard-coded paths in your app. Rails attempts to determine your local domain (for specs this is example.com, which is configurable) and creates paths off of that.

The idea here is that you have a base URL which may change. Say, for staging I use a Heroku local app: randomname-123-staging.heroku.com. My urls will be prefixed with that. However, in production I own a domain name. My urls there will start with mydomain.com. It makes no sense for me to have to update all of my URLs based on the environments base domain; this should be (and is) provided by Rails.

By using a generic domain, which is supposed to be guaranteed to not resolve to a real IP, the specs help you code to this possibility.

Sindee

If it is irrelevant to my app, why is it causing Poltergeist to fail to load the URLs generated by url_for, as my example shows? More importantly, how do I fix this situation? – Nealon

Your link does not support the claim that example.com is "guaranteed to not resolve to a real IP", and even if that were the case it would not help the situation. – Nealon

@Nealon Capybara is normally expecting a rack-app: github.com/jnicklas/capybara#calling-remote-servers; which ignores the domain. Using save_and_open_page by-passes that and let's the browser do the DNS look-up instead. You can try setting Capybara.app_host to localhost or an equivalent setting. – Sindee

If the save_and_open_page is removed, do the specs pass? – Sindee

I am using Poltergeist, which does support remote servers, so I don't get this (very desirable!) "ignores the domain" feature, and setting Capybara.app_host does not affect the return value of url_for, since that method does not interact with Capybara at all. The specs in my example pass, since I did not test for anything, but if you write any useful tests they will fail with a message like: expected to find text "test" in "Example Domain This domain is established to be used for illustrative..." – Nealon