How can I write javascript tests that don't timeout or fail randomly with rspec, capybara, and poltergeist?

Question

I have been having problems with my tests timing out and failing randomly. I have been looking around for best practices of how to write robust capybara integration tests but I don't find anything helpful.

Ever since we started writing a bunch of tests our CI server has been failing randomly, making our app look ver unstable, but the tests always (mostly) pass locally.

I want to find out how you and other experienced BDD/TDD gurus handle:

• How to deal with external javascript and stuff (KissMetrics, Google Analyics, etc) and
• Debugging and preventing timeout errors that break the build

Any help would be appreciated.

Epizoon

Which versions of poltergeist and phantomjs are you using? I've encountered seemingly random timing issues with poltergeist > 0.7.0 and phantomjs 1.7.0. They went away when we fixed poltergeist at 0.7.0 and phantomjs < 1.7.0 (I forget which exact number but I think 1.7.0 is when the trouble started). – Brassy

I am using poltergeist (1.0.2) and phantomjs 1.7.0. I just noticed that phantomjs 1.8.9 was released, but I haven't tried that. I will upgrade and test it out. With all of the trouble we have been having I wondered if anyone was using these tools. :) – Epizoon

Cool. Please report back here with your findings. Curious to know whether my problems might be solved, too. :-) – Brassy

I upgraded to phantomjs 1.7.0 to 1.8.1 and nothing changes. I have the latest poltergeist. (I still get timeouts and element not found errors masked as timeouts) – Epizoon

Nearly 4 years later. This still seems to be a common problem. While these two answers are some of the problem, there still seem to be many other things that can cause random failures. – Brawner

Answer 1

To follow up on this. I appreciated @jonleighton's invitation to file a bug on poltergeist, but the problems that I encountered were related to two separate problems:

1. Bad/wrong capybara assertions that would timeout because I was not following the suggestions clearly outlined in this post
2. 3rd party javascripts and things. Basically I had 3rd party javascripts like kissmetrics, google analytics, and even live help chat that would load each time a test was wrong, I eliminated this from happening and my tests got faster and appear to be more stable/consistent.

Epizoon

This doesn't explain exactly what was the cause of it. Something in the 3rd party scripts was causing it, do you have any clue? – Lathan

@Lathan the problems that I was encountering were related to timeouts and bad assertions in the test, specifically replacing page.should_not have_content('blah') with page.should have_no_content('blah') – Epizoon

Answer 2

I have found that in many cases I can help my integration tests be more deterministic by taking advantage of Poltergeist's blacklist feature. In my case, I have blacklisted host names like the following.

typekit.net
facebook.net
facebook.com
google.com
google-analytics.com
...

The idea is to turn off anything that a) is not really needed for testing and b) could affect the page load completion under the CI environment. I have noticed this helps tremendously. Also, using this PhantomJS option with Poltergeist helps too:

phantomjs_options: ['--ignore-ssl-errors=yes']

Z

Phantomjs docs specify using true/false for --ignore-ssl-errors, although yes/no is also supported. – Budge