How do poltergeist/PhantomJS and capybara-webkit differ?

Question

1. What are the differences between PhantomJS and capybara-webkit?
2. What are the advantages of capybara-webkit over PhantomJS?
3. Which of the two is the most efficient tool?
4. Others ...

Aileenailene

Answer 1

poltergeist is the capybara driver for PhantomJS, a headless browser which is built on WebKit. capybara-webkit is a capybara driver which uses WebKit directly.

poltergeist/PhantomJS has some big advantages over capybara-webkit:

• In my experience poltergeist/PhantomJS always times out when it should, whereas capybara-webkit sometimes hangs.
• Its error messages are much clearer. For example, it will actually tell you if it can't click an element that is on the page because there's another element in front of it.
• It can be told to re-raise Javascript errors as test errors (by instantiating the driver with js_errors: true).
• PhantomJS is much easier to install than standalone WebKit. PhantomJS provides a nearly dependency-free executable that you can download, while standalone WebKit has many OS library dependencies which you may have to upgrade or otherwise fiddle with.

Kithara

Bullet #2 saved my day. I switched from capybara-webkit to poltergeist because of your answer, and now I recommend it. I still didn't measure the performance changes, but it seemed good so far. – Septa

This makes for a more complete test suite since it also catches javascript errors and fails the test. – Cisterna

Answer 2

TL;DR

• Poltergeist/PhantomJS is easier to set up
• Poltergeist/PhantomJS has less dependencies
• Capybara-webkit is more stable and reliable and it’s better for CI

Long:

I have been using Poltergeist + PhantomJS for more than one year. My largest project has a lot of Ajax calls, file uploads, image manipulations, JS templates and pure CSS3 animations.

From time to time, Poltergeist and PhantomJS generated random errors.

Some of them were my mistakes. Testing Ajax is tricky. A common error was that at the end of the successful test the database_cleaner gem truncated the database, however, one Ajax call was still running and generated exception in the controller due the empty database. This isn’t always easy to fix unless you want to use sleep(). (I do not).

However, many errors with Poltergeist were not my mistakes. I have a test which does the same thing 30 times (for a good reason) and once in a while 1 of the 30 times it didn’t work. Poltergeist did not click on the button at all. It was a visible, not animated, normal button. I could fix it (by clicking on it again), however, that’s an ugly hack and feels wrong.

Sometimes the script that worked in all browsers generated random javascript errors with Poltergeist/PhantomJS. About 1 or 2 of 100 times.

With two different Ajax uploader plugin I have experienced that PhantomJS 1.9 and 2.0 behaves differently. 2.0 is more stable and consistent but it’s far from being perfect.

This was a huge pain with Jenkins. About every third run was a failure because 1 or 2 of the 400 features (js browser tests) generated random errors.

Two weeks ago I tried Capybara-webkit. It took me a couple of hours to migrate since they treat invisible elements differently. Capybara-webkit is more correct or strict in this. I noticed the same about overlapping elements.

Testing Ajax uploading and image manipulation requires custom scripts that I had to modify for Capybara-webkit.

I’m using Mac OS X for development, FreeBSD for production and Linux for Jenkins. Capybara-webkit was more complicated to set up than Poltergeist because it requires a screen and it has many dependencies. Only PhantomJS is truly headless and standalone. I could run PhantomJS on production servers if I wanted. I would not do that with capybara-webkit because of the dependencies.

Now I have 100% stable Jenkins CI. All the random javascript errors are the memories of the past. Capybara-webkit always clicks on the button I want it to click on. Javascript always works fine. Currently I have about 20-25 stable builds in a straight line.

For projects with a lot of Ajax, I recommend capybara-webkit.

My advice is based on the current, up to date versions in Aug, 2015.

Ashleighashlen

Thanks! Not having errors in CI is what I wanted to solve for. Intermittent/Random errors are notoriously difficult to debug, and suck when you have a large team and a lot of capybara request specs. – Holds

Interesting that you find capybara-webkit to be more strict than poltergeist. I found the opposite when migrating in the opposite direction. Possibly they are stricter about different things. – Kithara

Answer 3

capybara-webkit and PhantomJS both use Webkit under the hood to render web pages headlessly, i.e., without the need for a browser. They're different tools, however:

• capybara-webkit serves as an adapter for Capybara, a Ruby gem that lets you write and perform high-level UI testing for a Rails or Rack app.
• PhantomJS is a lower level tool that simply lets you run scripts against a web page. It can also be used to write UI tests as well (see Casper, for instance, or any of the other testing tools that build upon PhantomJS).

Hargeisa

Answer 4

PhantomJS does not support HTML5 features like Audio/Video which really sucks.

Blush