Do Watir-Webdriver and Capybara have performance issues when compared to Webdriver?

Question

I am about to change my test automation language from Java to Ruby (I have new job where Ruby fits better into the tech stack).

I have a lot of experience with Java and Webdriver but can see that wrappers such as Watir and Capybara appear to be used more in Ruby than directly accessing the Webdriver API.

My concern about using a such a library is performance. I normally try integrate 3rd party grids such as Saucelabs into my test frameworks but have learnt that caching of selenium web element objects is important as continually finding elements can have a performance impact.

If I use libraries such as Capybara, do I lose the ability to control caching strategies? I have previously investigated Geb and found that the framework continually re-created webelements rather than caching, and it appeared inflexible in changing that behaviour.

Is my concern that these libraries help you to avoid writing boiler plate code but at the cost of performance valid?

Succotash

Can you give an example when you need to continually find an element? I also have a test suite but I think I don't have such an issue there. – Clite

In my experience something as simple as el = find('#id'); el[:id].should == expected_id; el[:name].should == expected_name usually helps – Clite

Most of the time I want to access the element only once so I don't feel need in caching by default. – Clite

Do you use the page object pattern? – Succotash

Not in the form as it's written here - code.google.com/p/selenium/wiki/PageObjects. I use something similar. For example I don't follow rules "Generally don't make assertions", "Methods return other PageObjects", "Different results for the same action are modelled as different methods" – Clite

Answer 1

TL;DR

Convention over configuration; use page-object for caching.

---

Here are my thoughts on this matter. And please consider this less of an answer, and more of a response for discussion. I want feedback on this answer, feel free to give it.

One of the primary patterns in Ruby (and it actually comes from Rails), is Convention over Configuration. The basic idea is that when there is a convention, either dictated by the language or the community, you roll with it whenever possible. In your case, I would recommend using the community frameworks if at all possible. This will make it easier for other developers to use your code and easier to seek help if you need it.

As far as the actual caching goes, with this I am less familiar. I do know the page-object gem stores elements instead of recreating them during every usage*. This would appear to conform with your requirements of caching elements. In any case, I highly recommend this gem as it enforces the page-object model of testing.

Note: I am not sure if the page-factory mixin supports this caching of objects, or if it recreates the class on every usage.

*You can see how the Element is stored in page-object by viewing the source code to the Element class.

def initialize(element, platform)
  @element = element

Hamlett

Can you specify where caching is situated in page-object gem more precisely. I can't find it in linked file. – Clite

page-objects store an @element variable in the initializer. This is kept throughout the lifetime of the instance. So unless the element itself is dropping the reference to the page and searching each time, this means you should only be searching for the element once. – Hamlett

The "convention over configuration" argument is something I had not considered and is very compelling. When in Rome.... – Succotash

Lets write in Python, since it's much easy to find the Python programmer? – Awn

Answer 2

I won't comment on performance in general as I haven't done extensive comparisons (and you shouldn't really believe anyone who can't point to benchmarks). However, here's a performance tip (with or without wrappers):

Internally, WebDriver uses HTTP to send commands to the browser (or an intermediate server). Since the protocol is quite chatty, you can often gain a lot by switching out the default Net::HTTP-based client with one that supports Keep-Alive (assuming the server supports it). That way the Ruby client will open one socket per browser and reuse it for the duration of the session, instead of opening one per command.

Here's code that uses the net-http-persistent gem:

require "selenium/webdriver"
require "selenium/webdriver/remote/http/persistent"

client = Selenium::WebDriver::Remote::Http::Persistent.new
driver = Selenium::WebDriver.for :remote, url: 'http://...', http_client: client

Antrum

Answer 3

Unfortunately, I can't say anything specific about Capybara, but I can about watir-webdriver. By default, it always relocates element on the page, so you can't cache it. This behaviour can be turned off with

Watir.always_locate = false

though you can experience stale elements in this case (it, however, actually depends more on you just like with plain WebDriver). If you're planning you use watir-webdriver with page-object gem, its caching won't work as described in @screenmutt answer by default.

Also, there are two nice blog posts from Zejlko Fillipn about watir-webdriver performance when using SauceLabs and TestingBot.

Another issue of watir-webdriver is cross-browser performance. Because it uses XPath under the hood, you'll face significant performance problems when testing in IE.

Showily

Well done. As I said, I am less familiar with the caching. The majority of my answer was the configuration argument. – Hamlett

So are you say water-WebDriver converts all selectors to XPath? Even CSS selectors? – Succotash

CSS selectors are not converted. However, its support is far from perfect (see github.com/watir/watir-webdriver/issues/124). You can also force watir-webdriver to use CSS selectors by setting Watir.prefer_css = true, in this case it will fall back to XPath only when CSS selector cannot be created (e.g. when locating element by text). – Showily