From a technical perspective, how does Selenium click an element on a web page?

Asked 30/10, 2015 at 13:50 Answered 1/12, 2015 at 13:38

Solved selenium selenium-webdriver web-component polyfills remotewebdriver

Context is provided in case anyone knows of an alternative way to solve the larger issue.

Problem Context

I am spearheading the development of a test automation framework for a web application which uses Web Components. This has presented a problem when testing in Internet Explorer, because Internet Explorer does not support Web Components natively; instead, a polyfill is used to provide this functionality.

A primary repercussion of this is that much of Selenium will not work as expected. It cannot 'see' the Shadow DOM in Internet Explorer the way it can in Firefox and Chrome.

The alternative is to write a test framework which provides an alternate mechanism for accessing elements via JavaScript - this allows elements to be located through the polyfill.

Our current implementation checks the WebDriver being used, and either uses the original Selenium implementation of a method (in the case of Chrome or Firefox), or our own alternative implementation (in the case of Internet Explorer).

This means that we want our implementation to be as close as possible to Selenium's implementation, at its core, browser-interacty, level.

Problem

I am trying to replicate the functionality of Actions.click(WebElement onElement) (source), in a simplified form (without following the Builder design pattern of the Actions class, and making assumptions that the click is with the left mouse button and no other keys (Ctrl, Shift, Alt) are being held down).

I want to find the core code which handles the click does (specifically in Chrome, Firefox, and Internet Explorer), so I can replicate it as closely as possible, however I've found myself lost in a deep pit of classes and interfaces...

A new ClickAction (source) is created (to later be performed). Performing this includes a 'click()' call on an instance of the Mouse interface (source) ... aaaaand I'm lost. I see from generated JavaDoc that this is implemented by either EventFiringMouse (source) or HtmlUnitMouse (source), but I'm not sure which one will be implemented. I made an assumption (with little basis) that HtmlUnitMouse would be used, which has led me further down the rabbit hole looking at HTMLUnit code from Gargoyle Software...

In short, I am totally lost.

Any guidance would be much appreciated :)

Research

I have found that I was incorrect in my assumption that HTMLUnit is used by Chrome, Firefox, and Internet Explorer. Documentation shows that RemoteWebDriver (source) is subclassed by ChromeDriver, FirefoxDriver, and InternetExplorerDriver.

Biz answered 30/10, 2015 at 13:50 Comment(14)

The WebDriver implementation decides which one, there are ChromeDriver, FirefoxDriver, InternetExplorerDriver, HtmlUnitDriver, others.. Each one behaves differently, according to the underlying browser. So the first step is to decide one of them and dig deeper. – Fibrinolysis 30/10, 2015 at 16:50

Have you considered contacting the Selenium developers, and possibly working with them to enhance Selenium itself? – Exine 30/10, 2015 at 20:43

@Exine We had considered that (for the larger problem). It (that is, the Polyfills issue) was raised as an issue on Selenium GitHub several weeks ago and acknowledged by another poster, however there has been no recent activity. We have limited time and resources and given the circumstances, writing a workaround was considered the best option for us. – Biz 2/11, 2015 at 10:6

@AhmedAshour I've investigated both EventFiringMouse and HtmlUnitMouse in some depth however I was unable to make much progress with either. – Biz 2/11, 2015 at 10:8

Can you update your question with complete case with code (Java and HTML), using HtmlUnitDriver, or with HtmlUnit? – Fibrinolysis 2/11, 2015 at 10:24

As discussed above, I am looking for the general case of how the click itself in Actions.click(WebElement element) is executed. I am not looking at a specific case. I am looking at how this is done in Chrome, Firefox, and Internet Explorer (I am not sure if I was mistaken in my assumption that they use HtmlUnit...) – Biz 2/11, 2015 at 10:34

@AhmedAshour I have updated the question above to include the 'big three' web browsers (Chrome, Firefox, and Internet Explorer) as specific focuses. – Biz 2/11, 2015 at 10:37

Chrome, FF, and IE drivers are all RemoteDriver, which means the actual code is on the server side of your call. None uses HtmlUnit which is a browser simulator, and it doesn't use the RemoteDriver. I suggest that you use a browser that handles WebComponents, and if it doesn't correctly handle click, open a bug to the relevant team. – Fibrinolysis 2/11, 2015 at 10:59

@AhmedAshour We require to test on each of these three web browsers. As detailed above, we have raised an issue with Selenium however we do not have the resources to wait an unknown length of time for this to be fixed by an external team. I appreciate the clarification that they use RemoteDriver, but "just raise a bug with someone else and stop working on the problem" is not a helpful solution for us. – Biz 2/11, 2015 at 11:7

Shortly it's using closure-library.googlecode.com/git-history/docs/… , github.com/SeleniumHQ/selenium/blob/… . Google closure library is the key atom-level library, on which 90% of functionality is built. – Contracted 5/11, 2015 at 14:23

@Stanjer Is this true for all three browsers? – Biz 5/11, 2015 at 14:58

@Biz sorry, gave you the wrong link, here is the correct one github.com/SeleniumHQ/selenium/blob/… Briefly, Closure library is th every bottom level of the whole architecture, which is providing most of atomic operations, from getting attributes to synthesizing elements clicking. This realization and some extra logic called "atoms" in Selenium (you can see it from the path). Yes, for all the browsers. Another question is how would you inject this to each browser(and every has it's own way). – Contracted 5/11, 2015 at 15:13

@Biz ChromeDriver has supported ShadowDOM since version 2.14 . Also, instead of writing your own, I would suggest starting with a framework such as Selenide. The owner of Selenide project could probably tell you if he has ever tried shadow dom actions. Also, keep in mind that Selenium has "native" actions as well as "non-native" Javascript actions; important to realize the difference as you scan the code. – Exceptive 5/11, 2015 at 17:28

@Eilidh: You want to understand how WebElement.click works? – Ethno 19/11, 2015 at 10:28

The Essentials

The drivers for Chrome, Firefox, and Internet Explorer are all RemoteWebDrivers.

This means that any actions which Selenium performs are sent to the browser (the WebDriver), via an HttpRequest.

Once the request is received by the browser, it will perform the action as either a "native event" or synthetically. How a browser executes an action depends on the capabilities of the browser (and potentially a flag option).

"Native" events are OS-level events.

Actions executed synthetically are executed using JavaScript. "Automation Atoms" are used - as one infers from 'atom', they are small, simple functions to perform low-level actions.

References

RemoteWebDriver is subclassed by ChromeDriver, FirefoxDriver, InternetExplorerDriver, OperaDriver, and SafariDriver. (reference)
All implementations of WebDriver that communicate with the browser, or a RemoteWebDriver server shall use a common wire protocol. This wire protocol defines a RESTful web service using JSON over HTTP. (reference)
In WebDriver advanced user interactions are provided by either simulating the JavaScript events directly (i.e. synthetic events) or by letting the browser generate the JavaScript events (i.e. native events). Native events simulate the user interactions better whereas synthetic events are platform independent [...] Native events should be used whenever possible. (reference)
Browser Automation Atoms are building blocks intended to be used by Selenium implementations. By using the same pieces throughout the codebase, rather than reimplementing required functionality in multiple places, the project can reduce the number of bugs found, and can simplify the process of adding new functionality and drivers. (reference)

Automation Atoms

A summary of the available Automation Atoms
The raw JavaScript code for the Automation Atoms - this may serve as a useful starting point in developing simpler synthetic events, if necessary.

Biz answered 1/12, 2015 at 13:38 Comment(1)

Here's an updated link to the click code: github.com/SeleniumHQ/selenium/blob/master/javascript/atoms/… – Bloody 22/1, 2019 at 1:8

The wiki for the Selenium IE Driver states that it uses native events rather than JavaScript events to interact with the browser.

As the InternetExplorerDriver is Windows-only, it attempts to use so-called "native", or OS-level events to perform mouse and keyboard operations in the browser. This is in contrast to using simulated JavaScript events for the same operations.

Except for clicking <option> elements, where it uses JavaScript.

The IE driver handles this one scenario by using the click() Automation Atom, which essentially sets the .selected property of the element and simulates the onChange event in JavaScript.

Reach answered 24/11, 2015 at 12:2 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags