WebDriver click() vs JavaScript click()

Question

The Story:

Here on StackOverflow, I've seen users reporting that they cannot click an element via selenium WebDriver "click" command and can work around it with a JavaScript click by executing a script.

Example in Python:

element = driver.find_element_by_id("myid")
driver.execute_script("arguments[0].click();", element)

Example in WebDriverJS/Protractor:

var elm = $("#myid");
browser.executeScript("arguments[0].click();", elm.getWebElement());

The Question:

Why is clicking "via JavaScript" works when a regular WebDriver click does not? When exactly is this happening and what is the downside of this workaround (if any)?

^{I personally used this workaround without fully understanding why I have to do it and what problems it can lead to.}

Surbased

Answer 1

Contrarily to what the currently accepted answer suggests, there's nothing specific to PhantomJS when it comes to the difference between having WebDriver do a click and doing it in JavaScript.

The Difference

The essential difference between the two methods is common to all browsers and can be explained pretty simply:

• WebDriver: When WebDriver does the click, it attempts as best as it can to simulate what happens when a real user uses the browser. Suppose you have an element A which is a button that says "Click me" and an element B which is a div element which is transparent but has its dimensions and zIndex set so that it completely covers A. Then you tell WebDriver to click A. WebDriver will simulate the click so that B receives the click first. Why? Because B covers A, and if a user were to try to click on A, then B would get the event first. Whether or not A would eventually get the click event depends on how B handles the event. At any rate, the behavior with WebDriver in this case is the same as when a real user tries to click on A.

• JavaScript: Now, suppose you use JavaScript to do A.click(). This method of clicking does not reproduce what really happens when the user tries to click A. JavaScript sends the click event directly to A, and B will not get any event.

Why a JavaScript Click Works When a WebDriver Click Does Not?

As I mentioned above WebDriver will try to simulate as best it can what happens when a real user is using a browser. The fact of the matter is that the DOM can contain elements that a user cannot interact with, and WebDriver won't allow you to click on these element. Besides the overlapping case I mentioned, this also entails that invisible elements cannot be clicked. A common case I see in Stack Overflow questions is someone who is trying to interact with a GUI element that already exists in the DOM but becomes visible only when some other element has been manipulated. This sometimes happens with dropdown menus: you have to first click on the button the brings up the dropdown before a menu item can be selected. If someone tries to click the menu item before the menu is visible, WebDriver will balk and say that the element cannot be manipulated. If the person then tries to do it with JavaScript, it will work because the event is delivered directly to the element, irrespective of visibility.

When Should You Use JavaScript for Clicking?

If you are using Selenium for testing an application, my answer to this question is "almost never". By and large, your Selenium test should reproduce what a user would do with the browser. Taking the example of the drop down menu: a test should click on the button that brings up the drop down first, and then click on the menu item. If there is a problem with the GUI because the button is invisible, or the button fails to show the menu items, or something similar, then your test will fail and you'll have detected the bug. If you use JavaScript to click around, you won't be able to detect these bugs through automated testing.

I say "almost never" because there may be exceptions where it makes sense to use JavaScript. They should be very rare, though.

If you are using Selenium for scraping sites, then it is not as critical to attempt to reproduce user behavior. So using JavaScript to bypass the GUI is less of an issue.

Alluvium

Much better answer, this should be the accepted one. The answer above is talking about an edge case specific to PhantomJS, this one is much more accurate IMHO. – Polysyndeton

@Polysyndeton definitely. Marked as accepted. Perfect and a quite detailed answer. – Surbased

Awarding the bounty to Linh as planned, but I will start a new one to thank you for a yet another awesome answer. Thanks. – Surbased

Very good and comprehensible answer. In my experience WebDriver has had a lot of problems with AngularJS pages: With some elements WebDriver methods like click or sendKeys worked - but not always. There was no locating problem or other exception, the test case simply did not progress any further. Logging showed that the action was executed. Sometimes using Action helped. If I clicked on the element manually (after test case had stopped), everything worked fine. So we switched to raw JS on that occasions. I haven't worked with AngularJS for the last 2 years, so things might be better now. – Maurene

@Louis: You said "When WebDriver does the click, it attempts as best as it can to simulate what happens when a real user uses the browser.". Is that your opinion, or do you have a reference for that statement...? – Buettner

@Buettner I don't have a handy link. My answer is derived from experience with Selenium in particular, and with simulating user events in general. I have started using Selenium 5 years ago. During the time I've used Selenium, I have had to read Selenium's code to fix some problems and I have filed quite a few bug reports about event dispatching and discussed the bugs with the Selenium developers. "as best as it can" is the goal. I've certainly encountered (now fixed) bugs that prevented it from reaching that goal. Some bugs may remain. – Alluvium

Answer 2

The click executed by the driver tries to simulate the behavior of a real user as close as possible while the JavaScript HTMLElement.click() performs the default action for the click event, even if the element is not interactable.

The differences are:

• The driver ensures that the element is visible by scrolling it into the view and checks that the element is interactable.

  The driver will raise an error:

  • when the element on top at the coordinates of the click is not the targeted element or a descendant
  • when the element doesn't have a positive size or if it is fully transparent
  • when the element is a disabled input or button (attribute/property disabled is true)
  • when the element has the mouse pointer disabled (CSS pointer-events is none)

  
  A JavaScript HTMLElement.click() will always perform the default action or will at best silently fail if the element is a disabled.
  
  

• The driver is expected to bring the element into focus if it is focusable.

  A JavaScript HTMLElement.click() won't.
  

• The driver is expected to emit all the events (mousemove, mousedown, mouseup, click, ...) just like like a real user.

  A JavaScript HTMLElement.click() emits only the click event. The page might rely on these extra events and might behave differently if they are not emitted.

  These are the events emitted by the driver for a click with Chrome:

  mouseover {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
  mousemove {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
  mousedown {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
  mouseup {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
  click {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
  

  And this is the event emitted with a JavaScript injection:

  click {target:#topic, clientX:0, clientY:0, isTrusted:false, ... }
  

• The event emitted by a JavaScript .click() is not trusted and the default action may not be invoked:

  https://developer.mozilla.org/en/docs/Web/API/Event/isTrusted
  https://googlechrome.github.io/samples/event-istrusted/index.html

  Note that some of the drivers are still generating untrusted events. This is the case with PhantomJS as of version 2.1.
  
  

• The event emitted by a JavaScript .click() doesn't have the coordinates of the click.

  The properties clientX, clientY, screenX, screenY, layerX, layerY are set to 0. The page might rely on them and might behave differently.

It may be ok to use a JavaScript .click() to scrap some data, but it is not in a testing context. It defeats the purpose of the test since it doesn't simulate the behavior of a user. So, if the click from the driver fails, then a real user will most likely also fail to perform the same click in the same conditions.

What makes the driver fail to click an element when we expect it to succeed?

• The targeted element is not yet visible/interactable due to a delay or a transition effect.

  Some examples :

  https://developer.mozilla.org/fr/docs/Web (dropdown navigation menu) http://materializecss.com/side-nav.html (dropdown side bar)

  Workarrounds:

  Add a waiter to wait for the visibility, a minimum size or a steady position :

  // wait visible
  browser.wait(ExpectedConditions.visibilityOf(elem), 5000);
  
  // wait visible and not disabled
  browser.wait(ExpectedConditions.elementToBeClickable(elem), 5000);
  
  // wait for minimum width
  browser.wait(function minimumWidth() {
      return elem.getSize().then(size => size.width > 50);
  }, 5000);
  

  Retry to click until it succeeds :

  browser.wait(function clickSuccessful() {
      return elem.click().then(() => true, (ex) => false);
  }, 5000);
  

  Add a delay matching the duration of the animation/transition :

  browser.sleep(250);
  

  
  

• The targeted element ends-up covered by a floating element once scrolled into the view:

  The driver automatically scrolls the element into the view to make it visible. If the page contains a floating/sticky element (menu, ads, footer, notification, cookie policy..), the element may end-up covered and will no longer be visible/interactable.

  Example: https://twitter.com/?lang=en

  Workarounds:

  Set the size of the window to a larger one to avoid the scrolling or the floating element.

  Mover over the element with a negative Y offset and then click it:

    browser.actions()
       .mouseMove(elem, {x: 0, y: -250})
       .click()
       .perform();
  

  Scroll the element to the center of the window before the click:

  browser.executeScript(function scrollCenter(elem) {
    var win = elem.ownerDocument.defaultView || window,
      box = elem.getBoundingClientRect(),
      dy = box.top - (win.innerHeight - box.height) / 2;
    win.scrollTo(win.pageXOffset, win.pageYOffset + dy);
  }, element);
  
  element.click();
  

  Hide the floating element if it can't be avoided:

  browser.executeScript(function scrollCenter(elem) {
    elem.style.display = 'none';
  }, element);
  

Libratory

This has been a super helpful solution! Thank you – Overshoe

Answer 3

NOTE: let's call 'click' is end-user click. 'js click' is click via JS

Why is clicking "via JavaScript" works when a regular WebDriver click does not?

There are 2 cases for this to happen:

I. If you are using PhamtomJS

Then this is the most common known behavior of PhantomJS . Some elements are sometimes not clickable, for example <div>. This is because PhantomJS was original made for simulating the engine of browsers (like initial HTML + CSS -> computing CSS -> rendering). But it does not mean to be interacted with as an end user's way (viewing, clicking, dragging). Therefore PhamtomJS is only partially supported with end-users interaction.

WHY DOES JS CLICK WORK? As for either click, they are all mean click. It is like a gun with 1 barrel and 2 triggers. One from the viewport, one from JS. Since PhamtomJS great in simulating browser's engine, a JS click should work perfectly.

II. The event handler of "click" got to bind in the bad period of time.

For example, we got a <div>

• -> We do some calculation

• -> then we bind event of click to the <div>.

• -> Plus with some bad coding of angular (e.g. not handling scope's cycle properly)

We may end up with the same result. Click won't work, because WebdriverJS trying to click on the element when it has no click event handler.

WHY DOES JS CLICK WORK? Js click is like injecting js directly into the browser. Possible with 2 ways,

Fist is through devtools console (yes, WebdriverJS does communicate with devtools' console).

Second is inject a <script> tag directly into HTML.

For each browser, the behavior will be different. But regardless, these methods are more complicating than clicking on the button. Click is using what already there (end-users click), js click is going through backdoor.

And for js click will appear to be an asynchronous task. This is related a with a kinda complex topic of 'browser asynchronous task and CPU task scheduling' (read it a while back can't find the article again). For short this will mostly result as js click will need to wait for a cycle of task scheduling of CPU and it will be ran a bit slower after the binding of the click event. (You could know this case when you found the element sometimes clickable, sometimes not. )

When exactly is this happening and what is the downside of this workaround (if any)?

=> As mention above, both mean for one purpose, but about using which entrance:

• Click: is using what providing by default of browser.
• JS click: is going through backdoor.

=> For performance, it is hard to say because it relies on browsers. But generally:

• Click: doesn't mean faster but only signed higher position in schedule list of CPU execution task.
• JS click: doesn't mean slower but only it signed into the last position of schedule list of CPU task.

=> Downsides:

• Click: doesn't seem to have any downside except you are using PhamtomJS.
• JS click: very bad for health. You may accidentally click on something that doesn't there on the view. When you use this, make sure the element is there and available to view and click as the point of view of end-user.

P.S. if you are looking for a solution.

• Using PhantomJS? I will suggest using Chrome headless instead. Yes, you can set up Chrome headless on Ubuntu. Thing runs just like Chrome but it only does not have a view and less buggy like PhantomJS.
• Not using PhamtomJS but still having problems? I will suggest using ExpectedCondition of Protractor with browser.wait() (check this for more information)

(I want to make it short, but ended up badly. Anything related with theory is complicated to explain...)

Lag

Answer 4

Thank you for the good explanation, I was hitting the same issue and your explanation helped to resolve my issue.

button = driver.wait.until(EC.presence_of_element_located(
    (By.XPATH, "//div[@id='pagination-element']/nav[1]/span[3]/button[1]/span[1]/i[1]")
))
driver.execute_script("arguments[0].click();", button)

Pfosi

Answer 5

if (theElement.Enabled)
{
    if (!theElement.Selected)
    {
      var driver = (IJavaScriptExecutor)Driver;
      driver.ExecuteScript("arguments[0].click();", theElement); //ok

      //theElement.Click();//action performed on theElement, then pops exception   
     }
 }

I do not agree we will "almost never" use JS to simulate the click action.

Above theElement.Click(), we'll check the Radio button but then Exception pops as above image.

Actually, this is no page load action after the Click, the click is just to select the Radio button, and I don't know why the WebDriver Click() will cause this exception, can anyone explain why this exception occurred.

I use the Webdriver 3.141.59 and IE 11 and selenium-server-standalone-3.141.59.jar for a remote test.

Lidstone