Does anybody know how to identify shadow dom web elements using selenium webdriver?

Question

We are using selenium web driver and python for our test automation and trying to automate html5 app with shadow dom design. Unable to identify any elements that come under shadow-root. For eg. If I want to access any element under the shadow root given below then how can I do that? Any help is appreciated.

Cappello

https://mcmap.net/q/456550/-shadow-dom-support-for-selenium/954442, #28912299 – Nightspot

Answer 1

You can inject this piece of javascript that does this and then run the find_element methods on that element:

shadow_section = mydriver.execute_script('''return document.querySelector("neon-animatable").shadowRoot''')
shadow_section.find_element_by_css(".flex")

since you use often that you may create a function, , then the above becomes:

def select_shadow_element_by_css_selector(selector):
  running_script = 'return document.querySelector("%s").shadowRoot' % selector
  element = driver.execute_script(running_script)
  return element

shadow_section = select_shadow_element_by_css_selector("neon-animatable")
shadow_section.find_element_by_css(".flex")

on the resulting element you can put any of the methods:

find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector

To find multiple elements (these methods will return a list):

find_elements_by_name
find_elements_by_xpath
find_elements_by_link_text
find_elements_by_partial_link_text
find_elements_by_tag_name
find_elements_by_class_name
find_elements_by_css_selector

later edit:

many times the root elements are nested and the second nested element is no longer available in document, but is available in the current accessed shadow root. I think is better to use the selenium selectors and inject the script just to take the shadow root:

def expand_shadow_element(element):
  shadow_root = driver.execute_script('return arguments[0].shadowRoot', element)
  return shadow_root

#the above becomes 
shadow_section = expand_shadow_element(find_element_by_tag_name("neon-animatable"))
shadow_section.find_element_by_css(".flex")

To put this into perspective I just added a testable example with Chrome's download page, clicking the search button needs open 3 nested shadow root elements:

import selenium
from selenium import webdriver
driver = webdriver.Chrome()


def expand_shadow_element(element):
  shadow_root = driver.execute_script('return arguments[0].shadowRoot', element)
  return shadow_root

selenium.__file__
driver.get("chrome://downloads")
root1 = driver.find_element_by_tag_name('downloads-manager')
shadow_root1 = expand_shadow_element(root1)

root2 = shadow_root1.find_element_by_css_selector('downloads-toolbar')
shadow_root2 = expand_shadow_element(root2)

root3 = shadow_root2.find_element_by_css_selector('cr-search-field')
shadow_root3 = expand_shadow_element(root3)

search_button = shadow_root3.find_element_by_css_selector("#search-button")
search_button.click()

Celestine

Do you happen to know the equivalent of this in geckodriver? Your example does not work there – Centigram

@vnportnoy Can you please post what error you get there? Maybe is a browser issue if you try to expand the shadowDOM in the console of the browser like here: document.querySelector("neon-animatable").shadowRoot – Celestine

It gives a "cyclic object value" error when running this on firefox v70 using geckodriver v24. Definitely a browser issue but I need to use that specific one cause all my other crawlers are in that setup. – Centigram

In my opinion you have a separate issue that might need an entire new question – Celestine

Answer 2

You could try this way also

driver.execute_script('return document.querySelector("neon-animatable").shadowRoot.querySelector(".flex")')

Slough