How to pass the value in search on url https://www.virustotal.com/gui/home/search using Selenium and Python

Question

driver.get("https://www.virustotal.com/gui/home/search")
sbox = driver.find_element_by_id("searchInput")
sbox.send_keys("129.226.130.245")
sbox.send_keys(Keys.ENTER)

Please suggest, on how to pass values to search box

Kingpin

Answer 1

The problem here is that you can't find elements that lie within a #shadowroot. You can fix this by finding all the shadowroots that contain the element you are looking for. In each of the shadowroots you will need to use javascript's querySelector and find the next shadowroot, until you can access the element you were looking for.

Do the following to access the search input you were looking for:

driver =webdriver.Chrome()
driver.get("https://www.virustotal.com/gui/home/search")

# wait a bit untill search pops up
time.sleep(2)

# Retrieve the last shadowroot using javascript
javascript = """return document
.querySelector('vt-virustotal-app').shadowRoot
.querySelector('home-view').shadowRoot
.querySelector('vt-ui-search-bar').shadowRoot
.querySelector('vt-ui-text-input').shadowRoot"""
shadow_root = driver.execute_script(javascript)

# Find the input box
sbox = shadow_root.find_element_by_id("input")
sbox.send_keys("129.226.130.245")
sbox.send_keys(Keys.ENTER)

Lazybones

Answer 2

The search field with placeholder text as URL, IP address, domain, or file hash within the website https://www.virustotal.com/gui/home/search is located deep within multiple #shadow-root (open).

---

Solution

To send a character sequence to the search field you have to use shadowRoot.querySelector() and you can use the following Locator Strategy:

• Code Block:

  from selenium import webdriver
  import time
  
  options = webdriver.ChromeOptions() 
  options.add_argument("start-maximized")
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
  driver.get("https://www.virustotal.com/gui/home/search")
  time.sleep(7)
  search_field = driver.execute_script("return document.querySelector('vt-virustotal-app').shadowRoot.querySelector('vt-auth-checker home-view').shadowRoot.querySelector('vt-ui-search-bar').shadowRoot.querySelector('vt-ui-text-input').shadowRoot.querySelector('input#input')")
  search_field.send_keys("129.226.130.245")
  

• Browser Snapshot:

---

References

You can find a couple of relevant discussions in:

• Unable to locate the Sign In element within #shadow-root (open) using Selenium and Python

Inurbane

Answer 3

From what I can see, the element you are trying to search is not an input element.

You would be interested in INPUT tag, rather than some div. So you need to be more specific, something like

driver.findElement( By.xpath( "//div[@id='searchInput']//input" ) )

This syntax may not be correct, as its not tested running program. But you may want to refer this thread in order to get more precise answer. Locating child nodes of WebElements in selenium.

Hope this helps.

Macomber

I tried both ways but no luck sbox = driver.find_element_by_xpath('//div[@id='searchInput']//input') sbox.send_keys("129.226.130.245") sbox.send_keys(Keys.ENTER) sbox = driver.find_element_by_xpath('//*[@id="input"]') sbox.send_keys("129.226.130.245") sbox.send_keys(Keys.ENTER) – Kingpin

@id="input" winn not work, isnt it? input is type, not id. – Macomber

Answer 4

AFAICT, the website you're trying to automate isn't capable for getting Selenium WebDriver to interact with. It could be from the security/ firewall form its side to detect Selenium Webdriver as the scrape bot or sth... The reason why i'm saying that, because even when i used the Chrome Dev Tools to query from the Console, i was unable to get the element, e.g, getting the Search button on that site:

document.getElementsByClassName('search-button')

then it returned nothing.

I think there's nothing wrong in your scripts but the website itself.

Matchwood

Thank you for looking into this. Is there any alternative solution to get this working – Kingpin

Answer 5

The previously submitted answers from @undetectedSelenium and @DannKlijn clearly identify the fundamental problem: the OP had not accounted for navigating the multiple nested shadow-roots involved.
This answer is just an update, based on new features associated with Selenium 4.1 which allow a solution without invoking javascript. Per: Shadow DOM in Selenium, one can directly reference the shadow-roots in selenium:

url = "https://www.virustotal.com/gui/home/search"
driver.get(url)
WebDriverWait(driver, timeout=10).until(document_initialised)  

# based on "new" features for handling shadowroots in Selenium 4.1
try:
    elm= driver.find_element(By.CSS_SELECTOR,'home-view') \
        .shadow_root \
        .find_element(By.CSS_SELECTOR,'vt-ui-search-bar') \
        .shadow_root \
        .find_element(By.CSS_SELECTOR,'input#searchInput')

    elm.send_keys('enter URL of interest here') 
    elm.send_keys(Keys.ENTER)

except Exception as x:
    print('Error : ', x)

Afterdeck