Selenium Element not visible exception

Question

I have been tasked with writing a parser to click a button on a website and I am having issues to click only one of the buttons. The following code works on every button except one.

Here's the html: http://pastebin.com/6dLF5ru8

here's the source html: http://pastebin.com/XhsedGLb

python code:

 driver = webdriver.Firefox()  
 ...
 el = driver.find_element_by_id("-spel-nba")
 actions.move_to_element(el)
 actions.sleep(.1)
 actions.click()
 actions.perform()

I am getting this error.

ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with

as per Saifur I just tried waits with the same element not visible exception:

wait = WebDriverWait(driver, 10)
wait.until(EC.presence_of_element_located((By.XPATH, "//input[contains(@id,'spsel')][@value='nba']"))).click()

Capelin

selenium-python.readthedocs.org/en/latest/waits.html – Copal

Thanks, I just tried wait and it timed out – Capelin

No matching class name in the html you provided. You are using wrong selector – Dactyl

Saifur, I just tried to edit. It's back to where it was when I first posted. – Capelin

Try to use findelements with same selector and see how many elements it returns. I wonder if there are any hidden elements with same selector. It's a very common issue – Dactyl

Padraic, it's for a customer behind a private login. – Capelin

len(driver.find_elements_by_id("-spsel-nba")) is 1 – Capelin

Can you try with the xpath I provided? – Dactyl

like this: driver.find_element_by_xpath("//input[contains(@id,'spsel')][@value='nba']") – Capelin

use explcit wait before this like wait = WebDriverWait(driver, 10) wait.until(EC.presence_of_element_located((By.XPATH, "//input[contains(@id,'spsel')][@value='nba']"))) – Dactyl

any iframe on this page? – Dactyl

No, iframe on the page. – Capelin

is it timing out @ EC? and may ask the you the reason of not using click()? – Dactyl

no, it's not timing out on EC. when I click click it throws the same exception. wait.until(EC.presence_of_element_located((By.XPATH, "//input[contains(@id,'spsel')][@value='nba']"))).click() – Capelin

@Capelin Last try before I give up :) driver = webdriver.Firefox() ... wait = WebDriverWait(driver, 10) wait.until(EC.presence_of_element_located((By.XPATH, "//div[@class='sport-selectors']//input[contains(@id,'spsel')][@value='nba']"))).click() driver.find_element_by_xpath("//div[@class='sport-selectors']//input[contains(@id,'spsel')][@value='nba']").click() – Dactyl

Same exception. Oh well, client and his damn requests! – Capelin

Is it possible to add screenshots? – Dactyl

Let me revert back to the client and tell him. Not sure if he wants it out there. – Capelin

I've resolved this issue. If you look at the page source, almost all elements are dynamically created by JavaScript, which webdriver doesn't recognise them. A workaround is to inject some JavaScripts to perform an artificial click ;) – Praedial

Answer 1

If you look at the page source, you'll understand that almost all of theSELECT, DIV elements are faked and created from JavaScript, that is why webdriver cannot SEE them.

There's a workaround though, by using ActionChains to open your developer console, and inject an artificial CLICK on the desired element, which in fact, is the Label triggering the NBA data loading... here's a working example:

from selenium import webdriver
from selenium.webdriver.common import action_chains, keys
import time

driver = webdriver.Firefox()
driver.get('Your URL here...')
assert 'NBA' in driver.page_source
action = action_chains.ActionChains(driver)

# open up the developer console, mine on MAC, yours may be diff key combo
action.send_keys(keys.Keys.COMMAND+keys.Keys.ALT+'i')
action.perform()
time.sleep(3)
# this below ENTER is to rid of the above "i"
action.send_keys(keys.Keys.ENTER)
# inject the JavaScript...
action.send_keys("document.querySelectorAll('label.boxed')[1].click()"+keys.Keys.ENTER)
action.perform()

Alternatively to replace all the ActionChains commands, you can simply run execute_script like this:

driver.execute_script("document.querySelectorAll('label.boxed')[1].click()")

There you go, at least on my local file anyway... Hope this helps!

Praedial

@user4450305, no problem and I'm glad to help ;) This is actually a very tricky one, I've spent 10 mins on the Browser Inspector trying... before I looked at the page source and figured out what happened. +1 for your question. – Praedial

@Praedial Is action multi-browser compitble? and, can you please explain the javascript part a little bit? just for my understanding – Dactyl

@Saifur, yes action is multi-browser compatible, I normally use this to see all the interactive movements between code & browser, it's like a chain of actions you're seeing yourself doing directly. The execute_script is just a more direct way executing the javascript directly. And about the JavaScript, since the data in the site is basically manipulated by javascript and dynamically feeding the data and build the DOM. webdriver will not see them if they're either set to invisible or disabled, for example. In that case you'd want to INJECT the script to make the desire action happen – Praedial

I might have a similar problem: ElementNotVisibleException: error. Do you know any way to do it in Robot Framework? – Ottar

@Ottar - I haven't used Robot framework before, but notice that selenium is integrated. So in theory whatever applies to selenium should work for your Robot framework too. Let me know how you get on. – Praedial

@Praedial but RobotFramework uses only Selenium2 Library (written in Python) and I have no idea how to implement it. – Ottar

@Ottar - a search in SO returns this which may be what you're looking for. Once you get the driver handle, you can simply do driver.execute_script("javscript code here...") – Praedial

@Praedial I found RobotFramework's ExtendedSelenium2Library (rickypc.github.io/robotframework-extendedselenium2library/doc/…), which has a keyword "Execute Javascript" that executes JS code or a whole JS file. So I will now probably just need a pure javascript code. I will try to play with it. – Ottar

@Praedial I manage to execute javascript code with this "Execute Javascript" keyword from ExtendedSelenium2Library and now it works (despite the "WebDriverException"). JS code looks like this: var elem = document.getElementsByClassName("MyClass")[1]; elem.click().perform(); Thanks for your help! – Ottar

Answer 2

What worked for me was to find the element just before the problematic one (that is, just before it in terms of tab order), then call Tab on that element.

from selenium.webdriver.common.keys import Keys
elem = br.find_element_by_name("username")
elem.send_keys(Keys.TAB) # tab over to not-visible element

After doing that, I was able to send actions to the element.

Maris

Answer 3

The actual solution of this thread did not work for me.

however,

this one did :

element  = WebDriverWait(driver, 3).until(EC.visibility_of_element_located((By.XPATH, xpaths['your_xpath_path'])))

the trick is to use :

EC.visibility_of_element_located

the WebDriverWait

WebDriverWait

from this import :

from selenium.webdriver.support import expected_conditions as EC

from selenium.webdriver.support.ui import WebDriverWait

Swadeshi

Answer 4

I suggest you use xpath with explicit wait

//input[contains(@id,'spsel')][@value='nba']

Dactyl

same error:ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with – Capelin

@Saifur, normally it should work, but on this occasion if you check the page source, almost all of the elements are faked/dynamically created by JavaScript, webdriver cannot see them – Praedial

Answer 5

if "Element is not currently visible" then make it VISIBLE

f.e.

>>> before is hidden top is outside of page
<input type="file" style="position: absolute;top:-999999" name="file_u">


>>> after move top on in page area
DRIVER.execute_script("document.getElementByName('file_u').style.top = 0;")
time.sleep(1); # give some time to render
DRIVER.find_element_by_name("file_u").send_keys("/tmp/img.png")

Pr

Answer 6

Instead of get_element_by_id() you can try elem = browser.find_element_by_css_selector('#elemId') (go to that webpage and the element, right click it and Copy CSS Selector, or something like that.) This is what i did and it works. You also try find_element_by_link_text(text), find_element_by_partial_link_text(text), find_element_by_tag_name(tagName_case_insensitive_here), find_element_by_name(name) etc. Something will work. After the id the CSS Selector is your best bet.

Mccartan

Answer 7

I ended up using @twasbrillig's solution, but instead of finding the previous element and sending a TAB keypress, I find the desired element, send a TAB keypress with that element, and then a SHIFT + TAB keypress to the driver:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()  
el = driver.find_element_by_id("-spel-nba")
el.send_keys(Keys.TAB)
webdriver.ActionChains(driver).key_down(Keys.SHIFT).send_keys(Keys.TAB).key_up(Keys.SHIFT)

Back

Answer 8

I tried using the other methods but in the end found that the simplest way was to just try and click the button, and catch the error. This allows me to perform other actions based on if it worked (True) or didn't (False).

def click_button(html_object):
 try:
  html_object.click()
 except:
  return False #most likely because it is NotVisible object and can be ignored
 return True

...
...
click_button(actions)

Shaynashayne

Answer 9

The way I solved this in python was:

try:
    # the element you want to scroll to
    element = driver.find_element_by_id("some_id") 
    ActionChains(driver).move_to_element(element).perform()
    element.send_keys(Keys.TAB).key_up(Keys.SHIFT)
    #element.click()
except Exception as e:
    print(e)

Exarch