StaleElementReferenceException on Python Selenium

Question

I am getting the following error while using Selenium in python:

selenium.common.exceptions.StaleElementReferenceException: Message: u'stale element reference: element is not attached to the page document\n

Interestingly enough, the error pops up at different times in the for loop. Sometimes it gets through eg. 4 iterations and other times eg. 7.

Some of the relevant code being run is:

for i in range(0, 22):
    u = driver.find_elements_by_id("data")
    text = u[0].get_attribute("innerHTML")
    driver.find_elements_by_class_name("aclassname")[0].click()

What does this error mean and what is something I can try to fix this?

Artemisia

The error means "i cant find the element" more or less. The way to fix this is to slow it down somehow. maybe through implicit or explicit waits. – Augite

OK. What is an implicit wait? – Artemisia

Or, better yet, how would I do an explicit wait until the aclassname element is loaded? – Artemisia

This is already answered here: https://mcmap.net/q/182218/-how-to-navigate-to-a-new-webpage-in-selenium/3124333 – Signalize

@Signalize no answers solves the problem. As we get StaleElementReferenceException after 2-3 iterations. Can you give me exact solution – Partly

A very simple way that helped me, when I faced this problem was to implement time.sleep(xx) inside the loop. – Dyslalia

Answer 1

It means the element is no longer in the DOM, or it changed.

The following code will help you find the element by controlling and ignoring StaleElementExceptions and handling them just like any other NoSuchElementException. It waits for the element to NOT be stale, just like it waits for the element to be present. It also serves as a good example on how to properly wait for conditions in Selenium.

from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions

my_element_id = 'something123'
ignored_exceptions=(NoSuchElementException,StaleElementReferenceException,)
your_element = WebDriverWait(your_driver, some_timeout,ignored_exceptions=ignored_exceptions)\
                        .until(expected_conditions.presence_of_element_located((By.ID, my_element_id)))

To better understand the problem, imagine you are inside a for loop and think what happens during the iterations:

1. something changes when you click on the element (last line)
2. So the page is changing
3. You enter the next iteration. Now are trying to find a new element (your first line inside the loop).
4. You found the element
5. It finishes changing
6. You try to use it by getting an attribute
7. Bam! The element is old. You got it in step 4, but it finished changing on step 5

Stoneblind

Good discussion in general. Steps 4-6 might be incredibly subtle, too... A page I was scraping had a dropdown on it that, at some point in load, was being replaced by an exact copy of itself, so there was a race condition in selecting an <option>, clicking it, and submitting the form that resulted in one of several possible errors. – Iago

I have struggled for days with a stale element reference exception until I stumbled across your answer. After implementing ignored_exceptions, my code works and obtains all the requested data every time. Thanks! – Solecism

@Stoneblind M. this answers doesn't work me. It gives Timeout exception. Please help if there is another solution – Partly

@VishavGupta, if it gives you that error it means your ID was not properly set or the element is not present. Try filtering by other selectors instead of By.ID... try css, or xpath... – Stoneblind

@EmilioM. I have tried other selectors also but problem remains same. – Partly

@VishavGupta, we have no way of helping you, as your problem is in your code. The solution shown above works fine, and it's the correct way of handling Stale Elements. Good luck – Stoneblind

I additionally used a loop timer to check whether url has changed after click() operation. – Knowles

@EmilioM. how so I extend this to work on an href attribute? I'm trying to get all links on a page and this error keeps coming up – Illuminant

Change the By.ID to fit your needs, like css, or xpath. Read the By class documentation to get the details. – Stoneblind

still getting StaleElementReferenceException when doing WebDriverWait(your_driver, some_timeout,ignored_exceptions=ignored_exceptions)\ .until(expected_conditions.presence_of_element_located((By.ID, my_element_id))).click() as mentioned above. notice the .click() – Codex

Answer 2

Selenium Support Explicit and Implicit Waits. If you think waiting for certain amount of time is enough for your page to be loaded, use:

driver.implicitly_wait(secs)

but if you want to wait for a special event (e.g. waiting for a particular element to be loaded) you can do something like:

from selenium.webdriver.support.ui import WebDriverWait
...
...
def find(driver):
    element = driver.find_elements_by_id("data")
    if element:
        return element
    else:
        return False
element = WebDriverWait(driver, secs).until(find)

Indistinct

Answer 3

Beyond the answers here, if you are using ActionChains, and the page has changed, be sure to reinstantiate your ActionChains object (dont reuse an old one), otherwise your ActionChain will be using a stale DOM. I.e. do this;

action_chain = ActionChains(driver)     
action_chain.double_click(driver.find_element_by_xpath("//tr[2]/p")).perform()

Or better yet dont use an instantiation;

ActionChains(driver).double_click(driver.find_element_by_xpath("//tr[2]/p")).perform()

Soult

Hmm, I do not use instantiation, so follow your "blue coloured" code example, but still get this error. – Domestic

it's not working for me... – Leonie

Answer 4

For my python scripts, on quite simple pages, all above mentioned solutions didn't work. I found here -> https://www.softwaretestingmaterial.com/stale-element-reference-exception-selenium-webdriver/ the simplest way for my problem. It's just refreshing before clicking.

driver.refresh()
driver.find_element_by_id('normal').click()

Note: I'm using driver.implicitly_wait(5) instead of explicit waits but it works in both options.

Doubletongue

Answer 5

>>>Stale Exceptions can be handled using **StaleElementReferenceException** to continue the for loop execution.  

from selenium.common import exceptions  

# and customize your code of for loop as:  

for i in range(0, 22):  
   try:  
        u = driver.find_elements_by_id("data")  
        text = u[0].get_attribute("innerHTML")  
        driver.find_elements_by_class_name("aclassname")[0].click()  
   except exceptions.StaleElementReferenceException,e:
        print(e)
        pass  

Note: Python 3+ : replace exceptions.StaleElementReferenceException,e -> exceptions.StaleElementReferenceException as e

Joubert

Answer 6

I Would like to add one more solution here which is worked for me.

I was trying to access the button in the top menu panel on my webpage after refreshing the content area on the same page, Which gave me the following error,

    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <span id="dk1-combobox" class="dk-selected combinationText visibleElements "> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

Then I started to search for the solution to click the stale element on the web page. After two days of thinking and googling, I got a solution.

To access the stale element on the page, first, we need to focus the mouse over the particular section and perform click option

EditReport1 = driver.find_element_by_id('internalTab1')
ActionChains(driver).move_to_element(EditReport1).click(EditReport1).perform()

move_to_element will move the mouse over the stale element which we need to access once we got our control on the element the click operation is successfully performed.

This is worked for me. If anyone finds it working please comment your's as well which will help some other in future.

Thank you

Linetta

I tried it but this work for only for maximum three iterations. Please give some other solution as StaleElementReferenceException is very annoying – Partly

Answer 7

When webpage got refreshed or switched back from different window or form and trying to access an element user will get staleelementexception.

Use webdriverwait in try --except block with for loop: EX :

Code in which I got staleelementexception :

---

driver.find_element_by_id(tc.value).click()

driver.find_element_by_xpath("xpath").click()

---

Fix :

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver.find_element_by_id(tc.value).click()
for i in range(4):
   try:
        run_test = WebDriverWait(driver, 120).until( \
        EC.presence_of_element_located((By.XPATH, "xpath")))
        run_test.click()
        break
   except StaleElementReferenceException as e:
        raise e

Ossy

Why bother using a try/except block if all you do is raising the incoming exception? – Pedo

Answer 8

In my case, the problem

stale element reference: element is not attached to the page document

was because I tried to use actions.move_to_element(element).perform() on actions that created from old Selenium tab.

So the solution is to create new actions instance after opening new tab:

actions = ActionChains(browser)
actions.move_to_element(element).perform()

Fontana

Answer 9

from selenium.webdriver.support.expected_conditions import staleness_of

Import above saved the day for me.

Lalitta

Answer 10

It depends on the Website you are working with.

If you have a Website, that doesn't change itself, use Emilio's answer. WebDriverWait(...).until(...) is the correct / best solution for Websites, that don't change by themselves (and only by user interactions):

WebDriverWait(driver, delay, ignored_exceptions=(NoSuchElementException,StaleElementReferenceException)).until(expected_conditions.presence_of_element_located((By.ID, "my_id"))).click()

(For explanation see Emilio's answer)

If you work with a site, that changes by pushs from server or time based or something similar, the only working solution is retrying like in Rashids answer.

def retry_func(func, max_tries=MAX_TRIES):
    for i in range(max_tries):
        try:
            return func()
        except Exception as e:
            if i >= max_tries - 1:
                raise e


def click_button_helper(driver, selector, selector_type=By.XPATH, delay=DELAY, ignored_exceptions=IGNORED_EXCEPTIONS):
    WebDriverWait(driver, delay, ignored_exceptions=ignored_exceptions).until(
        EC.presence_of_element_located((selector_type, selector))).click()


def click_button(driver, selector, selector_type=By.XPATH, delay=DELAY, ignored_exceptions=IGNORED_EXCEPTIONS,
                 max_tries=MAX_TRIES):
    retry_func(lambda: click_button_helper(driver, selector, selector_type, delay, ignored_exceptions), max_tries)

Because, sorry @Emilio, your answer doesn't address specific cases like the one Sylvan LE DEUNFF mentioned.

The following race condition is still possible (and not solved by WebDriverWait(...).until(...), but usually by retrying):

1. The page is changing (e.g. after a click like in the question)
2. We are trying to find the element.
3. WebDriverWait(...).until(...) waits for us till our expected_conditions are fullfilled (so basically till it finishes changing)
4. We found the element
5. It starts changing again (by push, by reaching timestep, ...)
6. We try to use it
7. Bam! The element is old again.

Dervish

Answer 11

One scenario in my case : I queryed the element once , and assign it to an variable for reusage,

captchaInput = driver.find_element_by_css_selector('input[name="CaptchaCode"]')

then it runs the risk of StaleElementReferenceException, querying the element again with the above code help eliminate the exception.

Spiv

Answer 12

This is the python solution for this problem:

def clickAndCatchStaleRefException(locator):



    driver = sel2._current_browser()
    result = False
    attempts = 0

    locator = locator[6:]
    # This line is optional because sometimes you pass a xpath from a varibles file
    # that starts with 'xpath='. This should be omitted otherwise the find_element_by_xpath 
    # function will throw an error.
    # But if you pass an xpath directly you don't need this


    while attempts < 2:
        try:
            driver.find_element_by_xpath(locator).click()
            result = True
            break
        except EC as e:
            raise e
        finally:
            attempts += 1
    return result

Munitions

This is not a good solution. WebDriver has selenium.webdriver.support.wait.WebDriverWait and selenium.webdriver.support.expected_conditions to solve this problem properly, as specified in the other answer. Retrying to get an element may work, but it's not the correct way of doing it. – Stoneblind

I agree, but it doesn't work all the time. For example if you work with an application using jquery, some elements could be replaced unexpectedly and StaleElementReferenceException will be raised even after WebDriverWait is complete. In these particular cases, in never heard about a better solution than trying until you get the expected result. (Note : i am not using selenium to perform automated tests, but to scrap odds corporate's apps) – Ewers

Sylvan, If you check my answer above, it handles the specific case you are mentioning. That way is the best way to do it. You still wait, but use the standard tools Selenium provides instead of custom code. It's not only cleaner, but also safer, since it's a proven solution that works for everyone. – Stoneblind

Answer 13

I was getting the exception on trying to click multiple classes with the same name.

    for i in range(0, 5):
    actions = ActionChains(driver)
    button = driver.find_element_by_class_name('dlJyA')
    time.sleep(2)
    if button.is_displayed():
        actions.click(on_element=button).perform()

I fixed it by using time.sleep and checking if the button is displayed.

Gruel

Answer 14

I made a script to fill out a webform with data from an excel worksheet. But there were a few questions that would still have a stale element error

I kept getting a stale element exception error using the following code:

from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import StaleElementReferenceException

ElementName = WebDriverWait(browser, 20).until(
                 EC.presence_of_element_located((By.ID, 'ElementID')))
browser.execute_script("arguments[0].scrollIntoView(true);", ElementName)
ElementName.click()

I found this forum and tried several of the other solutions but no success. So after an hour, I got frustrated and decided to use a simple try-except condition and it worked.

Here is the code I used:

from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import StaleElementReferenceException

ElementName = WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.ID, 'ElementID')))
browser.execute_script("arguments[0].scrollIntoView(true);", ElementName)
        
        try: 
            ElementName.click()

        except StaleElementReferenceException as e:
            ElementName = WebDriverWait(browser, 20).until(
            EC.presence_of_element_located((By.ID, 'ElementID')))
            ElementName.click()

This refreshes the element if it's stale without using time.sleep() or refreshing the whole page using driver.refresh() (which would erase all of my earlier work on the webform I'm trying to fill out).

It's not as short or sexy as the other answers but it's simple and quick.

Although this method gets bulky if you have a lot of stale element clicks. So I also made a function for this try-exception:

from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import StaleElementReferenceException

def staleClick(ElementID):
    try: 
        ElementName.click()

    except StaleElementReferenceException as e:
        ElementName = WebDriverWait(browser, 20).until(
        EC.presence_of_element_located((By.ID, 'ElementID')))
        ElementName.click()   

# when running the code
ElementName = WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.ID, 'ElementID')))
browser.execute_script("arguments[0].scrollIntoView(true);", ElementName)

staleClick(ElementID)

Imaimage

Answer 15

Happened to me too in a while loop. Simple solution was just to wait until the element is stale before setting the variable again. Works like that:

if not first:
   WebDriverWait(driver, 10).until(EC.staleness_of(Element))

Element = ...

Collaborative

Answer 16

I am aware this is an old question, but depending on your purpose for using selenium, I use to different solutions. I the task is for webscraping I have the body of the webpage after all the javascript I need run have finished, and a BeautifulSoup object.

BeautifulSoup(webdriver.find_element_by_tag_name('body').get_attribute('innerHTML'))

The second is to use the Page Object Model (POM). See the link https://www.lambdatest.com/blog/page-object-model-in-selenium-python/

Overweary

This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review – Infrequent

Answer 17

In my case, I tried to find element, then pass it to another class constructor for reuse. when I call 'click' on it - I got StaleException

So I've just pass css class to constructor instead of object and find element in new class

Oskar

Answer 18

Adding time.sleep(3) resolved the issue for me.

for i in range(0, 22):
    u = driver.find_elements_by_id("data")
    text = u[0].get_attribute("innerHTML")
    driver.find_elements_by_class_name("aclassname")[0].click()
    time.sleep(3)

Olney

Thanks for your comment Prakash, glad to know it helped you – Olney

Answer 19

I've resolved the issue by setting the variable at the start of the while loop:

should_continue = True
while should_continue:
    driver.find_elements(By.CSS_SELECTOR, ".grayed b")

Indamine