Selenium error message "selenium.webdriver has no attribute execute script"

Question

I am using selenium to scrape an infinite scrolling page.

I am trying to use this code:

import time
import pandas as np
import numpy as np

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

browser = webdriver.Chrome()
url = 'https://twitter.com/search?f=tweets&q=csubwaystats%20since%3A2018-05-28%20until%3A2018-08-28'

browser.get(url)
time.sleep(1)

SCROLL_PAUSE_TIME = 0.5

# Get scroll height
last_height = webdriver.execute_script("return document.body.scrollHeight")

while True:
    # Scroll down to bottom
    webdriver.execute_script("window.scrollTo(0,document.body.scrollHeight);")

    # Wait to load page
    time.sleep(SCROLL_PAUSE_TIME)

    # Calculate new scroll height and compare with last scroll height
    new_height = webdriver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
        break
    last_height = new_height

I obtained this code from multiple sources, the most recent being:

How can I scroll a web page using selenium webdriver in python?

I updated it to include "webdriver" instead of "driver" because I import selenium as webdriver. It doesn't work otherwise.

My issue is that when I run the code I get:

AttributeError: module 'selenium.webdriver' has no attribute 'execute_script'

I don't really understand what this means and how to fix it? I haven't been able to find information on this.

I am new to python and so am probably missing something obvious but any advice would be appreciated.

Cho

please update the question with the exact code you use – Gautea

Just did so. Thank you! – Cho

Answer 1

webdriver is the name of the module, not your instance of it. In fact, you assigned the instance you created to the name browser with this line: browser = webdriver.Chrome()

so instead of calling webdriver.execute_script() (which will give you an AttributeError), you must call it using your instance, like this: browser.execute_script().

Gautea

Answer 2

To make it work you have to create an instance of webdriver, e.g.:

from selenium import webdriver

driver = webdriver.Chrome() # webdriver.Ie(), webdriver.Firefox()...
last_height = driver.execute_script("return document.body.scrollHeight")

You can download Chromedriver from here

You also need to add path to Chromedriver to your environment variable PATH or just put downloaded file into the same folder as your Python executable...

Caryl

Ahh, that's my bad. I didn't include the code where I did just this. I have from selenium import webdriver driver = webdriver.Chrome('location of chromedriver') ... and then I continued with what was above – Cho

Sorry I am not sure how to format comments properly – Cho

@agra94 , you should not just define driver, but use it in following lines (note that I used driver.execute_script(), not webdriver.execute_script()) – Caryl

Answer 3

AttributeError: module 'selenium.webdriver' has no attribute 'execute_script'

You are getting this error because 'execute_script' is not a class attribute, you just can not use it directly. Since it is an instance attribute you should create an instance of the class. Please check here to learn more about classes.

This will work fine now since 'execute_script' is running as an instance attribute.

last_height = browser.execute_script("return document.body.scrollHeight")

Your final code would have looked like this:

import time
import pandas as np
import numpy as np

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

browser = webdriver.Chrome()
url = 'https://twitter.com/search?f=tweets&q=csubwaystats%20since%3A2018-05-28%20until%3A2018-08-28'

browser.get(url)
time.sleep(1)

SCROLL_PAUSE_TIME = 0.5

# Get scroll height
last_height = browser.execute_script("return document.body.scrollHeight")

while True:
    # Scroll down to bottom
    webdriver.execute_script("window.scrollTo(0,document.body.scrollHeight);")

    # Wait to load page
    time.sleep(SCROLL_PAUSE_TIME)

    # Calculate new scroll height and compare with last scroll height
    new_height = webdriver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
        break
    last_height = new_height

Sloane

"Do not import selenium as webdriver, import webdriver from selenium"... Considering exception log OP has already imported webdriver from selenium. "You need the webdriver class"... webdriver is not a class, but module where WebDriver class is defined. "selenium is a python program that has python classes in it"... No comments :) – Caryl

Sorry, I should have clarified. I did use 'from selenium import webdriver' (without the quotations of course) I also did create an instance of webdriver but I use chrome. I have it in my path as well. – Cho

@Caryl Please forgive my ignorance about that, I'm a student who is trying to give answers which my efforts in them. I should not have bring up things I do not know though. – Sloane

@agra94 Well, I am sorry that I did not read your question properly. It is abvious that you already can automate stuff with Chrome, it is my bad. – Sloane

@CoreyGoldberg I have fixed indentation. – Sloane

Answer 4

For others check your function name. For me I wrote Java function name not the Python one

driver.execute_script("script") # Python
driver.ExecuteScript("script"); # Java

Posting this here because its the top google result for the error

Lotic