How to “scroll down” some part using selenium in python?

Question

Hope you are good I m trying to make an simple script but I got stuck on there I am trying to scroll the list to get more but i am unable to scroll down. Anybody have an idea how is that done.

here is my code:

import selenium
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from shutil import which
import time
import pandas as pd
import json
# from fake_useragent import UserAgent
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
chrome_path = which('chromedriver')
driver = webdriver.Chrome(executable_path=chrome_path)
driver.maximize_window()

driver.get('http://mapaescolar.murciaeduca.es/mapaescolar/#')

driver.find_element_by_xpath('//ul[@class="nav property-back-nav floating-box pull-right"]//button').click()
time.sleep(3)
driver.find_element_by_xpath('//button[@ng-click="openCloseFilters()"]').click()
time.sleep(3)
driver.find_element_by_xpath('//select[@title="Enseñanza"]/option[1]').click()



element = driver.find_element_by_xpath('//div[@id="container1"]')
driver.execute_script("return arguments[0].scrollIntoView(true);", element)

And the list I want to scroll down :

Tupiguarani

Answer 1

Because the scroll is actually inside an element, all the javascript command with window. will not work. Also the element is not interactable so Key down is not suitable too. I suggest using scrollTo javascript executor and set up a variable which will increase through your loop:

element = driver.find_element_by_xpath('//div[@id="container1"]')
time.sleep(10)

verical_ordinate = 100
for i in range(0, 50):
   print(verical_ordinate)
   driver.execute_script("arguments[0].scrollTop = arguments[1]", element, verical_ordinate)
   verical_ordinate += 100
   time.sleep(1)

I have tested with chrome so it should work.

Reference

https://developer.mozilla.org/en-US/docs/Web/API/Element/scrollTop

Checked

Answer 2

Try this :

    SCROLL_PAUSE_TIME = 0.5

# Get scroll height
last_height = driver.execute_script("return document.body.scrollHeight")

while True:
    # Scroll down to bottom
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

    # Wait to load page
    time.sleep(SCROLL_PAUSE_TIME)

    # Calculate new scroll height and compare with last scroll height
    new_height = driver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
        break
    last_height = new_height

Or simply select your element in tis case the div containing the scroll view and type this :

label.sendKeys(Keys.PAGE_DOWN);

Negative

Answer 3

Simply change the script to arguments[0].scrollTop = arguments[0].scrollHeight. You can call that script in some loop with a timeout to continuously fetch more data (that is continuously scrolling the div to bottom)

Example:

while True:
    time.sleep(1)
    driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", element)
    if no_new_data_available():
        break

Velasco

Answer 4

In your case that there is a list of items, so you can follow this method :

for c in range(1, 12): # check the list start from 0 or 1 
    time.sleep(5)  # Give time to loading the information
    element = driver.find_element_by_xpath(f'//*[@id="grid-search-results"]/ul/li[{c}]') # variable c refer to next item
    driver.execute_script("arguments[0].scrollIntoView();", element)

Cumbersome

Answer 5

# Use this line in a loop, accordingly how much screen to be scrolled down
# this just scrolls down to the height of browser screen, hence loop.

driver.execute_script("window.scrollTo(0,document.body.scrollHeight)")

# Now if the website needs time to load its data after scroll, add this in the loop..

time.sleep(5)

Unreserved

Answer 6

element.location_once_scrolled_into_view

I used this when trying to access an element that wasnt't because needed to be scrolled down.

Bohlin

Answer 7

Try this one:

driver.execute_script("window.scrollTo(0, 300);")

Flossieflossy