How can I scroll a web page using selenium webdriver in python?

Question

I am currently using selenium webdriver to parse through facebook user friends page and extract all ids from the AJAX script. But I need to scroll down to get all the friends. How can I scroll down in Selenium? I am using python though.

Answer 1

You have the option of using the below-stated command:

driver.execute_script("window.scrollTo(0, Y)") 

where Y is the height (on a full HD monitor it's 1080).

You can also use

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

to scroll to the bottom of the page.

If you want to scroll to a page with infinite loadings, like a social network, facebook etc. 

 

SCROLL_PAUSE_TIME = 0.5

 

# Get scroll height

last_height = driver.execute_script("return document.body.scrollHeight")

 

while True:

    # Scroll down to bottom

    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

 

    # Wait to load page

    time.sleep(SCROLL_PAUSE_TIME)

 

    # Calculate new scroll height and compare with last scroll height

    new_height = driver.execute_script("return document.body.scrollHeight")

    if new_height == last_height:

        break

    last_height = new_height