How to get link from elements with Selenium and Python

Question

Let's say, all Author/username elements in one webpage look like following... How can I get to the href part using python and Selenium?

users = browser.find_elements_by_xpath(?)

<span>

    Author: 

    <a href="/account/57608-bob">

        bob

    </a>

</span>

Thanks.

Minhminho

Answer 1

Use .//span[contains(text(), "Author")]/a as xpath expression.

For example:

from selenium import webdriver
driver = webdriver.Firefox()
driver.get('http://jsfiddle.net/9pKMU/show/')
for a in driver.find_elements_by_xpath('.//span[contains(text(), "Author")]/a'):
    print(a.get_attribute('href'))

Myel

Answer 2

Use find_elements_by_tag_name('a') to find the 'a' tags, and then use get_attribute('href') to get the link string.

Fannyfanon

Wouldn't I have to parse through all of the links on the page then to find ones that contain "account"? How can I find ones that are just in a span element with Author: ? – Minhminho

@Minhminho Ah, I missed the condition. If you want to find the "account" links, beautifulsoup is very helpful to do such jobs. You may use it like this: soup.findAll('span', text=re.compile(r'Author:') to find the target "span" and then find('a').attrs['href'] to get the link. It's more readable. – Fannyfanon

Ah, I forget something, beautifulsoup is only for page parse, if you want to do some actions in selenium, falsetru's answer is better. – Fannyfanon

URL = driver.find_element_by_tag_name('a').get_attribute('href') is how you return this as a string. – Watch

Answer 3

Using this code you can get the all links from a webpage

from selenium import webdriver
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("https://your website/")
# identify elements with tagname <a>
lnks=driver.find_elements_by_tag_name("a")
# traverse list
for lnk in lnks:
   # get_attribute() to get all href
   print(lnk.get_attribute("href"))
driver.quit()

Elison