Iterate through table rows and print column text with Python Selenium

Question

I have a table (<table>) with values in each row (<tr>) from its body (<tbody>).

The value I would lile to print out is in the <span> inside a <div> tag.

Inspecting the html, I see the value e.g. "Name" is in row 1 (tr[1]), column 2 (td[2]):

<tr class="GAT4PNUFG GAT4PNUMG" __gwt_subrow="0" __gwt_row="0">
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUHG GAT4PNUNG">
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUNG">
                <div __gwt_cell="cell-gwt-uid-324" style="outline-style:none;">
                    <span class="linkhover" title="Name" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;color:#00A;cursor:pointer;">Name</span>
                </div>
            </td>

I would like to loop through the table each row and print out the value in columns 2, td[2]

I am using Python with Selenium Webdriver

The full Xpath to the table row 1, column 2 is:

html/body/div[2]/div[2]/div/div[4]/div/div[2]/div/div[3]/div/div[5]/div/div[3]/div/div[4]/div/div[2]/div/div[4]/div/div[3]/div/div[2]/div/div/table/tbody/tr[1]/td[2]/div/span

I was thinking if i can start from the table, xpath as follows:

html/body/div[2]/div[2]/div/div[4]/div/div[2]/div/div[3]/div/div[5]/div/div[3]/div/div[4]/div/div[2]/div/div[4]/div/div[3]/div/div[2]/div/div/table/tbody

I can then use a for loop and use an index for the tr and td e.g for row1 use tr[i], for col2 use td[2].

html/body/div[2]/div[2]/div/div[4]/div/div[2]/div/div[3]/div/div[5]/div/div[3]/div/div[4]/div/div[2]/div/div[4]/div/div[3]/div/div[2]/div/div/table/tbody/tr[i]/td[2]/div/span

How can i loop through this table and print out the value of the Span class tag which is always in column 2 of the table?

I tried to get the start of the table into a variable and then I could maybe use this to loop through the rows and columns. I need some help please.

table = self.driver.find_element(By.XPATH, 'html/body/div[2]/div[2]/div/div[4]/div/div[2]/div/div[3]/div/div[5]/div/div[3]/div/div[4]/div/div[2]/div/div[4]/div/div[3]/div/div[2]/div/div/table/tbody')

Here's the full HTML:

    <table cellspacing="0" style="table-layout: fixed; width: 100%;">
    <colgroup>
    <tbody>
        <tr class="GAT4PNUFG GAT4PNUMG" __gwt_subrow="0" __gwt_row="0">
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUHG GAT4PNUNG">
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUNG">
                <div __gwt_cell="cell-gwt-uid-324" style="outline-style:none;">
                    <span class="linkhover" title="Name" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;color:#00A;cursor:pointer;">Name</span>
                </div>
            </td>
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUNG">
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUNG">
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUNG">
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUBH GAT4PNUNG">
        </tr>
        <tr class="GAT4PNUEH" __gwt_subrow="0" __gwt_row="1">
            <td class="GAT4PNUEG GAT4PNUFH GAT4PNUHG">
            <td class="GAT4PNUEG GAT4PNUFH">
                <div __gwt_cell="cell-gwt-uid-324" style="outline-style:none;">
                    <span class="linkhover" title="Address" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;color:#00A;cursor:pointer;">Address</span>
                </div>
            </td>
            <td class="GAT4PNUEG GAT4PNUFH">
            <td class="GAT4PNUEG GAT4PNUFH">
            <td class="GAT4PNUEG GAT4PNUFH">
            <td class="GAT4PNUEG GAT4PNUFH GAT4PNUBH">
        </tr>
        <tr class="GAT4PNUFG" __gwt_subrow="0" __gwt_row="2">
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUHG">
            <td class="GAT4PNUEG GAT4PNUGG">
                <div __gwt_cell="cell-gwt-uid-324" style="outline-style:none;">
                    <span class="linkhover" title="DOB" style="white-space:nowrap;overflow:hidden;text-overflow:ellipsis;empty-cells:show;display:block;color:#00A;cursor:pointer;">DOB</span>
                </div>
            </td>
            <td class="GAT4PNUEG GAT4PNUGG">
            <td class="GAT4PNUEG GAT4PNUGG">
            <td class="GAT4PNUEG GAT4PNUGG">
            <td class="GAT4PNUEG GAT4PNUGG GAT4PNUBH">
        </tr>
        <tr class="GAT4PNUEH" __gwt_subrow="0" __gwt_row="3">
            ---
        <tr class="GAT4PNUFG" __gwt_subrow="0" __gwt_row="4">       
            ---
    </tbody>
</table>

Melatonin

Answer 1

The developer has put an ID into the table. I have it working now. It is printing all the cell values from column 2. The code is:

table_id = self.driver.find_element(By.ID, 'data_configuration_feeds_ct_fields_body0')
rows = table_id.find_elements(By.TAG_NAME, "tr") # get all of the rows in the table
for row in rows:
    # Get the columns (all the column 2)        
    col = row.find_elements(By.TAG_NAME, "td")[1] #note: index start from 0, 1 is col 2
    print col.text #prints text from the element

Melatonin

It won't work if thead has child tr. I recommend adding: tbody = table_id.find_element_by_tag_name('tbody') – Mobility

Answer 2

The XPath you currently using is quite fragile since it depends on the complete document structure and the relative position of the elements. It can easily break in the future.

Instead, locate the rows using their class or other attributes. For instance:

for row in driver.find_elements_by_css_selector("tr.GAT4PNUFG.GAT4PNUMG"):
    cell = row.find_elements_by_tag_name("td")[1]
    print(cell.text)

Eclogite

There is more than 1 table in the html hence i used the full xpath so i can get the right table. There are no ids in the elements. Thanks for your suggestion, trying it now. – Melatonin

@RiazLadhani yeah, this means that you should first locate the correct table (provide an HTML code of it if you need help) and find rows inside it. In other words, use table.find_elements_by_css_selector() instead of driver.find_elements_by_css_selector(). – Eclogite

I have pasted some html now in my question. I forgot to paste when i posted the quezzie. I will try table.find_elements, thanks. – Melatonin

@RiazLadhani good, a follow-up question: how do you distinguish tables between one another? How do you know this table is the desired one? Thanks. – Eclogite

Using firefox I inspect element on the table cell which has the value "Name" i get the full xpath which is: html/body/div[2]/div[2]/div/div[4]/div/div[2]/div/div[3]/div/div[5]/div/div[3]/div/div[4]/div/div[2]/div/div[4]/div/div[3]/div/div[2]/div/div/table/tbody/tr[1]/td[2]/div/span – Melatonin

@RiazLadhani so, correct me if I'm wrong. You need a table where there is at least one span tag with text="Name"? Thanks. – Eclogite

Name is in row 1, column 2 the past part of the xpath: /tr[1]/t‌​d[2]/div/span – Melatonin

The 1st row in the table of column2 has the value Name. The next row in the same column has the value Address, next row has DOB. I want to iterate through the rows of column 2 and print out the value. I can the work out the rest which is to use an assert to check the expected values match the actual value. Thanks – Melatonin

I have just tried the following: table = self.driver.find_elements(By.XPATH, 'html/body/div[2]/div[2]/div/div[4]/div/div[2]/div/div[3]/div/div[5]/div/div[3]/div/div[4]/div/div[2]/div/div[4]/div/div[3]/div/div[2]/div/div/table/tbody') td = self.driver.find_elements(By.TAG_NAME, "td") for i in td: print (td) – Melatonin

I get the following output [<selenium.webdriver.remote.webelement.WebElement object at 0x01A5B690>, <selenium.webdriver.remote.webelement.WebElement object at 0x01A5B5F0>, <selenium.webdriver.remote.webelement.WebElement object at 0x01A5B390>, – Melatonin

I have now tried the following: table_xpath = "html/body/div[2]/div[2]/div/div[4]/div/div[2]/div/div[3]/div/div[5]/div/div[3]/div/div[4]/div/div[2]/div/div[4]/div/div[3]/div/div[2]/div/div/table/tbody" rows = table_xpath.find_elements(By.TAG_NAME, "tr") cols = table_xpath.driver.find_elements(By.TAG_NAME, "td") for rows in cols: print cols[2].text – Melatonin

I get the following error: File "C:\Webdriver\ClearCore 501\Pages\data_objects_saved_page.py", line 85, in verify_variables_created rows = table_xpath.find_elements(By.TAG_NAME, "tr") AttributeError: 'str' object has no attribute 'find_elements' – Melatonin

@Eclogite I can find the table and the rows (tr) with tag_name, but I have to use xpath to find columns (td) under the tr because the td's have attributes that I have to use to filter out whose attributes do not have the specific value. How can I do it? – Cohosh

Answer 3

Probably a little late to this. But heres my code and works like a charm.

def find_in_table(self, name):
        check_table = self.isElementPresent("//table[@class='assessment_list_table_tableStyle__Qw-rz']",
                                            locatorType="xpath")
        while not check_table:
            time.sleep(10)
            check_table = self.isElementPresent("//table[@class='assessment_list_table_tableStyle__Qw-rz']",
                                                locatorType="xpath")

        table_id = self.driver.find_element(By.XPATH, "//table[@class='assessment_list_table_tableStyle__Qw-rz']")
        rows = table_id.find_elements(By.TAG_NAME, "tr")
        for x in range(1, len(rows)):
            col = rows[x].find_elements(By.TAG_NAME, "td")[0]
            s = col.text
            if s == name:
                return x

1. Check whether the table exists
2. Get the table ID with find_elements
3. Using table ID to find the rows in table
4. iterate through the table and finding the text in the first column (0)
5. Returns the row value when text matches the one in column

XPATH of the Table Element can be obtained using the selenium plugin in IntelliJ. The plugin is so useful to find elements and more accurate than the ones in as extension in browsers.

(isElementPresent method is a method I used to check whether an element is present using seleniums getElement method and returning boolean if the element exists)

Arcadia