Python Selenium Webdriver: Repeating the same information in for loop

Question

I am trying to retrieve text from the website (https://www.doc.govt.nz/parks-and-recreation/places-to-go/otago/places/dunedin-area/?tab-id=50578). I am trying to extract information present on the website. Here is my code that is retrieving the text:

driver = webdriver.Chrome(driverLocation)
driver.get('https://www.doc.govt.nz/parks-and-recreation/places-to-go/otago/places/dunedin-area/?tab-id=50578')
driver.implicitly_wait(20)

for element in driver.find_elements_by_xpath('//div[@class="profile-detail"]'):

    desc = element.find_element_by_xpath('//div[@class="profile-detail-body"]').text
    info = element.find_element_by_xpath('//div[@class="profile-info"]').text

    print(desc)
    print(info)

The problem is that it always repeats the first entry of the information present on the page (i.e. Allans Beach Track information). When I tried to retrieve the information using two separate loops, it works fine. Could you please guide me where am I making the mistake?

mds731 · Accepted Answer · 2020-02-12 10:48:16Z

The idea is to understand the structure of the elements and then define your scripting strategy .

I believe you are trying to access each list item present in the list "class="profileRepeater" which has multiple entries of "class="profile-info" and "profile-detail-body".

driver = webdriver.Chrome(driverLocation)
driver.get('https://www.doc.govt.nz/parks-and-recreation/places-to-go/otago/places/dunedin-area/?tab-id=50578')
driver.implicitly_wait(20)

parentElement = driver.find_element_by_class_name('profileRepeater')

for listItemElement in parentElement.find_element_by_tag_name('li'):

    desc = listItemElement.find_element_by_class_name('profile-detail-body').text
    info = listItemElement.find_element_by_class_name('profile-info').text

    print(desc)
    print(info)

this should print all of the different values in search results list items

Guy · Accepted Answer · 2020-02-12 10:52:03Z

0

When using xpath to locate an element from another element you need to specify current context . before the second path

for element in driver.find_elements_by_xpath('//div[@class="profile-detail"]'):
    desc = element.find_element_by_xpath('.//div[@class="profile-detail-body"]').text
    info = element.find_element_by_xpath('.//div[@class="profile-info"]').text

You can use class_name instead without any conditions

for element in driver.find_elements_by_class_name('profile-detail'):
    desc = element.find_element_by_class_name('profile-detail-body').text
    info = element.find_element_by_class_name('profile-info').text

edited Feb 12, 2020 at 10:52

answered Feb 12, 2020 at 10:46

Guy

51.2k10 gold badges49 silver badges96 bronze badges

Collectives™ on Stack Overflow

Python Selenium Webdriver: Repeating the same information in for loop

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related