6

I am trying to scrape an airbnb listing. I cant figure out a way to get the full list of amenities other than clicking on "more". I am using selenium to simulate the click, but it does nt seem to work.

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains

url = 'https://www.airbnb.com/rooms/4660676'
driver = webdriver.Firefox()
driver.get(url)
elem = driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]')
actions.click(elem).perform()
2
  • What do you mean by "doesn't seem to work"? What happens? Nothing? Do you get an error? Commented Jan 22, 2016 at 16:10
  • you haven't define your actions instance of ActionChains so line actions.click(elem).perform() have no sense Commented Jan 22, 2016 at 20:31

2 Answers 2

9

The XPath itself is correct, but you have not defined the actions:

from selenium.webdriver.common.action_chains import ActionChains

elem = driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]')

actions = ActionChains(driver)
actions.click(elem).perform()

Note that you can simply use the WebElement's click() method instead:

elem = driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]')
elem.click()

Works for me.


If you are getting NoSuchElementException error, you might need to wait for the link to be clickable via WebDriverWait and element_to_be_clickable.

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

element = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, '//a[@class="expandable-trigger-more"]'))
)
element.click()
Sign up to request clarification or add additional context in comments.

4 Comments

@Borat14 works for me, you are doing smth more than just clicking more button probably. Post the complete code you have so far. Thanks.
Thanks @alecxe, on the first methods i get this error: WebDriverException: Message: Element is not clickable at point (661.5, 10.316665649414062). Other element would receive the click: <button type="submit" class="btn btn-primary btn-large" id="sticky-md-book-it-btn"></button> on the last two methods i get "string indices must be integers"
This is the code from selenium import webdriver from selenium.webdriver.common.action_chains import ActionChains url = 'https://www.airbnb.com/rooms/4660676' driver = webdriver.Firefox() driver.get(url) elem = driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]') actions = ActionChains(driver) actions.click(elem).perform()
@Borat14 the "string indexes must be integers" is because of the selenium 2.49 bug, downgrade to 2.48: pip install selenium==2.48 (stackoverflow.com/questions/34969006/…).
1

A very simple way of achieving this is given below

driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]').click()

It works for me hope will work for you as well.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.