Selenium can't find xpath or css selector

Question

I'm trying to scrape this site that has pagination. The problem I'm facing is having selenium locate the next button.

What I've tried:

next_button = driver.find_element_by_css_selector(
    'ul[class="css-12ke8jn e65zztl0"] button[aria-label="Next"]').click()

and

page_amount = driver.find_element_by_css_selector(
    '/html/body/div[1]/div[2]/div/div/div/div[2]/div[1]/main/div[3]/div/div[2]/div[2]/nav/ul/button').click()

None of these work and I'm kinda stuck. The reason I'm using aria-label for the first one is because when the next button is selected the previous button changes to the same class as the next button. Note: The button is inside a ul.

the 'next' button is just an arrow at the bottom of the page if that's what you're asking. — nick
– nick, Commented Jan 23, 2021 at 18:00

Alin Stelian · Accepted Answer · 2021-01-23 19:53:45Z

2

It might not work finding the element because it's not visible in UI - it is loaded but not visible, the easiest way is to move to that element and click on it.

next_button = driver.find_element_by_css_selector('[aria-label=\'Next\']')
actions = ActionChains(driver)
actions.move_to_element(next_button).perform()
next_button.click()

answered Jan 23, 2021 at 19:53

Alin Stelian

8971 gold badge6 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

nick Over a year ago

Sorry for marking this correct so late. I was trying other solutions and yours worked! Also why did you put '\' in the aria-label?

Alin Stelian Over a year ago

The CSS selector value is - [aria-value=‘Next’] and the css selector method from Python also uses single quotes to declare the parameter and in order to work we need to escape the single quote from the css selector. Test it by removing the \ from the code - you’ll get an errror

Karan Verma · Accepted Answer · 2021-01-23 20:08:03Z

1

next_button = driver.find_element_by_xpath('//button[@class="css-1lkjxdl eanm77i0"]').click()

You was using xpath variable and finding it by css. for css selector you have to use the class (.css-1lkjxdl) and use the above code it will work and accept the answer. Thanks!!

answered Jan 23, 2021 at 20:08

Karan Verma

376 bronze badges

2 Comments

nick Over a year ago

How did you get that xpath? The button never gives that when it's enabled or disabled. Your solution works but how..

Karan Verma Over a year ago

I only searched your query using inspect and found the xpath by right click and copy xpath

Laurent LAPORTE · Accepted Answer · 2021-01-23 18:04:42Z

0

aria-label is an attribute, not an element.

Your xpath should be fixed as follow:

button[@aria-label="Next"]

To find this button anywhere in the page, you can try:

//button[@aria-label="Next"]

Then, you can try:

button = driver.find_element_by_xpath('//button[@aria-label="Next"]')

answered Jan 23, 2021 at 18:04

Laurent LAPORTE

23.2k7 gold badges64 silver badges111 bronze badges

3 Comments

nick Over a year ago

For the css_selector I got 'Message: invalid selector: An invalid or illegal selector was specified' and for xpath I just got None

Laurent LAPORTE Over a year ago

Do you use find_element_by_xpath?

nick Over a year ago

Yes and I also tried find_element_by_css_selector

Collectives™ on Stack Overflow

Selenium can't find xpath or css selector

3 Answers 3

2 Comments

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related