WebScraping with Python Selenium

Question

I am trying to do WebScraping on the Frankfurter Allgemeine Zeitung Archiv. I need to click on the button 'erweiterte suche' but I keep getting the below error

Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="g_00271e6200ad4985a0a9c8f9b0032c42"]"}.

I have my code here:

import os
import requests
import openpyxl
import smtplib
import time

from selenium import webdriver

driver.get("https://fazarchiv.faz.net")

search_elem = driver.find_element_by_xpath("""//*[@id="searchForm"]/div[1]/input""")
search_elem.click()
search_elem.send_keys('Bürokratie')

erweiterte_suche = driver.find_element_by_xpath("""//*[@id="g_00271e6200ad4985a0a9c8f9b0032c42"]""")
erweiterte_suche.click()

What am I doing wrong?

I would suggest not using selenium altogether because it's not being updated anymore. It's too obsolete — Anonumous Pomp
– Anonumous Pomp, Commented Nov 18, 2019 at 12:08
@AnonumousPomp What do you mean? Selenium is de-facto standard of autonomous interaction. — Alexey R.
– Alexey R., Commented Nov 18, 2019 at 12:18

NewbieCoder · Accepted Answer · 2019-11-18 11:57:58Z

1

I guess, you want to click "Suche" button. The gray one.

So you can search the word "Bürokratie".

Your code also have missing parts. Like driver setting.

This code work for me, you can use:

    from selenium import webdriver
    driver = webdriver.Firefox()

    driver.get("https://fazarchiv.faz.net")

    search_elem = driver.find_element_by_xpath("""//*[@id="searchForm"]/div[1]/input""")
    search_elem.click()
    search_elem.send_keys('Bürokratie')

    erweiterte_suche = driver.find_element_by_css_selector("""#searchForm > div.boxRow.clearfix > input""")
    erweiterte_suche.click()

answered Nov 18, 2019 at 11:57

NewbieCoder

712 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Guy · Accepted Answer · 2019-11-18 11:45:49Z

0

The id is dynamic and changes every time the site is loaded. Use class instead

erweiterte_suche = driver.find_element_by_xpath('//*[@class="boxExtendedSearch ext-hidden globalArrowRedRight"]')

answered Nov 18, 2019 at 11:45

Guy

51.2k10 gold badges49 silver badges96 bronze badges

Comments

Alexey R. · Accepted Answer · 2019-11-18 11:50:31Z

0

Try this:

erweiterte_suche = driver.find_element_by_xpath("//div[@class='clearfix']/a[1]")

or this:

erweiterte_suche = driver.find_element_by_xpath("//a[text()='erweiterte Suche']")

answered Nov 18, 2019 at 11:50

Alexey R.

8,7362 gold badges13 silver badges32 bronze badges

Collectives™ on Stack Overflow

WebScraping with Python Selenium

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related