1

I know that this question was asked many times on stackoverflow. I tried different solutions but did not get it work. Here is a simple MWE to automate the search on Youtube. Any body familiar with this can help explain the raison ?

from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys


options = Options()
options.add_argument('--disable-extensions')
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--remote-debugging-port=9515')
options.add_argument('--disable-setuid-sandbox')
options.add_argument("--start-maximized")

driver = webdriver.Chrome(service=Service("/usr/bin/chromedriver"), options=options)

url = "https://www.youtube.com/"
driver.get(url)

search_area = driver.find_element(By.XPATH, '//*[@id="search"]')

driver.implicitly_wait(10)

search_area.send_keys('Lionel Messi', Keys.ENTER)


print(search_area.text)
12
  • Does this answer your question? Selenium 'send_keys' does not work for YouTube Commented Jun 30, 2022 at 18:03
  • Exactly the same question was just 2 hours ago.... Commented Jun 30, 2022 at 18:03
  • @JaSON, I really choose Youtube just to write the example. It is not my goal. However, If I understand correctly the xpath can not be unique, which triggers the errors. It is really better to trigger the error in find_element Commented Jun 30, 2022 at 18:08
  • I'm not sure I understand what do you mean by "unique XPath"... There can be dozens, hundreds XPaths for the same node. You just should write your own XPath using unique attributes/relations instead of copy/paste from useless Chrome dev-tools helper. The problem with Youtube - there are more than 1 element with the same @id ("search"). It's not normal, but this happens Commented Jun 30, 2022 at 18:15
  • What error are you running into? In the meantime this might help. stackoverflow.com/questions/70814704/… Commented Jun 30, 2022 at 18:16

1 Answer 1

1

There are 2 problems with your code:

  1. You should use unique locator.
    The locator you are using here matching 5 elements on that page, so Selenium returns you the first element on the page matching the locator you passing here while you need the second match.
    This locator will work better //input[@id="search"]
  2. You need to add a delay.
    Selenium returns you search_area element at the moment that element is found on the page while it is still may be not ready to be interacted with.
    The best way to overcome this issue is to use Explicit Waits.
    So, this code should work better:
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys


options = Options()
options.add_argument('--disable-extensions')
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--remote-debugging-port=9515')
options.add_argument('--disable-setuid-sandbox')
options.add_argument("--start-maximized")

driver = webdriver.Chrome(service=Service("/usr/bin/chromedriver"), options=options)
wait = WebDriverWait(driver, 20)

url = "https://www.youtube.com/"
driver.get(url)

search_area = wait.until(EC.visibility_of_element_located((By.XPATH, "//input[@id='search']")))

search_area.send_keys('Lionel Messi', Keys.ENTER)

print(search_area.text)

P.S.
driver.implicitly_wait(10) is NOT a delay command, this will not put a delay of 10 seconds on the line where you put it. This only defines the timeout for Selenium to way for element presence. Also, you put if AFTER locating the element, so this could not affect the previously located element.

Sign up to request clarification or add additional context in comments.

10 Comments

where did you get this //ytd-searchbox[@id="search"] I am using inspect tools and copy the XPATH?
Try googling something like "selenium locators best practices"
I see.. Unfortunately, I have no Selenium with Python on my PC now, while to give you an answer there I would need to run the code I think about. Giving answers without trying that will no be good enough there...
Thanks. at least, I understand the problem. All the ID and XPATH are wrong. Using wait.until() trigger a timeout exception...I
Locators are correct (unique) there, but are bad. What I can advice is to wait for elements visibility rather than for elements presence.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.