Getting a blank navigation dropdown when trying to access website using Selenium (using Python)

Question

I am trying to automate a process where I can visit a website, hover over the menu navigation bar and click on options from the dropdown to visit that page.

I am working with Selenium (using python) for this and using "https://www.shoppersstop.com/" to test my code.

When I run the below code, the website opens correctly but I am getting a blank dropdown which is why when I try to click on any option from the dropdown I am getting a no elements found error.Here is the screenshot of what I am seeing.

I have tried using the same code to view dropdown in Amazon.In website and it worked correctly. I have also tried adding time.sleep() in between the actions but still this doesn't work.

Error message below: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//a[text()='All Indian Wear']"} (Session info: chrome=123.0.6312.107); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception

Below is my code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
from bs4 import BeautifulSoup
import pandas as pd
import requests
import time
import re

# Set Chrome options
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--user-agent=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36")

# Initialize driver with Chrome options
driver = webdriver.Chrome(options=chrome_options)

driver.get("https://www.shoppersstop.com/")     # Open the website
driver.maximize_window()        #Maximize window

try:
    WebDriverWait(driver, 20).until(lambda driver: driver.execute_script('return document.readyState') == 'complete')
    #time.sleep(10)
    actions = ActionChains(driver)
    women_category = driver.find_element(By.XPATH, "//html/body/main/header/nav/div[1]/div/ul[2]/li[4]/a")
    actions.move_to_element(women_category)          # Hover over the Women's category to reveal the dropdown
    actions.perform()
    #time.sleep(30)
    indian_wear = driver.find_element(By.XPATH, "//a[text()='All Indian Wear']")
    actions.move_to_element(indian_wear).perform().click()
    #time.sleep(20)
except Exception as e:
    print(e)

driver.quit()       # Close the browser

Already mentioned above.

I get the same thing. I don't know what it is. I'm guessing it may be some sort of protection against bots? — JeffC
– JeffC, Commented Apr 15, 2024 at 17:09
I agree. Below solution using SeleniumBase works for me though ! — Annie
– Annie, Commented Apr 16, 2024 at 17:23

Michael Mintz · Accepted Answer · 2024-04-15 18:03:39Z

0

Regular Selenium gets blocked on that site for certain sections, but https://github.com/seleniumbase/SeleniumBase gets through successfully:

pip install seleniumbase, and then run the following with python:

from seleniumbase import SB

with SB(uc=True, test=True) as sb:
    sb.driver.get("https://www.shoppersstop.com/")
    sb.hover_and_click('a[title="WOMEN"]', "//a[text()='All Indian Wear']")
    
    breakpoint()

Type c and press Enter in the console to continue from a Python breakpoint().

Note that SeleniumBase auto-detects selectors, so you can use CSS or XPath for the built-in SeleniumBase methods.

answered Apr 15, 2024 at 18:03

Michael Mintz

16.1k9 gold badges51 silver badges109 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Annie Over a year ago

Thank you @Michael for introducing me to SeleniumBase. Above code was not working initially but introducing a sb.sleep(3) before the hover_and_click statement fixed it. Probably because of page load time constraints it wasn't working at first. Thanks a lot once again !

Collectives™ on Stack Overflow

Getting a blank navigation dropdown when trying to access website using Selenium (using Python)

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related