Getting the HTML element using Selenium WebDriver

Question

I'm trying to get price of a product on amazon using Selenium:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By

url = \
"https://www.amazon.in/Celevida-Kesar-Elaichi-Flavor-Metal/dp/B081WJ6536/ref=sr_1_5?crid=3NRZERQ8H4T8L&keywords=dr+reddys+celevida&qid=1672124472&sprefix=%2Caps%2C5801&sr=8-5"

services = Service(r"C:\Users\Deepak Shetter\chromedriver_win32\chromedriver.exe")
driver = webdriver.Chrome(service=services)
driver.get(url)
price = driver.find_element(By.CLASS_NAME, "a-offscreen")
print("price is "+price.text)

As you can see in this image the html for the price is of class="a-offscreen". But when I run my code on pycharm it return None. How can I get the price string? (btw I checked it using Beautiful soup and it worked fine)

Edit : This time I used another url : https://www.amazon.in/Avvatar-Alpha-Choco-Latte-Shaker/dp/B08S3TNGYK/?_encoding=UTF8&pd_rd_w=ofFKu&content-id=amzn1.sym.1f592895-6b7a-4b03-9d72-1a40ea8fbeca&pf_rd_p=1f592895-6b7a-4b03-9d72-1a40ea8fbeca&pf_rd_r=PT3Y6GWJ7YHADW09VKNK&pd_rd_wg=lBWZa&pd_rd_r=0a44c278-bcfa-49c2-806b-cf8eb292038a&ref_=pd_gw_ci_mcx_mr_hp_atf_m

In this case it has 2 price elements one with the class="a-offscreen" and another one with calss="a-price-whole".

my code :

price = driver.find_element(By.CLASS_NAME, "a-price-whole")

this time return value is 1,580.

I'm surprised your code runs at all as the path to your chromedriver is invalid — jackal
– jackal, Commented Dec 27, 2022 at 7:38
@Fred That's not inappropriate. When I run the code without `` it showed an error. That's why I added another \. — Deepak
– Deepak, Commented Dec 27, 2022 at 8:43
I was referring to \D and 2 occurrences of \c. Take a look at how pathnames should be constructed for Windows. Hint: raw strings — jackal
– jackal, Commented Dec 27, 2022 at 8:57

Prophet · Accepted Answer · 2022-12-27 11:41:12Z

The following code works:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 10)

url = "https://www.amazon.in/Celevida-Kesar-Elaichi-Flavor-Metal/dp/B081WJ6536/ref=sr_1_5?crid=3NRZERQ8H4T8L&keywords=dr+reddys+celevida&qid=1672124472&sprefix=%2Caps%2C5801&sr=8-5"
driver.get(url)
price = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#corePrice_desktop .a-span12 .apexPriceToPay"))).text
print(price)

The output is:

₹566

jackal · Accepted Answer · 2022-12-27 08:13:23Z

0

Using Safari, the following code works:

from selenium import webdriver
from selenium.webdriver.common.by import By

with webdriver.Safari() as driver:
    driver.get('https://www.amazon.in/Celevida-Kesar-Elaichi-Flavor-Metal/dp/B081WJ6536/ref=sr_1_5?crid=3NRZERQ8H4T8L&keywords=dr+reddys+celevida&qid=1672124472&sprefix=%2Caps%2C5801&sr=8-5')
    price = driver.find_element(By.CLASS_NAME, "a-offscreen")
    print(price.text)

Which gives this output:

₹566.00

Therefore it appears that your use of the ChromeDriver may be flawed

answered Dec 27, 2022 at 8:13

jackal

29.1k3 gold badges9 silver badges27 bronze badges

Collectives™ on Stack Overflow

Getting the HTML element using Selenium WebDriver

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related