I want to use Selenium to find specific text

Question

I was going to use Selenium to crawl the web

from selenium import webdriver
from selenium.webdriver.common.by import By
import time

options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome('./chromedriver', options=options)
driver.get('https://steamdb.info/tag/1742/?all')
driver.implicitly_wait(3)

li = []
games = driver.find_elements_by_xpath('//*[@class="table-products.text-center.dataTable"]')

for i in games:
    time.sleep(5)
    li.append(i.get_attribute("href"))

print(li)

After accessing the steam url that I was looking for, I tried to find something called an appid

The picture below is the HTML I'm looking for

I'm trying to find the number next to "data-appid="

But if I run my code, nothing is saved in the "games"

Rihhard · Accepted Answer · 2022-11-02 14:26:38Z

1

Correct me if I'm wrong but from what I can see this steam page requires you to log-in, are you sure that when webdriver opens the page that same data is available to you ?

Additionally when using By, the correct syntax would be games = driver.find_element(By.CSS_SELECTOR('//*[@class="table-products.text-center.dataTable"]'))

answered Nov 2, 2022 at 14:26

Rihhard

664 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

I want to use Selenium to find specific text

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related