Python web scraping script does not find element by css selector

Question

I'm trying to get this web scraper to get current electricity price from this website, it's in finnish but it's right under "Hinta nyt". https://sahko.tk/

Here's my code:

import requests
from bs4 import BeautifulSoup

url = "https://sahko.tk/"

element_selector = ""
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")
elements = soup.find_all(element_selector)

if len(elements) == 0:
    print("No element found with selector '%s'" % element_selector)
else:
    element_text = elements[0].text
    print(element_text)

I left the element_selector to empty because what ever I tried it just did not work. I'm not even sure if I'm on the right tracks.

Andrej Kesely · Accepted Answer · 2022-12-06 19:02:34Z

1

The data you see is embedded inside <script> in that page. To parse the current price you can use next example:

import re
import json
import requests

url = "https://sahko.tk/"

data = requests.get(url).text

data = re.search(r"function prices_today\(\)\{var t= (.*?});", data).group(1)
data = json.loads(data)

print("Hinta nyt", data["now"], "snt/kWh")

Prints:

Hinta nyt 33.27 snt/kWh

answered Dec 6, 2022 at 19:02

Andrej Kesely

196k15 gold badges60 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python web scraping script does not find element by css selector

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related