0

Through Beautifulsoup module I have extracted an HTML page. From that page, I further extracted a Javascript script tag. Inside the script tag, there is an object literal that I would like to work with. You can see what I would like to achieve:

<script>
            var main_obj = {

            "link":"",
            "stock":"",
            "price":[{"qty":1000,"value":"100$"}, {"qty":10000,"value":"1000$"}]

           } 

</script>

I would like to access the qty and value variables inside the object literals of price variable which is inside main_obj. Thankyou

3

1 Answer 1

1

One option would be to use selenium. In particular, you can use execute_script to convert to a JSON string which Python can easily parse.

Since I don't know what the URL of the page you are working with is, I just created a local HTML file which included your script tag example. Using headless is not mandatory and I only added that option so the browser window doesn't open.

test.html

<!DOCTYPE html>
<html>
<body>
<script>
    var main_obj = {
        "link": "",
        "stock": "",
        "price": [{"qty": 1000, "value": "100$"}, {"qty": 10000, "value": "1000$"}]
    }
</script>
</body>
</html>

Script

In[2]: import os
  ...: import json
  ...: 
  ...: from selenium import webdriver
  ...: 
  ...: chrome_options = webdriver.ChromeOptions()
  ...: chrome_options.add_argument('--headless')
  ...: driver = webdriver.Chrome(chrome_options=chrome_options)
  ...: 
  ...: driver.get('file://{}/test.html'.format(os.getcwd()))
  ...: json_string = driver.execute_script('return JSON.stringify(main_obj)')
  ...: driver.quit()
  ...: 
  ...: json_data = json.loads(json_string)
In[3]: json_data
Out[3]: 
{'link': '',
 'stock': '',
 'price': [{'qty': 1000, 'value': '100$'}, {'qty': 10000, 'value': '1000$'}]}
In[4]: for item in json_data['price']:
  ...:     print('Quantity: {:d}\tValue: ${:.2f}'.format(
  ...:         item['qty'], float(item['value'].rstrip('$'))
  ...:     ))
  ...: 
Quantity: 1000  Value: $100.00
Quantity: 10000 Value: $1000.00
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.