Getting the "text" from a script element using Selenium

Question

I'm working on a web scraper, and the site I'm scraping has a script element on the page that looks like this:

<script type="text/javascript">
                        jQuery(window).load(function($) {
                        Morris.Line({
                          element: 'mpr-graph',
                          data: [
                            {'date': '25-04-2017','y':'1.05'},
                            {'date': '25-04-2017','y':'1.50'},
                            ...

What I want:

I want to get to the data property of the object passed to Morris.Line so that I can turn the data into something usable.

I've managed to select the correct element as a Selenium WebElement using the surrounding div's id and the tag name script, but now I'm stuck.

Is there a way to get the text of a script element using Selenium? The text property is empty since it only returns the text shown on the page for a given element.

What I've tried:

Since I was able to get the text in the browser console by grabbing the text property of the element, I tried using execute_script.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script');")

This returns a WebElement, so we're back to square 1, but at least we know it's working so we can move on to:

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').text;")

I thought this might work since it works in the browser console, but Selenium returns nothing.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').innerHTML;")

As above.

JeffC · Accepted Answer · 2017-11-16 03:20:08Z

5

You should be able to use an XPath to find the SCRIPT tag based on its contents

script_text = driver.find_element_by_xpath("//script[contains(.,'mpr-graph')]").text

If for some reason that isn't specific enough (more than one SCRIPT tag contains "mpr-graph") then you can adjust it to whatever text is unique among SCRIPT tags.

answered Nov 16, 2017 at 3:20

JeffC

26.4k5 gold badges35 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Riko Hamblin Over a year ago

Great Solution @JeffC !

Collectives™ on Stack Overflow

Getting the "text" from a script element using Selenium

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related