Manipulating the html source-code of a third party website using python

Question

When I inspect the element of a website (google alerts: https://www.google.com/alerts) I type in a name and press 'more options'. When I inspect the element of this page for one of the buttons such as 'How Often', I notice that in the HTML source code:

<div class="goog-inline-block goog-flat-menu-button-caption" id=":3" role="option" aria-setsize="3" aria-posinset="2">At most once a day</div>

If I manually edit the 'At most once a day' to the exact text of another option (through my inspector)(for example I type: 'At most once a week', it actually creates the change on the website. Therefore, I was curious if there is a way in python (maybe through selenium or beautiful soup) to actually edit the html code of a third-party website to create these types of changes? If not, I am struggling to find a way to change this drop-down button selection using selenium and would appreciate any insights.

Thanks!

JeffC · Accepted Answer · 2015-09-22 23:52:19Z

2

You can edit the HTML of the page but that's not what you want to do. You would essentially change the label of the selection but not the behavior. Think of it like changing the sign to the Men's bathroom to say "Women's." It doesn't change that bathroom to women's, it would just be really confusing for the women walking in there... :)

What you want to do is select a particular option on the page. To get the scenario to work, you will need to do what a user does in script. To change that dropdown (it's not really a SELECT dropdown, it just looks and behaves like one) you will need to click on it then click on the element that you want. Those elements for the two clicks have IDs so you should be able to click them easily.

answered Sep 22, 2015 at 23:52

JeffC

26.4k5 gold badges35 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

alecxe · Accepted Answer · 2015-09-22 17:04:44Z

1

There is Google alerts API Python client - galerts. Not sure if it still works since it's 4 years old.

Alternatively, you can use the browser automation approach and make the changes through the browser controlling it using selenium. For this particular dropdown, locate it and click the element having At most once a week text:

dropdown = driver.find_element_by_css_selector("div.frequency_select")
dropdown.click()

once_a_week = driver.find_element_by_xpath("//*[. = 'At most once a week']")
once_a_week.click()

answered Sep 22, 2015 at 17:04

alecxe

476k127 gold badges1.1k silver badges1.2k bronze badges

Collectives™ on Stack Overflow

Manipulating the html source-code of a third party website using python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related