Using Python to enter data into form and get data from results page

Question

I am using Python 2.7 on a 32 bit Windows machine.

I am trying to enter species data into http://explorer.natureserve.org and retrieve the results, but am having difficulty understanding how to do it. Needless to say I am relatively new to Python.

I have the following code:

import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()
driver.get("http://explorer.natureserve.org")
assert "NatureServe" in driver.title
SciName = driver.find_element_by_name('searchSciOrCommonName')
SciName.send_keys("Arabis georgiana")
SciName.send_keys(Keys.RETURN) 
assert "No results found." not in driver.page_source

The above works, but now I need to select the element Arabis georgiana on the results page, which will take me to another page. How do I get the results page back into Python and redirect to the page that I actually want?

alecxe · Accepted Answer · 2015-02-05 21:53:42Z

1

You need to set the searchSciOrCommonName field value this way:

br.form = list(br.forms())[0]
br.form['searchSciOrCommonName'] = 'butterfly'
response = br.submit()

Then, you can parse the HTML response via, for example, BeautifulSoup:

from bs4 import BeautifulSoup

soup = BeautifulSoup(response)

for item in soup.select('table[border="1"] > tr i')[1:]:
    print(item.text.strip())

which would print:

Aglais io
Callophrys mossii hidakupa
Callophrys mossii marinensis
Cercyonis pegala incana
...
Psora nipponica
Flowering Plants
Asclepias tuberosa

edited Feb 5, 2015 at 21:53

answered Feb 5, 2015 at 21:46

alecxe

476k127 gold badges1.1k silver badges1.2k bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Using Python to enter data into form and get data from results page

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related