Scraping Webpage using Python

Question

Im using selenium to access my school grades and after that i want to be able to scrape my grades from the site but i dont know how

Here is my login code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.PhantomJS("C:\Python27\phantomjs-1.9.0-windows\phantomjs.exe") 
driver.get("https://ps.rsd.edu/public/")

elem = driver.find_element_by_name("account")
elem.send_keys("Username")
elem2 = driver.find_element_by_name("pw")
elem2.send_keys("Password")
elem.send_keys(Keys.RETURN)

driver.quit()

print "done"

I think the easiest way is to use Beautifulsoup but im not sure

well i guess i could do that but id like to just get the grades off the site and can i do screenshots if the window isnt there because im running the driver with phantomjs — Serial
– Serial, Commented May 5, 2013 at 23:47
so i can use to selenium to get the info and return it because this is just a part of a larger program that takes my grades and makes it my desktop background — Serial
– Serial, Commented May 5, 2013 at 23:54

Yi Zeng · Accepted Answer · 2013-05-06 02:04:17Z

2

I'll answer this here as the other question is about how to parse the table using Beautifulsoup.

So given the table at http://gist.github.com/C-Dubb/5522909

for cell in driver.find_elements_by_css_selector(".grid tr a[href$='fg=S2']"):
    print cell.text
    # if you want the number only, you need to strip the grades here
    # also need to check if S2 cell is empty or not

answered May 6, 2013 at 2:04

Yi Zeng

33k13 gold badges99 silver badges128 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Scraping Webpage using Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related