Selenium Webdriver/Browser with Python

Question

I need to build a Python scraper to scrape data from a website where content is only displayed after a user clicks a link bound with a Javascript onclick function, and the page is not reloaded. I've looked into Selenium in order to do this and played around with it a bit, and it seems Selenium opens a new Firefox web browser everytime I instantiate a driver:

 >>> driver = webdriver.Firefox()

Is this open browser required, or is there a way to get rid of it? I'm asking because the scraper is potentially part of a web app, and I'm afraid if multiple users start using it, I will have a bunch of browser windows open on my server.

related: Selenium - Can I hide the browser?. See also Ghost.py — jfs
– jfs, Commented Mar 2, 2013 at 22:18

rcdsystems · Accepted Answer · 2013-04-05 21:59:46Z

0

Yes, selenium automates web browsers. You can add this at the bottom of your python code to make sure the browser is closed at the end:

driver.quit()

answered Apr 5, 2013 at 21:59

rcdsystems

1,5481 gold badge10 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Selenium Webdriver/Browser with Python

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related