Running ChromeDriver with Python selenium on Heroku

Question

So I have a Flask server on Heroku which has been working fine as expected for some time.Now, as per new requirements, I need to add functionality to the Flask server to fetch a page from an external website.Because of reasons best known to me, I am using Selenium along with Chrome web driver to do this.Locally I was able to set this up and it works fine but I am quite unsure as to how to set it up on the Heroku server. I read a bit about buildpacks and found this buildpack for ChromeDriver :

https://elements.heroku.com/buildpacks/jimmynguyc/heroku-buildpack-chromedriver

However, I am not sure how to proceed further.How do I install chromium browser itself and what else is needed to tie it all up ?

Emanoeli M. · Accepted Answer · 2017-07-06 20:12:22Z

25

I had the same issue and the following steps worked fine for me:

I added the following buildpacks on heroku: https://github.com/heroku/heroku-buildpack-xvfb-google-chrome (to install chrome, since chromedriver requires it) and https://github.com/heroku/heroku-buildpack-chromedriver.
I created an environment variable GOOGLE_CHROME_BIN, with the path of chrome on heroku: /app/.apt/usr/bin/google-chrome and an environment variable called CHROMEDRIVER_PATH with the path of chromedriver on heroku: /app/.chromedriver/bin/chromedriver.

In my python file, I configured chromedriver:

chrome_options = Options()
chrome_options.binary_location = GOOGLE_CHROME_BIN
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--no-sandbox')
driver = webdriver.Chrome(executable_path=CHROMEDRIVER_PATH, chrome_options=chrome_options)

(First, I tried to configure chromedriver with no arguments, but I faced the following error: "Chrome failed to start: crashed". --disable-gpu and --no-sandbox solved the problem for me).

answered Jul 6, 2017 at 20:12

Emanoeli M.

3513 silver badges4 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Russell D Over a year ago

Thanks for taking the time to reply to this question, even though it's older. It solved my problem using additional buildpacks on heroku.

Ronnie Over a year ago

Hi @Russell D, can you share your solution?

Russell D Over a year ago

@Ronnie - I followed the solution provided by Emanoeli M...unfortunately, I tore down my Heroku deployed apps...I was paying for them but not using what they were doing so I pulled the plug and removed the instances I set up so I don't have access to what I did.

Ronnie Over a year ago

@Russell, Thanks for your reply. I've been able to make it work though performance is not stable.. Sometimes it is not able to find the element and gets timed out. stackoverflow.com/questions/56661348/…

Jacek Trociński · Accepted Answer · 2020-11-15 11:57:43Z

In your Heroku app go to Settings and add the following build packs:

In addition, in your Python script you have to set a few Chrome options so that your script runs on Heroku without error.

import time

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager

gChromeOptions = webdriver.ChromeOptions()
gChromeOptions.add_argument("window-size=1920x1480")
gChromeOptions.add_argument("disable-dev-shm-usage")
gDriver = webdriver.Chrome(
    chrome_options=gChromeOptions, executable_path=ChromeDriverManager().install()
)
gDriver.get("https://www.python.org/")
time.sleep(3)
gDriver.save_screenshot("my_screenshot.png")
gDriver.close()

Here is a more detailed post that I created in case you are still having problems: https://www.jtrocinski.com/posts/Heroku-Use_Selenium_to_run_Google_Chrome_in_Python.html

aahnik · Accepted Answer · 2021-01-20 16:18:58Z

0

For pyppeteer

If anybody is running on Heroku and facing the same error:

Add the buildpack : The url for the buildpack is below :

https://github.com/jontewks/puppeteer-heroku-buildpack

Ensure that you're using --no-sandbox mode

launch({ args: ['--no-sandbox'] })

answered Jan 20, 2021 at 16:18

aahnik

1,9212 gold badges15 silver badges33 bronze badges

Collectives™ on Stack Overflow

Running ChromeDriver with Python selenium on Heroku

3 Answers 3

4 Comments

Comments

For pyppeteer

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

Comments

For pyppeteer

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related