I need to use the google cloud api to write my Dataflow jobs.
As I understand it, I can't use pip install google-cloud-dataflow since Apache Beam wont' work on Python 3, so I've been using googleapiclient.discovery. However, when I issue the build() command, it bombs out citing the error:
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)
Background notes:
I'm sitting behind a Corporate Proxy, with HTTP(S)_PROXY set at the environment level
I also have CA_BUNDLE and REQUESTS_CA_BUNDLE set to my custom certs
I've installed
certifi, but no loveI've attempted to run
/Applications/Python\ 3.6/Install\ Certificates.commandbut couldn't find the .command in my virtualenv. Also, would prefer not to go down this path as it will make my Prod deployment a nightmare
Here's my code:
from oauth2client.client import GoogleCredentials
from googleapiclient.discovery import build
credentials = GoogleCredentials.get_application_default()
dataflow = build('dataflow', 'v1b3', credentials=credentials)
Result:
Traceback (most recent call last): File "test_dataflow_creds.py", line 6, in dataflow = build('dataflow', 'v1b3', credentials=credentials) File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper return wrapped(*args, **kwargs) File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/googleapiclient/discovery.py", line 222, in build requested_url, discovery_http, cache_discovery, cache) File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/googleapiclient/discovery.py", line 269, in _retrieve_discovery_doc resp, content = http.request(actual_url) File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/httplib2/init.py", line 1924, in request cachekey, File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/httplib2/init.py", line 1595, in _request conn, request_uri, method, body, headers File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/httplib2/init.py", line 1501, in _conn_request conn.connect() File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/httplib2/init.py", line 1291, in connect self.sock = self._context.wrap_socket(sock, server_hostname=self.host) File "/Users/user/.pyenv/versions/3.7.0/lib/python3.7/ssl.py", line 412, in wrap_socket session=session File "/Users/user/.pyenv/versions/3.7.0/lib/python3.7/ssl.py", line 850, in _create self.do_handshake() File "/Users/user/.pyenv/versions/3.7.0/lib/python3.7/ssl.py", line 1108, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)
