I have a self-hosted Airflow which has Google Oauth implemented. I need to extract data from an REST API within an Airflow DAG that requires authentication. To do this, I want to use a Python script to authenticate with the API via a Google service account, obtain an access token, and then use that token to make API calls and extract the data. If anyone has done something similar or can share examples, it would be incredibly helpful. Thank you!
I have implemented Google OAuth on Airflow web server as follows:
from flask_appbuilder.security.manager import AUTH_OAUTH
AUTH_TYPE = AUTH_OAUTH
# registration configs
AUTH_USER_REGISTRATION = True # allow users who are not already in the FAB DB
AUTH_USER_REGISTRATION_ROLE = "Admin" # this role will be given in addition to any AUTH_ROLES_MAPPING
OAUTH_PROVIDERS = [{
"name": "google",
"icon": "fa-google",
"token_key": "access_token",
"remote_app": {
"client_id": os.getenv("OAUTH_CLIENT_ID"),
"client_secret": os.getenv("OAUTH_CLIENT_SECRET"),
"api_base_url": "https://www.googleapis.com/oauth2/v2/",
"client_kwargs": {"scope": "email profile"},
"request_token_url": None,
"access_token_url": "https://accounts.google.com/o/oauth2/token",
"authorize_url": "https://accounts.google.com/o/oauth2/auth",
"jwks_uri": "https://www.googleapis.com/oauth2/v3/certs",
}
}]