feat: add Google Workload Identity Federation - #72
Open
NGQ-Hiro wants to merge 9 commits into
Open
Conversation
- Updated `pyproject.toml` to include `google-auth` dependency. - Introduced `WorkloadIdentityAuthenticator` for authenticating using Google Workload Identity Federation. - Modified `GoogleSheetsBaseStream` to utilize `WorkloadIdentityAuthenticator` when credentials are provided. - Added new configuration options for workload identity credentials in the tap's schema.
The default google-auth AWS credential source can fail two ways when IMDS is unavailable: a RefreshError (proxy returns an error page) or a TransportError (connection times out). Only RefreshError was caught, so timeout environments never reached the botocore fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…orkload Identity Federation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Google Workload Identity Federation (WIF) with AWS
tap-google-sheetssupports authenticating to the Google Sheets and Drive APIsusing Google Workload Identity Federation (WIF). WIF is a Google Cloud feature
that allows Google APIs to trust external identities — in this case AWS — without
needing a long-lived Google service account key.
This is intended for deployments running on AWS — most notably EKS with IRSA
(IAM Roles for Service Accounts) — and local development using AWS SSO.
How it works
WIF does not replace AWS credentials — it reuses them to prove your identity to
Google. The flow is:
Key points:
SSO, static env vars, and IMDS.
a specific SA email (federated principals have no email, so they cannot be added as
sheet collaborators directly).
Prerequisites
(created in Google Cloud Console → IAM → Workload Identity Federation).
with the
roles/iam.workloadIdentityUserbinding on the WIF pool.AWS_WEB_IDENTITY_TOKEN_FILE+AWS_ROLE_ARN).aws sso login --profile <profile>).*.googleapis.com(bothsts.googleapis.comand
iamcredentials.googleapis.commust be reachable).Configuration
workload_identityfalseoauth_credentialswhentrue.workload_identity_credentialsworkload_identity_credentials_fileworkload_identity_credentialsis not set.When
workload_identityistrue, exactly one ofworkload_identity_credentialsor
workload_identity_credentials_filemust be provided; otherwise the tapraises a
ValueError.Example config
{ "workload_identity": true, "workload_identity_credentials": "{\"type\":\"external_account\",\"audience\":\"//iam.googleapis.com/projects/123456789/locations/global/workloadIdentityPools/my-pool/providers/aws-provider\",\"subject_token_type\":\"urn:ietf:params:aws:token-type:aws4_request\",\"token_url\":\"https://sts.googleapis.com/v1/token\",\"service_account_impersonation_url\":\"https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/my-sa@my-project.iam.gserviceaccount.com:generateAccessToken\",\"credential_source\":{\"environment_id\":\"aws1\",\"regional_cred_verification_url\":\"https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15\"}}", "sheets": [{"sheet_id": "<spreadsheet_id>"}] }Or point the tap at a credential file:
{ "workload_identity": true, "workload_identity_credentials_file": "/var/secrets/google/wif-credentials.json", "sheets": [{"sheet_id": "<spreadsheet_id>"}] }Authentication priority
The tap selects an authentication method in the following order:
workload_identityistrue.client_id,client_secret, andrefresh_tokenare present.Limitations & notes
OIDC identity pool) raise
NotImplementedError.*.googleapis.com. AConnection reset by peererror on
sts.googleapis.comoriamcredentials.googleapis.comindicates anetwork/firewall issue — not a code bug.