feat(gcp): add GCP resource providers, HTTP bindings, and nuke - #1336
Draft
Mkassabov wants to merge 25 commits into
Draft
feat(gcp): add GCP resource providers, HTTP bindings, and nuke#1336Mkassabov wants to merge 25 commits into
Mkassabov wants to merge 25 commits into
Conversation
Introduce Alchemy GCP providers for Compute, GKE, Storage, Pub/Sub, Cloud Run, SQL, and related services, with HTTP bindings, a gcp-gke Kubernetes adapter, and `pnpm nuke:gcp`.
Contributor
|
Install the packages built from this commit: Alchemyalchemy bun add https://pkg.ing/alchemy/c5229a6@alchemy.run/better-auth bun add https://pkg.ing/@alchemy.run/better-auth/c5229a6@alchemy.run/cloudflare-runtime bun add https://pkg.ing/@alchemy.run/cloudflare-runtime/c5229a6@alchemy.run/frontend-frameworks bun add https://pkg.ing/@alchemy.run/frontend-frameworks/c5229a6@alchemy.run/node-utils bun add https://pkg.ing/@alchemy.run/node-utils/c5229a6@alchemy.run/pr-package bun add https://pkg.ing/@alchemy.run/pr-package/c5229a6@alchemy.run/floci bun add https://pkg.ing/@alchemy.run/floci/c5229a6Distilled@distilled.cloud/core bun add https://pkg.ing/@distilled.cloud/core/cecf35f@distilled.cloud/aws bun add https://pkg.ing/@distilled.cloud/aws/cecf35f@distilled.cloud/axiom bun add https://pkg.ing/@distilled.cloud/axiom/cecf35f@distilled.cloud/cloudflare bun add https://pkg.ing/@distilled.cloud/cloudflare/cecf35f@distilled.cloud/hetzner bun add https://pkg.ing/@distilled.cloud/hetzner/cecf35f@distilled.cloud/neon bun add https://pkg.ing/@distilled.cloud/neon/cecf35f@distilled.cloud/planetscale bun add https://pkg.ing/@distilled.cloud/planetscale/cecf35f |
Contributor
|
Please update PR description with DX (heading + code snippet showing resources, effectful sevices, bindings) etc. |
Point the submodule at alchemy-run/distilled#475 so Alchemy can consume the GCP credentials DX without waiting for merge.
Contributor
|
Can you show me a GCP.Function with bindings (something for twitter) |
sam-goodwin
reviewed
Aug 25, 2026
Factory coverage for distilled GCP CRUD: observe-ensure-sync resources, HTTP Binding.Service layers, live tests, and pnpm nuke:gcp.
Vertex missing-resource probes return BadRequest (and list on the global endpoint); gate Vertex lifecycles behind GCP_TEST_AIPLATFORM. Network wait now uses the operation id, surfaces quota errors, and retries NETWORKS quota. Regional backend buckets default to EXTERNAL_MANAGED and omit CDN/header fields GCP rejects. Instant snapshots retry the 30s-too-recent insert.
Labels cannot be set on insert; a crash before setLabels left an LbEdgeExtension forwarding rule invisible to list. Stamp ownership in the description, treat rules that target an Alchemy HTTP(S) proxy as owned, and delete forwarding rules before proxies/url maps. Listing delete always sends deleteCommercial so Analytics Hub commercial listings can be torn down when the API allows it.
Delete was using persisted ACTIVE and retrying BadRequest while the bucket was already DELETE_REQUESTED or still UPDATING after a link change. Observe live lifecycle, wait out pending, and treat DELETE_REQUESTED as gone. Billing saved-query probes now catch BadRequest when the billing account type does not support them.
…s gone Regional Secret Manager needs a regional endpoint the global SDK cannot reach, so lifecycle tests stay behind GCP_TEST_REGIONAL_SECRETS with a probe that asserts BadRequest. Backup-schedule GET after the parent database is deleted returns BadRequest instead of NotFound; treat that as gone and retry create while the database is still coming up.
Treat missing DLP/CCI parents as gone, prefix DLP job ids with i-/r-, drop invented Service Management error tags, and interpolate Dataplex bucket names as Outputs. Gate API Hub, Integrations, Tag Manager, and Datastream private-connection lifecycles behind entitlement env vars; Pub/Sub Lite probes accept sunset NotFound.
Treat Translate glossary-entry list BadRequest as empty, skip malformed Wasm plugin version names on delete, and keep ValidateAttestation from re-yielding Output. Skip Dataplex lake, NCC hub/spoke, intercept/mirror, Workstations cluster, and wasm version lifecycles unless the matching entitlement env is set so per-service timeout 240 stays skip-clean.
Data taxonomy sunset probes also accept TooManyRequests. DataScan and remaining Dataplex create tests stay behind GCP_TEST_DATAPLEX. Client TLS policy delete can exceed 90s; wait treats BadRequest as gone.
catchTag('BadRequest') on get/list ops whose union is only NotFound |
Forbidden | GcpOpError failed tsc. Keep those as NotFound/Forbidden and
leave BadRequest on create where the SDK actually types it.
sam-goodwin
reviewed
Aug 26, 2026
sam-goodwin
reviewed
Aug 26, 2026
sam-goodwin
reviewed
Aug 26, 2026
sam-goodwin
reviewed
Aug 26, 2026
Stop GET-after-destroy on Firestore backup schedules (parent DB gone returns untyped BadRequest). Wait out Logging locations-link LROs, gate billing saved queries behind GCP_TEST_BILLING_SAVED_QUERY, skip incomplete/global glossary parents on Translate read, and give TlsRoute delete 180s.
Treat a missing bucket as gone during force-destroy, wait out backend bucket insert LROs, retry instance delete while STOPPING, and give network replace-delete 180s. Gate slow gen2 functions, instance stop bindings, and zone-capacity reservations behind env flags with probes kept ungated.
Catch OperationPending on BackendService global ops (TargetSslProxy create) and give Subnetwork/TargetSslProxy 180s so VPC teardown can finish.
…obes Vertex GET/list on us-central1 often returns BadRequest (global-only endpoint or AgentService v1). Assert that tag and skip the follow-up list so the probes stay typed without catching untyped list errors.
Phrase matcher ids are server-assigned; GET of a generated name returns BadRequest. Observe via output name or displayName. Gate Service Management delete LROs behind GCP_TEST_SERVICEMANAGEMENT.
Create+destroy already completed inside 120s; waitUntilGone after destroy blew the timeout. Isolated retest passed in 117.8s.
Add Action-driven live calls for the Binding.Service ops that had no test presence, including skipIf/entitlement probes that record the typed GCP error when the parent resource cannot be created.
Network insert now retries any NetworkOperationFailed (quota/race), not only quota-shaped messages. Instant snapshots wait longer for the 30s source-disk cooldown. FAST skips HA/Classic VPN, firewall, instant snapshot, and Artifact Registry attachment lifecycles.
Wait longer for Compute image, instance, router, and snapshot operations, and retry Analytics Hub deletes on replica 500s. Bump slow lifecycle test timeouts so per-service recapture stays under the 240s wall.
Every Providers.ts-registered GCP Resource now has a dedicated
{Service}/{Resource}.test.ts: probe GET on a missing row, plus
create/update/delete skipIf-gated on creds, FAST, or entitlement.
RegionTargetTcpProxy has no setBackendService API. Reconcile now deletes and recreates when the observed backend, proxy header, or other immutable fields drift so an engine update still converges.
Network firewall insert cannot send rules; syncRules now leaves GCP's reserved goto_next priorities (2147483548–2147483647) alone. Network attachments derive name/selfLink when GET omits them and replace when connectionPreference changes. Health sources expand to regional backend-service URLs.
Cloud Run Service/Job (and Cloud Functions) are Platforms with a
{ env, iam } binding contract. HTTP bindings yield the distilled op
once (no double provide) and grant a role on the runtime service
account, matching AWS policyStatements / Cloudflare AccountApiToken.
Memorystore Read/Write/ReadWrite uses alchemy/Redis over REDIS_URL.
GCP.Function is an alias of Run.Service for the Effect-native DX.
Contributor
Author
|
Updated the PR description with an Effect-native `GCP.Function` (Cloud Run Service) that binds Pub/Sub + Memorystore Redis — the twitter-bot shape. `GCP.Function` is an alias of `GCP.Run.Service`. |
Sam's spot-check was the standard for the whole GCP surface, not three files. HTTP bindings no longer double-provide Credentials/HttpClient, and every Binding.Service host.binds a role onto Cloud Run / Function (AWS policyStatements / Cloudflare AccountApiToken). WorkerPool is the same effectful host as Service/Job.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GCP resource factory coverage: 1051
Resourceexports (each with a dedicated{Resource}.test.ts), 192 HTTPBinding.Services, live tests underpackages/alchemy/test/GCP, andpnpm nuke:gcp.Effect-native Function with bindings
Cloud Run is the GCP analog of Lambda/Worker. Bindings grant IAM onto the runtime service account (AWS
policyStatements/ CloudflareAccountApiToken) and inject env. Memorystore speaks RESP through the sharedalchemy/Redisclient.GCP.FunctionisGCP.Run.Service. Jobs are the same shape withGCP.Run.Job(runs to completion). Zip-source Cloud Functions stay onGCP.CloudFunctions.Functionand still receive binding env + IAM on the runtime SA.Observe-ensure-sync reconcilers, idempotent delete,
list()of alchemy-labeled rows. HTTP bindings close overCredentials+HttpClientonGCP.providers()and grant least-privilege IAM on the host. Distilled pin:cecf35fe9597699b66e8e5ad3e3db6b135e2a44c.Catalog
Every distilled create+get+delete trio is implemented or out-of-scope (
{SCRATCH}/gcp-catalog.txt): 1127 implemented rows, 0 missing, 3 out-of-scope (Cloud Tasks tasks, Spanner sessions, Storage objects). Vertex/Apigee/consumer APIs are implemented with skipIf-gated live tests, not catalogued away.Every registered Resource has
packages/alchemy/test/GCP/{Service}/{Resource}.test.ts(1051/1051).Live recapture (188 dirs)
Per-service
timeout 240 pnpm test test/GCP/{dir} --profile testing --retry 0. The fulltest/GCPtree cannot finish in a 240s wall (~2124 tests).GCP_TEST_*): NetworkFirewallPolicy 2/2, RegionNetworkFirewallPolicy 2/2, NetworkAttachment 2/2, RegionCompositeHealthCheck 3/3Bindings census
192
Binding.Serviceexports, each with a live Action/HTTP invocation inBindings.test.ts. Memorystore also hasReadRedis/WriteRedis/ReadWriteRedisoveralchemy/Redis.Nuke residue (catalogued)
pnpm nuke:gcp --dry-runafter the recapture:replicaState: UNAVAILABLEDESTROY_SCHEDULED(24h; no immediate delete API)