Skip to content

feat(core): add CDK-level VPC support - #277

Draft
svidgen wants to merge 25 commits into
mainfrom
feat/vpc-support
Draft

feat(core): add CDK-level VPC support#277
svidgen wants to merge 25 commits into
mainfrom
feat/vpc-support

Conversation

@svidgen

@svidgen svidgen commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

CDK-Level VPC Support (Phase 1)

Customer DX

import * as ec2 from 'aws-cdk-lib/aws-ec2';

const vpc = new ec2.Vpc(app, 'AppVpc', { maxAzs: 2, natGateways: 1 });

export const blocksStack = await BlocksStack.create(app, stackName, {
  backendHandlerPath: join(__dirname, 'index.handler.ts'),
  backendCDKPath: join(__dirname, 'index.ts'),
  vpc: { vpc },
});

That's it. Blocks handles:

  • Lambda placement in private subnets
  • Security group creation
  • VPC endpoint provisioning (based on which BBs are in scope)
  • SG wiring (e.g., Lambda → Aurora on 5432)
// Bring-your-own VPC (shared/platform-managed):
const sharedVpc = ec2.Vpc.fromLookup(app, 'SharedVpc', { vpcId: 'vpc-abc123' });
await BlocksStack.create(app, stackName, {
  ...,
  vpc: { vpc: sharedVpc, provisionEndpoints: false },
});

How it works

  1. Each BB registers its VPC requirements in its CDK constructor:

    this.registerVpcGatewayEndpoint(ec2.GatewayVpcEndpointAwsService.DYNAMODB);
    this.registerVpcInterfaceEndpoint(ec2.InterfaceVpcEndpointAwsService.SECRETS_MANAGER);
  2. At finalization (after all BBs are constructed), the framework walks the scope tree, collects registered endpoints, deduplicates, and provisions them against the VPC. SSM + CloudWatch Logs are always included.

  3. bb-data (Aurora) detects the VPC context on its scope chain. If present, it places Aurora in the shared VPC's isolated subnets and wires SG rules. If absent, it falls back to creating its own standalone VPC (existing behavior preserved).

API

interface BlocksVpcOptions {
  vpc: ec2.IVpc;
  subnets?: ec2.SubnetSelection;    // default: PRIVATE_WITH_EGRESS
  provisionEndpoints?: boolean;      // default: true
}

What's included

  • vpc prop on BlocksStack and BlocksBackend
  • registerVpcGatewayEndpoint / registerVpcInterfaceEndpoint on CDK Scope
  • Endpoint registration in all BB CDK constructors
  • bb-data VPC context detection + shared VPC placement
  • test-infra/ persistent test VPC stack
  • test-apps/vpc-smoke/ E2E smoke suite
  • docs/design/VPC-DESIGN.md

What's NOT included (Phase 2, after configurable compute)

  • VpcNetwork Building Block (per-handler VPC opt-in)
  • Per-namespace subnet selection

See docs/design/VPC-DESIGN.md for the full design.

Phase 1 VPC integration for AWS Blocks:

- Add BlocksVpcOptions types to packages/core/src/cdk/vpc-types.ts
- Add registerVpcRequirements() method to CDK Scope class
- Add VPC initialization and finalization to BlocksStack/BlocksBackend
- Add vpc prop to BlocksStack.create() and BlocksBackend.create()
- Lambda handler is placed in private subnets with security group when VPC enabled
- Finalization step: walk scope tree → collect requirements → deduplicate → provision endpoints
- SSM and CloudWatch Logs endpoints always included when VPC is enabled

Per-BB VPC requirement declarations:
- bb-kv-store: dynamodb (gateway)
- bb-distributed-table: dynamodb (gateway)
- bb-file-bucket: s3 (gateway)
- bb-data: secretsmanager + rds-data (interface), subnet role: isolated
- bb-distributed-data: none (DSQL uses public HTTPS)
- bb-async-job: sqs (interface)
- bb-agent: bedrock-runtime (interface)
- bb-knowledge-base: bedrock-runtime (interface)
- bb-email-client: ses (interface)
- bb-app-setting: ssm (interface)
- bb-realtime: execute-api (interface)
- bb-auth-cognito: ssm (interface)
- bb-auth-oidc: ssm (interface)

bb-data refactor:
- materialize() checks for VPC context before creating its own VPC
- When shared VPC is available, Aurora is placed in isolated subnets
- Security group rule: Lambda SG → Aurora on port 5432
- Falls back to creating isolated VPC when no shared VPC (backward compatible)

Test infrastructure:
- test-apps/vpc-smoke/ with inline VPC creation
- Instantiates KVStore, DistributedTable, FileBucket, AsyncJob, AppSetting
- Exercises auto endpoint detection

Design doc added at docs/design/VPC-DESIGN.md
@changeset-bot

changeset-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 4213fe6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 22 packages
Name Type
@aws-blocks/core Minor
@aws-blocks/bb-kv-store Patch
@aws-blocks/bb-distributed-table Patch
@aws-blocks/bb-file-bucket Patch
@aws-blocks/bb-data Patch
@aws-blocks/bb-distributed-data Patch
@aws-blocks/bb-async-job Patch
@aws-blocks/bb-agent Patch
@aws-blocks/bb-knowledge-base Patch
@aws-blocks/bb-email-client Patch
@aws-blocks/bb-app-setting Patch
@aws-blocks/bb-realtime Patch
@aws-blocks/bb-auth-cognito Patch
@aws-blocks/bb-auth-oidc Patch
@aws-blocks/auth-common Patch
@aws-blocks/bb-auth-basic Patch
@aws-blocks/bb-dashboard Patch
@aws-blocks/bb-cron-job Patch
@aws-blocks/bb-logger Patch
@aws-blocks/bb-tracer Patch
@aws-blocks/bb-metrics Patch
@aws-blocks/blocks Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

…istent VPC

1. Remove string→endpoint mapping: BBs now call registerVpcEndpoint()
   with actual CDK service objects (GatewayVpcEndpointAwsService or
   InterfaceVpcEndpointAwsService). The collection layer deduplicates
   by service identity and provisions directly. No intermediate string
   mapping table.

2. Test ALL BBs in vpc-smoke: KVStore, DistributedTable, FileBucket,
   AsyncJob, AppSetting, Realtime, AuthCognito, Logger, Metrics, and
   Tracer all have at least one round-trip assertion in the vpc-smoke
   test app.

3. Persistent test VPC (test-infra/): Creates a VPC with 2 AZs, 1 NAT,
   public/private/isolated subnets, and pre-provisions common endpoints
   (DynamoDB, S3, SSM, Secrets Manager, CloudWatch Logs). Deployed once
   and NOT torn down between test runs. The vpc-smoke app references it
   via Vpc.fromLookup() using the VPC_TEST_VPC_ID env var.
…rora in test VPC

- Add Database BB (Aurora via fromExisting) to vpc-smoke test app
- Add Aurora Serverless v2 cluster + RDS Data API endpoint to test-infra VPC stack
- Add Realtime client-side subscribe test (WebSocket within VPC)
- Add Database smoke tests: SELECT 1 and create/insert/read/drop

Verified: registerVpcEndpoint is correctly wired on Scope (protected method
delegates to free function). BlocksBackend already accepts and wires vpc prop
identically to BlocksStack. Design doc already matches latest draft.
- Split registerVpcEndpoint (instanceof-based) into two explicit methods:
  registerVpcGatewayEndpoint / registerVpcInterfaceEndpoint
- Simplify BlocksVpcOptions to { vpc, subnets?, provisionEndpoints? }
  - Rename lambdaSubnets → subnets
  - Replace endpoints: 'auto' | 'none' with provisionEndpoints boolean
  - Remove VpcEndpointRegistration type from public API
- Update all 12 BB packages to use new explicit registration methods
- Strip persistent test VPC (test-infra/vpc-test-stack.ts) to bare minimum:
  VPC + subnets (2 AZs) + 1 NAT + VPC ID output only
- Update vpc-smoke Database test: no connection option, self-provisions Aurora
- Update VPC-DESIGN.md to reflect simplified API
- Add changeset (@aws-blocks/core: minor, all touched BBs: patch)
svidgen added 17 commits July 28, 2026 21:48
…ehensive suite)

The test was importing BB instances directly which fails under the
browser condition (-C browser). Now the backend exposes an ApiNamespace
with methods that exercise each BB, and the test calls them via the
generated RPC client — same as the comprehensive e2e suite.
…n smoke test

Two fixes:
1. finalizeVpc now creates endpoints as standalone constructs in the app
   scope, not via vpc.addXxx() (which puts them in the VPC's own scope/stack)
2. vpc-smoke uses provisionEndpoints: false temporarily — the persistent
   test VPC has stale endpoints from earlier failed runs that conflict.
   Once account is cleaned up, re-enable to test auto-provisioning.
…port

Avoids the workspace name conflict and module resolution issues entirely.
The test reads the API URL from sandbox outputs and makes direct JSON-RPC
calls to the deployed Lambda — no generated client.js needed.
…p vpc-ref minimal

Root cause: the vpc-ref lookup stack persisted between runs because
destroySandbox only targeted the main stack. Now:
- sandbox-destroy uses 'cdk destroy --all' to clean up everything
- vpc-ref stack tagged with destroy removal policy
- after hook is still try/catch for safety but should now succeed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant