feat(docs): refactor Building Block documentation — per-block docs/, committed catalog with CI sync gate

.changeset/docs-restructure-require-resolve.md

@@ -0,0 +1,16 @@
+---
+"@aws-blocks/blocks": patch
+"@aws-blocks/create-blocks-app": patch
+---
+
+docs: per-block docs folders + committed BB catalog with CI sync check; CLAUDE/agents docs resolved via require.resolve
+
+`@aws-blocks/blocks` now ships one docs folder per Building Block under `docs/<block>/`
+(`README.md` / `API.md` / `DESIGN.md`), plus a committed, marker-delimited Building Block
+catalog in the package README that a `sync-docs --check` CI gate keeps in sync. The README's
+catalog section and the scaffolded `AGENTS.md` (`@aws-blocks/create-blocks-app`) now direct
+tools and agents to locate docs programmatically via
+`require.resolve('@aws-blocks/blocks/docs/<block>/README.md')` (and
+`require.resolve('@aws-blocks/blocks/docs/README.md')` for the catalog) rather than assuming a
+`node_modules/` path or following the human-facing relative links. Also adds a Security
+Considerations section to the package README.

.github/workflows/block-catalog-check.yml

@@ -0,0 +1,25 @@
+name: Block Catalog Check
+
+# Fails the PR if the generated Building Block catalog table in
+# packages/blocks/README.md is out of date. The script only reads the committed
+# package READMEs + root README (Node builtins, zero deps), so no `npm ci` needed.
+# On failure its stderr tells the engineer to run `npm run sync-docs` and commit.
+
+on:
+  pull_request:
+    branches: [main]
+
+permissions:
+  contents: read
+
+jobs:
+  check:
+    name: Catalog in sync
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+      - uses: actions/setup-node@v5
+        with:
+          node-version-file: '.nvmrc'
+      - name: Verify Building Block catalog is in sync
+        run: node scripts/sync-catalog.mjs --check

.github/workflows/pr-checks.yml

@@ -72,7 +72,7 @@ jobs:
       - run: npm ci
 
       - name: Sync block docs (prebuild step)
-        run: node scripts/sync-block-docs.mjs
+        run: node scripts/gen-block-docs.mjs
 
       - name: Check blocks docs-sync integrity
         env:
@@ -86,14 +86,16 @@ jobs:
             exit 0
           fi
 
-          # sync-block-docs.mjs regenerates packages/blocks/docs/ from sibling READMEs
+          # gen-block-docs.mjs regenerates packages/blocks/docs/ from sibling READMEs
           # (docs/ is gitignored, built at publish time). We compare the packed
           # @aws-blocks/blocks tarball at THIS PR's head against the PR base branch —
           # like-for-like (same algo, same build state), so only a change introduced by
           # THIS PR trips the check. A PR that doesn't alter what ships needs no bump.
+          # The base pack falls back to the pre-rename sync-block-docs.mjs so this job works
+          # against bases from either side of the gen-block-docs.mjs rename.
           pack_blocks() {
             local dest="$1"
-            node scripts/sync-block-docs.mjs >/dev/null
+            node scripts/gen-block-docs.mjs >/dev/null
             # --ignore-scripts intentionally packs source + synced docs, NOT a freshly built dist/ — this diffs shipped content, not the published build.
             ( cd packages/blocks && npm pack --ignore-scripts --pack-destination "$dest" >/dev/null )
             set -- "$dest"/*.tgz
@@ -109,7 +111,7 @@ jobs:
           mkdir -p /tmp/base-pack
           BASE_INTEGRITY=$(
             cd /tmp/blocks-base
-            node scripts/sync-block-docs.mjs >/dev/null
+            node scripts/gen-block-docs.mjs >/dev/null 2>/dev/null || node scripts/sync-block-docs.mjs >/dev/null
             ( cd packages/blocks && npm pack --ignore-scripts --pack-destination /tmp/base-pack >/dev/null )
             set -- /tmp/base-pack/*.tgz
             [ $# -eq 1 ] || { echo "expected exactly one .tgz in /tmp/base-pack, found $#" >&2; exit 1; }

package.json

@@ -83,6 +83,8 @@
     "create-demo": "node packages/create-blocks-app/dist/index.js --template demo",
     "update-template": "tsx scripts/update-template-from-demo.ts",
     "build:tech-design-pdf": "node scripts/build-tech-design-pdf.mjs",
+    "sync-docs": "node scripts/sync-catalog.mjs --write",
+    "sync-docs:check": "node scripts/sync-catalog.mjs --check",
     "review-pr": "bash scripts/review-pr.sh",
     "publish:local": "tsx scripts/publish/publish.ts",
     "version": "changeset version && npm install --package-lock-only",

packages/blocks/README.md

@@ -109,31 +109,78 @@ onAuthChange(authApi, (user) => {
 
 ## Building Blocks
 
-Each block is its own package; full per-block docs ship in this package under **`docs/<package-name>.md`**, and **`docs/index.md`** has a decision tree to help you pick.
+Start from what you need:
+
+- **Store data**
+  - Simple key → value (caches, flags, user prefs) → `KVStore` (bb-kv-store)
+  - Structured records with indexes and queries → `DistributedTable` (bb-distributed-table) — **default for most data**
+  - Relational / SQL (joins, transactions) → see [Choosing a data block](#choosing-a-data-block) below
+  - Files, blobs, uploads, static assets → `FileBucket` (bb-file-bucket)
+  - A single config value or secret → `AppSetting` (bb-app-setting)
+- **Authenticate users**
+  - Username/password, prototypes/MVPs → `AuthBasic` (bb-auth-basic)
+  - Cognito user pools, MFA, groups → `AuthCognito` (bb-auth-cognito)
+  - External identity provider (OIDC) → `AuthOIDC` (bb-auth-oidc)
+- **Run work outside the request/response**
+  - Fire-and-forget background jobs → `AsyncJob` (bb-async-job)
+  - Scheduled / recurring tasks → `CronJob` (bb-cron-job)
+- **Push live updates to browsers** (chat, presence, dashboards) → `Realtime` (bb-realtime)
+- **Build AI features**
+  - Agent with tool use + conversation → `Agent` (bb-agent)
+  - Semantic document retrieval (RAG) → `KnowledgeBase` (bb-knowledge-base)
+- **Send transactional email** → `EmailClient` (bb-email-client)
+- **Serve a raw HTTP endpoint** (webhook receiver, health check, redirect, non-JSON response) → `RawRoute` (core); everything else goes through `ApiNamespace` RPC
+- **Observe and operate**
+  - Structured logs → `Logger` (bb-logger)
+  - Custom metrics → `Metrics` (bb-metrics)
+  - Distributed traces → `Tracer` (bb-tracer)
+  - Auto CloudWatch dashboard → `Dashboard` (bb-dashboard)
+
+### Choosing a data block
+
+Default to `DistributedTable` for your data models unless your domain specifically requires SQL engine capabilities.
+
+Reach for one of the SQL blocks when you need to filter or join results across more than one related record, filter models on many dimensions with no preset hierarchy, store large objects, require transactions, or otherwise need the flexibility or familiarity of SQL that NoSQL does not offer.
+
+If you need SQL, prefer `DistributedDatabase` for basic Postgres-compatible querying. Use `Database` specifically when you need a full (more expensive) Postgres implementation where the engine itself provides and enforces foreign keys, row level security, triggers, views, large transactions (more than 3,000 rows), or integration with an existing Postgres database. Note it carries an idle cost at minimum 0.5 ACU, or a cold start when scaling from zero, unlike the other two blocks.
+
+## Building Block documentation
+
+Every Building Block ships its docs inside the `@aws-blocks/blocks` package under `docs/<block>/`: each `docs/<block>/` folder contains that block's `README.md`, plus `API.md`, `DESIGN.md`, and `CHANGELOG.md` where present. To read them, locate the bundled folder:
 
-| Building Block | Import | Use it for |
-|---|---|---|
-| `Scope` | `@aws-blocks/blocks` | Resource boundaries / grouping for your backend |
-| `ApiNamespace` | `@aws-blocks/blocks` | Type-safe APIs wired to the frontend automatically |
-| `KVStore` | `@aws-blocks/blocks` | Simple key-value get/put/delete (prefs, flags, caches) |
-| `DistributedTable` | `@aws-blocks/blocks` | Structured data with indexes and queries — **default for most data** |
-| `DistributedDatabase` | `@aws-blocks/blocks` | Serverless SQL (Aurora DSQL) — zero-ops, scales to zero |
-| `Database` | `@aws-blocks/blocks` | Full PostgreSQL (Aurora) — FKs, RLS, triggers, or an existing DB |
-| `AuthBasic` | `@aws-blocks/blocks` | Username/password auth for prototypes, internal tools, MVPs |
-| `AuthCognito` | `@aws-blocks/blocks` | Cognito User Pools — MFA, groups, hosted identity |
-| `AuthOIDC` | `@aws-blocks/blocks` | Sign-in gated by an external OIDC identity provider |
-| `Realtime` | `@aws-blocks/blocks` | Push to browsers — chat, presence, live dashboards |
-| `AsyncJob` | `@aws-blocks/blocks` | Fire-and-forget background work (emails, uploads, reports) |
-| `CronJob` | `@aws-blocks/blocks` | Scheduled / recurring tasks |
-| `FileBucket` | `@aws-blocks/blocks` | File storage — uploads, downloads, presigned URLs |
-| `AppSetting` | `@aws-blocks/blocks` | A single config value or secret (flags, API keys) |
-| `KnowledgeBase` | `@aws-blocks/blocks` | Semantic document retrieval / RAG (Bedrock + S3 Vectors) |
-| `Agent` | `@aws-blocks/blocks` | AI agent — tool use, streaming, conversation persistence |
-| `EmailClient` | `@aws-blocks/blocks` | Transactional email (SES) |
-| `Logger` / `Metrics` / `Tracer` / `Dashboard` | `@aws-blocks/blocks` | Observability — structured logs, metrics, traces, CloudWatch dashboard |
-| `Hosting` | `@aws-blocks/blocks` | Deploy a frontend (SPA / static / Next.js SSR) on CloudFront + S3 |
-
-> **Not sure which data block?** Start with `DistributedTable` (DynamoDB). Reach for SQL only when you need joins across records, many-dimensional filtering, large transactions, or an existing Postgres database — `DistributedDatabase` for serverless Postgres, `Database` for full Aurora Postgres (FKs, RLS, triggers; carries idle cost / cold starts the other two don't). The full rationale is in `docs/index.md`.
+```bash
+node -p "require('path').dirname(require.resolve('@aws-blocks/blocks/docs/README.md'))"
+```
+
+If resolution fails, fall back to `node_modules/@aws-blocks/blocks/docs`. That folder holds this guide (`README.md`), the framework's own `API.md`, `TROUBLESHOOTING.md`, and `CHANGELOG.md` (version history — read when troubleshooting), plus one subfolder per block; the catalog below lists every block.
+
+<!-- BEGIN:block-catalog -->
+| Block | What it does | Keywords |
+|-------|--------------|----------|
+| auth-common | Shared interfaces and UI components for all AWS Blocks auth Building Blocks. | — |
+| bb-agent | AI agent with streaming, tool calling, and conversation persistence. | — |
+| bb-app-setting | A single application configuration value backed by SSM Parameter Store. | — |
+| bb-async-job | Background job processing backed by SQS and Lambda. | queue, job, background, async, worker, submit, batch, retry, status, transitions, SQS |
+| bb-auth-basic | Simple username/password authentication with JWT sessions, password policy, and optional code-confirmed signup and password reset. | — |
+| bb-auth-cognito | Authentication backed by Amazon Cognito User Pools. | — |
+| bb-auth-oidc | OIDC sign-in gate for AWS Blocks applications. | — |
+| bb-cron-job | Scheduled task execution backed by EventBridge Scheduler and Lambda. | cron, schedule, timer, periodic, recurring, rate, EventBridge, background, interval |
+| bb-dashboard | Auto-generated CloudWatch Dashboard for application observability. | — |
+| bb-data | Full PostgreSQL database — provisions Aurora Serverless v2 by default, or connects to an existing PostgreSQL database (Supabase, Neon, etc.) via `fromExisting()`. | — |
+| bb-distributed-data | Serverless SQL database backed by Amazon Aurora DSQL. | — |
+| bb-distributed-table | Structured data storage backed by DynamoDB with secondary indexes and rich query capabilities. | — |
+| bb-email-client | Transactional email sending via Amazon SES. | — |
+| bb-file-bucket | File storage backed by Amazon S3. | — |
+| bb-knowledge-base | Semantic document retrieval backed by Amazon Bedrock Knowledge Bases. | — |
+| bb-kv-store | Simple key-value storage backed by DynamoDB. | — |
+| bb-logger | Structured logging with consistent JSON format, log levels, and contextual metadata. | — |
+| bb-metrics | Custom application metrics backed by Amazon CloudWatch (via Embedded Metric Format). | — |
+| bb-realtime | Real-time pub/sub messaging backed by API Gateway WebSocket + DynamoDB. | — |
+| bb-tracer | Distributed tracing backed by AWS X-Ray. | — |
+| core | Core primitives for building full-stack applications with the AWS Blocks. | — |
+| hosting | Low-level CDK L3 constructs for deploying web applications on AWS | — |
+| pipeline | CDK Pipelines-based CI/CD construct for AWS Blocks applications. | — |
+<!-- END:block-catalog -->
 
 ## Local development and deploying
 
@@ -144,6 +191,8 @@ Each block is its own package; full per-block docs ship in this package under **
 | Data | persists to `.bb-data/` (delete to reset) | lives in AWS |
 | Use for | rapid iteration, tests | pre-production validation against real services |
 
+> **Deploying needs AWS credentials.** `npm run dev` is fully local (no creds). `npm run sandbox` and `npm run deploy` provision real AWS resources, so configure credentials first — e.g. `aws configure sso` + `aws sso login`, or `aws configure` (verify with `aws sts get-caller-identity`). Use **least-privilege** credentials scoped to the services your blocks deploy — not broad `Administrator` access.
+
 `npm run deploy` does a full production deploy; `npm run sandbox:destroy` tears the sandbox down. The same backend code runs in all three — blocks switch implementations automatically.
 
 `npm run deploy` streams CloudFormation events to **stdout** as they happen, so `npm run deploy | tee deploy.log` shows live progress instead of going silent for minutes, and a deploy failure keeps its reason on **stderr** (that stays the place to grep for why a deploy failed). On POSIX the deploy also survives a stray reap: a single `SIGTERM`, or any `SIGHUP` — a closed terminal, a backgrounded `npm run deploy &` — logs a line and keeps streaming rather than abandoning a stack update CloudFormation is still applying. Press Ctrl-C, or send `SIGTERM` twice, to stop it. That signal resilience is POSIX-only: Windows has no process groups and no OS-delivered `SIGTERM`/`SIGHUP`, so there a kill on the process tree still ends the deploy.
@@ -183,9 +232,22 @@ Run with `npm run test:e2e`. Write the test first, iterate against mocks until i
 - **`Database` when `DistributedTable` would do** — Aurora costs more and has cold starts; reach for SQL only when you need it.
 - **Curling REST-style paths** — there is no `GET /api/getData`. All calls are JSON-RPC to a single `POST /aws-blocks/api`; use the typed import instead.
 
+## Security Considerations
+
+- Use `await auth.requireAuth(context)` in every method that shouldn't be public — ApiNamespace methods are **unauthenticated by default**
+- Use `new AppSetting(scope, id, { secret: true })` for API keys and credentials — never hardcode or use `.env` files
+- Always attach a schema to KVStore/AppSetting that accepts user data — the RPC layer validates structure but not business logic
+- Do not add broad `*` IAM policies — each Building Block already grants least-privilege scoped to its own resources
+- Never change `blockPublicAccess` on FileBucket — serve public files through CloudFront instead
+- Configure `CORS_ALLOWED_ORIGINS` explicitly for production — avoid wildcards
+- For cross-domain deployments, pass `crossDomain: true` to auth constructors (enables `SameSite=None; Secure; Partitioned`)
+- Enable `monitoring: { enabled: true, snsTopicArn: '...' }` on Hosting for production alerts
+- Add WAF and API Gateway throttling via CDK for public-facing apps — not included by default
+- Logger provides serialization safety (circular refs, type coercion) but does NOT redact sensitive content — never pass raw credentials, tokens, or secrets to Logger methods; sanitize context objects before logging
+
 ## Reference
 
-- **Per-block documentation:** `docs/<package-name>.md` (e.g. `docs/bb-distributed-table.md`); `docs/index.md` for the catalog + decision tree.
+- **Per-block documentation:** `docs/<block>/README.md` (overview), plus `docs/<block>/API.md` (full API reference) and `docs/<block>/DESIGN.md` (architecture & rationale) where present — e.g. `docs/bb-distributed-table/README.md`. The catalog + decision tree live in `docs/README.md`.
 - **UI components** (`@aws-blocks/blocks/ui`): `Authenticator`, `AuthenticatedContent`, `AccountMenuBar`, `onAuthChange`, `broadcastAuthChange` — framework-agnostic, return DOM nodes. See the `@aws-blocks/auth-common` README.
 - **SSR** (`@aws-blocks/blocks/server`): `withAuth` forwards browser cookies to API calls during server rendering. See the `@aws-blocks/core` README.
 - **Wire protocol & debugging:** the client is JSON-RPC 2.0 over a single endpoint — you should never call it directly. For `curl`-level troubleshooting, see [TROUBLESHOOTING.md](./TROUBLESHOOTING.md).

packages/blocks/package.json

@@ -45,10 +45,12 @@
     "./utils": {
       "types": "./dist/utils.d.ts",
       "default": "./dist/utils.js"
-    }
+    },
+    "./docs/*": "./docs/*"
   },
   "scripts": {
-    "prebuild": "node ../../scripts/sync-block-docs.mjs",
+    "prebuild": "node ../../scripts/gen-block-docs.mjs",
+    "prepack": "node ../../scripts/gen-block-docs.mjs",
     "build": "tsc --build",
     "test": "node --test dist/conditional-exports.test.js dist/vendorize-map.test.js"
   },

packages/create-blocks-app/resources/AGENTS.md

@@ -5,9 +5,7 @@
 - **Backend:** `aws-blocks/index.ts` — APIs, auth, data models
 - **Frontend:** `src/` — imports backend APIs via `import { api } from 'aws-blocks'`
 - **Tests:** `test/e2e.test.ts` — run with `npm run test:e2e`
-- **Full guide:** `node_modules/@aws-blocks/blocks/README.md` — architecture, workflow, best practices, common mistakes
-- **Block catalog + decision tree:** `node_modules/@aws-blocks/blocks/docs/index.md`
-- **Per-block docs:** `node_modules/@aws-blocks/blocks/docs/<package-name>.md`
+- **AWS Blocks docs** ship inside the `@aws-blocks/blocks` package. Find the docs folder once: `node -p "require('path').dirname(require.resolve('@aws-blocks/blocks/docs/README.md'))"` (fallback: `node_modules/@aws-blocks/blocks/docs`). Read everything relative to it: `README.md` (dev guide + catalog + decision tree — start here), then `<block>/README.md`, plus `<block>/API.md` and `<block>/DESIGN.md` where present.
 
 ## Workflow
 
@@ -19,7 +17,7 @@
 ## Rules
 
 - **Use Building Blocks** for all persistence and cloud abstractions — never local files, in-memory arrays, or local databases.
-- **Read block docs** at `node_modules/@aws-blocks/blocks/docs/<package-name>.md` before using a block.
+- **Read block docs** before using a block — start with its `README.md`, then `API.md` / `DESIGN.md` where present — not every block has them, so a missing file is not an error (see the **AWS Blocks docs** bullet above for where the docs folder lives).
 - **The JSON-RPC transport is invisible** — do not construct RPC payloads manually. Import and call the typed API directly.
 
 ## Deploying (requires AWS credentials)