From 95cac9076bd5d4a9967b6813da74c1e3a6321c00 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Tue, 8 Sep 2026 03:37:45 +0530 Subject: [PATCH 01/41] Create 2026-09-07-what-to-double-down-on.md --- .../2026-09-07-what-to-double-down-on.md | 199 ++++++++++++++++++ 1 file changed, 199 insertions(+) create mode 100644 docs/research/2026-09-07-what-to-double-down-on.md diff --git a/docs/research/2026-09-07-what-to-double-down-on.md b/docs/research/2026-09-07-what-to-double-down-on.md new file mode 100644 index 0000000..5352c1f --- /dev/null +++ b/docs/research/2026-09-07-what-to-double-down-on.md @@ -0,0 +1,199 @@ +# What we build, and how + +**Date:** 2026-09-07 + +Agents are the users now. Someone runs codegen once to scaffold their tools, +then asks their own model to improve them. We stop building anything the +user's own agent already does. What's left is four parts. + +## codegen — the scaffold + +The front door. Three jobs: + +**1. Bake in the safety defaults nobody applies by hand.** Chrome's guidance +sets hard limits and per-tool flags; across 40 endpoints everyone forgets +some. The generator applies them to every tool, every run: + +- Name and parameter names max 30 characters, tool descriptions max 500, + parameter descriptions max 150. The generator writes text that fits; + spec text that's too long gets cut at a sentence end and marked as + machine-written. +- Tool outputs max 1.5K characters. You can't check this before the tool + runs, so the generated code passes every result through a helper that + truncates it — and that helper sits in the part of the file only the + generator edits, so nobody deletes it by accident. +- Read tools get marked read-only automatically (done). Outputs containing + free-text fields get marked as untrusted user content, including fields + nested inside arrays and objects (0.7). Origin scoping (`exposedTo`) ships + as a config option once `registerTool` supports it; until then the report + flags it. +- Write and destructive tools ask the user to confirm each call, in the + generator-owned part of the file (done). + +**2. Say plainly that the schema is not the security boundary.** A nice JSON +Schema doesn't make a call safe — the app still has to validate types, +permissions, and state when the tool runs. So `execute` always calls the +app's real endpoint, never a generated shortcut around it, and every +generated file carries one comment line saying exactly this. We never +describe the tool as a security feature. The phrase is "scaffolding with +safe defaults." + +**3. Generate intent-level tools, and call the output a draft.** One tool per +endpoint is the wrong shape — agents do better with a few coarse tools like +"search flights" than with a mirror of your CRUD API. So generation gets a +grouping step: cluster endpoints by resource, emit the obvious coarse tools, +leave mutations commented out. The report shows the grouping as a proposal; +the user or their agent adjusts it through the overrides file; regeneration +keeps their decisions. Without this, codegen is just a 1:1 mapper with +extra steps — this is what makes it not that. + +## journeys — a helper in the repo, a small file per flow + +Multi-step flows stay in the product. The honest split: + +**What codegen can't do:** read an OpenAPI spec and discover that "create a +trip" is really search → set details → create → open the editor. That +knowledge lives in the product, not the API contract. No automatic journey +detection, ever — a CLI guessing flows produces plausible garbage. + +**What codegen owns:** the machinery that's identical for every journey — +a shared draft, step tools that fill it, a submit gate. Shipped as a +`createJourney` helper scaffolded into the user's repo, same as the tools. + +**What the user (or their agent) writes:** one small file per flow. Two real +examples from beenthere's generated surface: + +```ts +// journeys/document-trip.webmcp.ts +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + searchPlaces: { tool: getAutocompleteTool }, // GET /v1/places/autocomplete + setDetails: { input: { title: "string", startDate: "string", endDate: "string" } }, + }, + submit: { tool: "create-trip", confirm: true }, // POST /v1/trips/ +}); +``` + +Why this journey exists: `create-trip`'s input has a `locationObject` with 7 +required fields (`placeId`, `fullAddress`, `country`, ...) that an agent +cannot invent — it comes from `get-autocomplete`. Called cold, the agent +hallucinates a place and fails server-side validation. The journey's draft +carries the resolved place from step 1 into the submit, so the hard input is +real by construction. + +Why it's named `document-trip` and not `plan-trip`: beenthere is a journal +for trips you've *been on*, not a planner. Naming it `plan-trip` would pass +every mechanical check — verb-first, short, maps to a real endpoint — and +still be wrong. That class of mistake is the skill file's job; see below. + +```ts +// journeys/collect-stamp.webmcp.ts +export const collectStamp = createJourney({ + name: "collect-stamp", + goal: "Generate a stamp for a city the user has a qualifying trip for", + steps: { + pickTrip: { input: { tripId: "string" } }, + checkEligibility: { tool: getTripStampEligibilityTool }, // GET .../stamp-eligibility + }, + submit: { tool: "generate-stamp", confirm: true }, // POST /v1/stamps/generate +}); +``` + +Why this one exists: `generate-stamp` is a paid generation call, and a stamp +only makes sense for a trip that qualifies. Without the journey shape, an +agent generates stamps for cities the user never visited; with it, the +eligibility check gates the spend, and the submit reuses the trip from the +draft. + +The helper does everything generic at runtime: registers the step tools with +correct descriptions, budgets, and annotations; the submit tool refuses +until the required steps are filled, asks the human to confirm, calls the +real endpoint through the existing tool's `execute`, clears the draft. The +dashboard lets you play the journey as an agent would — call steps out of +order, see the gate block, see the confirmation. `verify` checks the file: +submit gate present, budgets met, no PII from the draft leaking into tool +outputs, step count small enough for an agent to track. + +No JSON config DSL — the declaration references the user's schemas and +tools, so it's code anyway; a second config language is weight for nothing. +The first scaffold of an existing app can't emit journeys (the spec doesn't +contain them); the most the report does is hint: "3 stamp endpoints and a +per-trip eligibility read look like one flow — declare a journey?" The skill +file teaches the user's agent how to turn that hint into the files above. + +## skill file — the rules, sitting in the user's repo + +One markdown file the generator drops next to the tools. It teaches whatever +agent the user runs the WebMCP best practices: naming, description limits, +which tools to expose, the safety rules, how to collapse a 1:1 draft into a +few intent-level tools. + +This is how the rules reach the user's model when it edits their tools — no +API keys, no network calls, works even when the prompt is lazy. Their agent +knows their product, which our CLI never can, so this is also where good +grouping decisions actually happen. + +The file gets tested like code: a set of evals runs it against real models, +so a wording change that makes agents write worse tools shows up as a failed +test, not a vibe. The sharpest fixture: "given beenthere's generated surface, +write the trip journey" has a known right answer — retrospective, named like +`document-trip`. If a wording change makes agents produce `plan-trip` again, +a test fails. + +The hardest rule it teaches: **before naming an intent-level tool or +journey, understand the product** — what it is, who uses it, when in the +user's life the action happens. If that's not written down anywhere, the +agent asks the user. A wrong-tense or wrong-role name (`plan-trip` on a +memories product) passes every mechanical check and still ships wrong; this +is the one failure no deterministic check can catch, so the discipline lives +here, in the file the agent reads while it works. + +## audit package — the checks + +All the checks live in their own package, `@webmcp-stack/audit`. One copy, +so nothing can drift. Three readers: `verify` (a thin wrapper that reads +local files and calls the package), the dashboard's report view, and the +paste-a-URL audit when it ships — same checks, pointed at a live site. + +The checks are fixed and run the same way every time, no model involved: +the character limits, whether the output matches what the OpenAPI schema +says, whether the tool set is small enough for an agent to pick from. + +`verify` is the CI gate — it runs the package and exits 1 on errors, so CI +can block a bad tool set. There are also optional checks that use a model — +"would an agent pick these tools for these common requests? is this tool set +shaped around what users actually want?" Those need a key, are off by +default, and never change the exit code. That rule doesn't bend. + +## dashboard — the report, not an editor + +Nobody opens a dashboard to hand-edit descriptions anymore; they ask a +model. So the editing gets deleted — no more writing back to the overrides +file from the UI. + +What's left: see your tools, see the verify results, run the model-based +review. The one control that stays is the enable/disable toggle, because +deciding what to expose to agents is a decision we want a human to make. + +## What gets removed + +- **The LLM suggestion layer** (`--suggest`, `--llm`, `src/llm.ts`). Frozen + the day the skill file ships, deleted a release later. The user's own + model does this better. +- **The big roadmap** (Test, Control, Observe, Secure). Narrowed to two + jobs: write the rules down, check tools against them. + +## Order of work + +1. Ship the 0.7 tool standard — character limits, nested untrusted-content + checks, disabled tools stop registering. Mostly specced already. +2. Extract the checks into `@webmcp-stack/audit`; `verify` becomes a thin + wrapper and gets the character-limit checks. +3. The grouping step — intent-level tools by default, proposal in the report. +4. The skill file, with evals — including the "how to define a journey" + guidance. +5. Dashboard surgery — editing out, report in. Delete the LLM layer. +6. Journeys scaffold — declared flows wired by codegen, per the journeys + spec. From abdebae1bf0eee582f737d1c54e0e44cb41ed99a Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Wed, 9 Sep 2026 21:59:14 +0530 Subject: [PATCH 02/41] to be added --- .../2026-09-07-what-to-double-down-on.md | 384 +++++++++++++----- packages/codegen/assets/journey.webmcp.ts | 159 ++++++++ packages/codegen/assets/skill/SKILL.md | 117 ++++++ 3 files changed, 565 insertions(+), 95 deletions(-) create mode 100644 packages/codegen/assets/journey.webmcp.ts create mode 100644 packages/codegen/assets/skill/SKILL.md diff --git a/docs/research/2026-09-07-what-to-double-down-on.md b/docs/research/2026-09-07-what-to-double-down-on.md index 5352c1f..6eaf13d 100644 --- a/docs/research/2026-09-07-what-to-double-down-on.md +++ b/docs/research/2026-09-07-what-to-double-down-on.md @@ -4,7 +4,10 @@ Agents are the users now. Someone runs codegen once to scaffold their tools, then asks their own model to improve them. We stop building anything the -user's own agent already does. What's left is four parts. +user's own agent already does. What's left: make the existing surface more +reliable, add the two valuable things (skill file, journeys), and remove +what's unnecessary. Parked for later: the audit package and the dashboard +rework. ## codegen — the scaffold @@ -14,21 +17,22 @@ The front door. Three jobs: sets hard limits and per-tool flags; across 40 endpoints everyone forgets some. The generator applies them to every tool, every run: -- Name and parameter names max 30 characters, tool descriptions max 500, - parameter descriptions max 150. The generator writes text that fits; - spec text that's too long gets cut at a sentence end and marked as - machine-written. -- Tool outputs max 1.5K characters. You can't check this before the tool - runs, so the generated code passes every result through a helper that - truncates it — and that helper sits in the part of the file only the +- Names max 30 characters: enforced at generation and checked by `verify` + (shipped in 0.7). Still to build: tool descriptions max 500 and parameter + descriptions max 150 — composed to fit at generation, trimmed at a + sentence end and marked machine-written when inherited spec text + overflows, and measured by `verify` so CI gates on them. +- Tool outputs max 1.5K characters (to build). You can't check this before + the tool runs, so the generated code passes every result through a helper + that truncates it — and that helper sits in the part of the file only the generator edits, so nobody deletes it by accident. -- Read tools get marked read-only automatically (done). Outputs containing - free-text fields get marked as untrusted user content, including fields - nested inside arrays and objects (0.7). Origin scoping (`exposedTo`) ships - as a config option once `registerTool` supports it; until then the report - flags it. +- Read tools get marked read-only automatically (shipped). Outputs + containing free-text fields get marked as untrusted user content, + including fields nested inside arrays, objects, and nullable unions + (shipped in 0.7). Origin scoping (`exposedTo`) ships as a config option + once `registerTool` supports it; until then the report flags it. - Write and destructive tools ask the user to confirm each call, in the - generator-owned part of the file (done). + generator-owned part of the file (shipped). **2. Say plainly that the schema is not the security boundary.** A nice JSON Schema doesn't make a call safe — the app still has to validate types, @@ -47,21 +51,163 @@ the user or their agent adjusts it through the overrides file; regeneration keeps their decisions. Without this, codegen is just a 1:1 mapper with extra steps — this is what makes it not that. -## journeys — a helper in the repo, a small file per flow +## journeys — one file we ship, small files the user's agent writes -Multi-step flows stay in the product. The honest split: +**Our code is exactly one file: `journey.webmcp.ts`.** When `generate` runs, +it writes this file into the user's repo next to `runtime.webmcp.ts`, under +the same contract as the runtime file: fully ours, regenerated on every run, +never hand-edited. ~90 lines, no dependencies beyond the runtime helpers +that already ship. This is the entire journey machinery — complete, not +abbreviated (the same file lives at +`packages/codegen/assets/journey.webmcp.ts`): -**What codegen can't do:** read an OpenAPI spec and discover that "create a -trip" is really search → set details → create → open the editor. That -knowledge lives in the product, not the API contract. No automatic journey -detection, ever — a CLI guessing flows produces plausible garbage. - -**What codegen owns:** the machinery that's identical for every journey — -a shared draft, step tools that fill it, a submit gate. Shipped as a -`createJourney` helper scaffolded into the user's repo, same as the tools. +```ts +import { + getModelContext, toolResult, toolError, asToolError, requestUserConfirmation, + type WebMcpToolResult, +} from "./runtime.webmcp"; + +type Json = Record; + +export interface JourneyStep { + description: string; // what the agent reads: "Search real places and store the pick." + input: Json; // the step's input fields, as a JSON Schema object + provides: string[]; // draft fields this step leaves behind; submit waits for them + run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + // ^ what the step does — usually calls an existing tool's execute. + // Gets the draft so far. Returns the fields to store. Default: store the input. +} + +export interface JourneyDef { + name: string; // "document-trip" — step tools derive from it + goal: string; // the one sentence every step repeats to the agent + steps: Record; + submit: { + description: string; // what the human confirms + build: (draft: Readonly) => unknown; // assemble the real tool's input + run: (input: never, signal?: AbortSignal) => Promise; // the real tool's execute + }; +} + +export function createJourney(def: JourneyDef) { + const modelContext = getModelContext(); + let draft: Json = {}; // ← THE DRAFT. One plain object per page load. + + function missing(): string[] { + return Object.entries(def.steps).flatMap(([key, step]) => + step.provides + .filter((field) => draft[field] === undefined) + .map((field) => `${def.name}-${key} (stores "${field}")`), + ); + } + + async function registerSteps(signal?: AbortSignal): Promise { + if (!modelContext) return; + for (const [key, step] of Object.entries(def.steps)) { + await modelContext.registerTool({ + name: `${def.name}-${key}`, + description: `${step.description} Part of "${def.name}": ${def.goal}`, + inputSchema: step.input, + annotations: { readOnlyHint: true }, + execute: async (input, context) => { + try { + const stored = step.run + ? await step.run(input as Json, context?.signal, { ...draft }) + : (input as Json); + Object.assign(draft, stored); + const left = missing(); + return toolResult( + left.length === 0 + ? `Stored. The journey is ready — call ${def.name}-submit.` + : `Stored. Still needed: ${left.join(", ")}.`, + ); + } catch (error) { + return asToolError(error); + } + }, + }, { signal }); + } + } + + async function registerSubmit(signal?: AbortSignal): Promise { + if (!modelContext) return; + await modelContext.registerTool({ + name: `${def.name}-submit`, + description: def.submit.description, + inputSchema: { type: "object", properties: {} }, + annotations: { readOnlyHint: false }, + execute: async (_input, context) => { + context?.signal?.throwIfAborted(); + const left = missing(); + if (left.length > 0) { + return toolError(`Not ready to submit. Call these first: ${left.join(", ")}.`); + } + const confirmed = await requestUserConfirmation( + `Allow the agent to: ${def.submit.description}`, + ); + if (!confirmed) return toolError("The user declined this action."); + try { + const result = await def.submit.run(def.submit.build(draft) as never, context?.signal); + draft = {}; // a submitted journey starts clean + return result as WebMcpToolResult; + } catch (error) { + return asToolError(error); + } + }, + }, { signal }); + } + + return { + async register(signal?: AbortSignal): Promise { + await registerSteps(signal); + await registerSubmit(signal); + }, + inspectDraft: (): Json => ({ ...draft }), // for the dashboard and tests + }; +} +``` -**What the user (or their agent) writes:** one small file per flow. Two real -examples from beenthere's generated surface: +Reading that code, the three pieces are: + +1. **The draft** — `let draft = {}`, one plain object per page load. Steps + write into it; the submit reads from it. It dies with the page: a + half-finished journey doesn't survive a reload, which is what you want. +2. **Step tools** — `registerSteps`: ordinary registered WebMCP tools named + `-` (`document-trip-search-places`). A step's `execute` + stores its result into the draft and replies with what's still missing, + so the agent always knows the next move. +3. **The submit gate** — `registerSubmit`: one more tool, `-submit`, + whose `execute` does four things in order: refuse with the missing list → + ask the human to confirm → run the real tool's `execute` with + `build(draft)` → clear the draft. No way around it, because the real + input only exists inside `build(draft)`. + +**What the CLI does around that one file — three mechanical jobs:** + +1. **Copy it in.** The same code path that already scaffolds + `runtime.webmcp.ts` on every `generate`. +2. **Register journeys on page load.** The generated `index.ts` — the file + that today exports `registerAllTools()` — also imports every export of + `journeys/*.webmcp.ts` and calls its `.register()`. Dropping a new + journey file into that folder makes it live with zero wiring. +3. **Check journey files in `verify`.** Submit gate present, every step + described within budget, step count ≤5, no PII-shaped draft field leaking + into a step's output. + +Plus one report line at generate time when endpoints cluster like a flow +("3 stamp endpoints and a per-trip eligibility read look like one flow — +declare a journey?"). A hint, never an auto-generation. + +**What we never write: the journey definitions themselves.** Every +`journeys/*.webmcp.ts` file is the user's agent's code, written with the +skill file's guidance — only the product side knows the flow. What codegen +can't do: read an OpenAPI spec and discover that "create a trip" is really +search → set details → create → open the editor. That knowledge lives in +the product, not the API contract. A CLI guessing flows produces plausible +garbage. + +Two real examples of those user-side files, from beenthere's generated +surface: ```ts // journeys/document-trip.webmcp.ts @@ -69,10 +215,26 @@ export const documentTrip = createJourney({ name: "document-trip", goal: "Record a trip you've been on and open the editor to write its story", steps: { - searchPlaces: { tool: getAutocompleteTool }, // GET /v1/places/autocomplete - setDetails: { input: { title: "string", startDate: "string", endDate: "string" } }, + "search-places": { + description: "Search real places and store the pick.", + input: { type: "object", properties: { input: { type: "string" } }, required: ["input"] }, + provides: ["locationObject"], + run: async (input, signal) => { + const res = await executeGetAutocomplete({ input: String(input.input) }, signal); + return { locationObject: pickFrom(res) }; // the user's pick + }, + }, + "set-details": { + description: "Set the trip's title and dates.", + input: { type: "object", properties: { title: { type: "string" }, startDate: { type: "string" }, endDate: { type: "string" } }, required: ["title"] }, + provides: ["title"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft as CreateTripInput, + run: executeCreateTrip, // POST /v1/trips/ }, - submit: { tool: "create-trip", confirm: true }, // POST /v1/trips/ }); ``` @@ -94,10 +256,25 @@ export const collectStamp = createJourney({ name: "collect-stamp", goal: "Generate a stamp for a city the user has a qualifying trip for", steps: { - pickTrip: { input: { tripId: "string" } }, - checkEligibility: { tool: getTripStampEligibilityTool }, // GET .../stamp-eligibility + "pick-trip": { + description: "Pick the trip to base the stamp on.", + input: { type: "object", properties: { tripId: { type: "string" } }, required: ["tripId"] }, + provides: ["tripId"], + }, + "check-eligibility": { + description: "Check what stamp this trip qualifies for.", + input: { type: "object", properties: {} }, + provides: ["eligibility"], + run: async (_input, signal, draft) => ({ + eligibility: await executeGetTripStampEligibility({ tripId: String(draft.tripId) }, signal), + }), + }, + }, + submit: { + description: "Generate the stamp (a paid generation call).", + build: (draft) => ({ name: cityFrom(draft.eligibility) }), + run: executeGenerateStamp, // POST /v1/stamps/generate }, - submit: { tool: "generate-stamp", confirm: true }, // POST /v1/stamps/generate }); ``` @@ -105,77 +282,83 @@ Why this one exists: `generate-stamp` is a paid generation call, and a stamp only makes sense for a trip that qualifies. Without the journey shape, an agent generates stamps for cities the user never visited; with it, the eligibility check gates the spend, and the submit reuses the trip from the -draft. - -The helper does everything generic at runtime: registers the step tools with -correct descriptions, budgets, and annotations; the submit tool refuses -until the required steps are filled, asks the human to confirm, calls the -real endpoint through the existing tool's `execute`, clears the draft. The -dashboard lets you play the journey as an agent would — call steps out of -order, see the gate block, see the confirmation. `verify` checks the file: -submit gate present, budgets met, no PII from the draft leaking into tool -outputs, step count small enough for an agent to track. +draft. (A step's `run` receives the draft so far — that's how +`check-eligibility` reads the `tripId` that `pick-trip` stored.) No JSON config DSL — the declaration references the user's schemas and tools, so it's code anyway; a second config language is weight for nothing. -The first scaffold of an existing app can't emit journeys (the spec doesn't -contain them); the most the report does is hint: "3 stamp endpoints and a -per-trip eligibility read look like one flow — declare a journey?" The skill -file teaches the user's agent how to turn that hint into the files above. +The skill file teaches the user's agent how to turn the report's cluster +hint into the files above. ## skill file — the rules, sitting in the user's repo -One markdown file the generator drops next to the tools. It teaches whatever -agent the user runs the WebMCP best practices: naming, description limits, -which tools to expose, the safety rules, how to collapse a 1:1 draft into a -few intent-level tools. +The actual file is written and reviewable: +`packages/codegen/assets/skill/SKILL.md` (standard skill format — YAML +frontmatter with a `description` written as the trigger, then the rules). +The generator drops it into the user's repo on first run, next to the tools. +It teaches whatever agent the user runs: the naming rules, description +budgets, exposure decisions, the execute contract, how to edit generated +files without fighting regeneration, and the journey pattern above. This is how the rules reach the user's model when it edits their tools — no API keys, no network calls, works even when the prompt is lazy. Their agent knows their product, which our CLI never can, so this is also where good grouping decisions actually happen. -The file gets tested like code: a set of evals runs it against real models, -so a wording change that makes agents write worse tools shows up as a failed -test, not a vibe. The sharpest fixture: "given beenthere's generated surface, -write the trip journey" has a known right answer — retrospective, named like -`document-trip`. If a wording change makes agents produce `plan-trip` again, -a test fails. - The hardest rule it teaches: **before naming an intent-level tool or journey, understand the product** — what it is, who uses it, when in the user's life the action happens. If that's not written down anywhere, the agent asks the user. A wrong-tense or wrong-role name (`plan-trip` on a memories product) passes every mechanical check and still ships wrong; this is the one failure no deterministic check can catch, so the discipline lives -here, in the file the agent reads while it works. - -## audit package — the checks - -All the checks live in their own package, `@webmcp-stack/audit`. One copy, -so nothing can drift. Three readers: `verify` (a thin wrapper that reads -local files and calls the package), the dashboard's report view, and the -paste-a-URL audit when it ships — same checks, pointed at a live site. - -The checks are fixed and run the same way every time, no model involved: -the character limits, whether the output matches what the OpenAPI schema -says, whether the tool set is small enough for an agent to pick from. - -`verify` is the CI gate — it runs the package and exits 1 on errors, so CI -can block a bad tool set. There are also optional checks that use a model — -"would an agent pick these tools for these common requests? is this tool set -shaped around what users actually want?" Those need a key, are off by -default, and never change the exit code. That rule doesn't bend. - -## dashboard — the report, not an editor - -Nobody opens a dashboard to hand-edit descriptions anymore; they ask a -model. So the editing gets deleted — no more writing back to the overrides -file from the UI. - -What's left: see your tools, see the verify results, run the model-based -review. The one control that stays is the enable/disable toggle, because -deciding what to expose to agents is a decision we want a human to make. +in the file the agent reads while it works. + +### The eval plan + +Following the two references the user pointed at +([OpenAI](https://developers.openai.com/blog/eval-skills), +[philschmid](https://www.philschmid.de/testing-skills)): an eval is +prompt → captured run → checks → score. Concretely for this skill: + +1. **Prompt set** — 10–20 cases in `evals/skill/`, each a situation plus its + own expected checks. Categories: trigger tests ("add a webmcp tool for + X" should activate the skill), core tasks (write a tool from a schema, + improve a generated description, enable a withheld write tool), the + journey fixture (below), and negative controls (an unrelated coding + prompt — the skill must not trigger). +2. **The sharpest fixture** — a copy of beenthere's generated surface with + the prompt "write the trip journey." The right answer is retrospective: + name matches `document|record|log`, and `/plan|book/` is an automatic + fail. Regex-checkable, no judge needed. +3. **Harness** — run the agent CLI headlessly (`codex exec --json` or + equivalent) in a clean copy of the fixture repo per case, 3–5 trials per + case since behavior is nondeterministic. Grade outcomes, not paths. +4. **Deterministic checks first** — produced file parses; name verb-first + and ≤30 chars; description ≤500 and says what returns; writes keep the + confirmation call; `execute` calls the real endpoint; nothing edited + above the generated marker; journeys use `createJourney` with a submit + gate. +5. **LLM rubric second, selectively** — for what regex can't grade (is the + tool intent-shaped? did the agent ask about product context?), a second + pass constrained to a structured schema (`overall_pass`, per-check + results), so scores diff across runs. +6. **Operate like tests** — every real failure becomes a new case; once a + case hits ~100% it graduates into a regression suite; run the suite with + the skill unloaded occasionally — if everything still passes, the model + absorbed the skill and we retire it. + +## Out of scope for now + +Two pieces from earlier versions of this direction are parked — real, but +later, after the reliability work above lands: + +- **The audit package** (`@webmcp-stack/audit`): extracting the checks into + their own package with `verify` as a thin wrapper, plus the paste-a-URL + audit. `verify` keeps growing inside codegen for now; extraction happens + when the URL-audit product actually starts. +- **The dashboard as a report**: browse + scorecard + model-based review, + with the editing UI removed. Not now. (Removing the LLM layer is *not* + parked — that's deletion, and deletion is in scope.) ## What gets removed @@ -187,13 +370,24 @@ deciding what to expose to agents is a decision we want a human to make. ## Order of work -1. Ship the 0.7 tool standard — character limits, nested untrusted-content - checks, disabled tools stop registering. Mostly specced already. -2. Extract the checks into `@webmcp-stack/audit`; `verify` becomes a thin - wrapper and gets the character-limit checks. -3. The grouping step — intent-level tools by default, proposal in the report. -4. The skill file, with evals — including the "how to define a journey" - guidance. -5. Dashboard surgery — editing out, report in. Delete the LLM layer. -6. Journeys scaffold — declared flows wired by codegen, per the journeys - spec. +Already shipped (0.7–0.8.2, current on npm): withheld-by-default writes, +confirmation gates, the naming rules, descriptions that say what a tool +returns, nested untrusted-content marking, and `verify` with its scorecard +and `--url` check. + +What's left, in order: + +1. Close the budget gaps: the 500/150 description limits in generation and + `verify`, and the 1.5K output-truncation helper in the generated region. +2. The skill file (`assets/skill/SKILL.md` — written, needs the scaffold + wiring) plus its eval harness. +3. Journeys — ship the `createJourney` helper (`assets/journey.webmcp.ts` — + written, needs the scaffold wiring) and the verify checks for journey + files. +4. The grouping step — intent-level tools by default, proposal in the + report. +5. Delete the LLM layer (`--llm`, `--suggest`, shipped in 0.8) once the + skill file has shipped and nobody has complained. + +Parked (out of scope for now): the audit-package extraction, the +dashboard-as-report rework, and the paste-a-URL audit. diff --git a/packages/codegen/assets/journey.webmcp.ts b/packages/codegen/assets/journey.webmcp.ts new file mode 100644 index 0000000..9b572d7 --- /dev/null +++ b/packages/codegen/assets/journey.webmcp.ts @@ -0,0 +1,159 @@ +/** + * Written by webmcp-codegen on every `generate` run. Do not edit by hand; + * your changes will be lost. This file is fully ours — journey definitions + * (your code) live in journeys/*.webmcp.ts and import createJourney from here. + * + * createJourney: multi-step agent flows with a shared draft and one submit. + * + * The three pieces, literally: + * + * 1. THE DRAFT — one plain object per page load (`let draft = {}` below). + * Nothing fancier: steps write their results into it, the submit reads + * from it. It dies with the page; a half-finished journey does not + * survive a reload, which is what you want. + * + * 2. STEP TOOLS — ordinary registered WebMCP tools, one per step, named + * "-" (e.g. "document-trip-search-places"). A step's + * execute stores what it produced into the draft, then replies with what + * is still missing, so the agent always knows the next move. + * + * 3. THE SUBMIT GATE — one more registered tool, "-submit". Its + * execute, in order: refuses with the list of missing steps, asks the + * human to confirm, runs the real tool's execute with the assembled + * input, clears the draft. There is no way to submit around it, because + * the real input only exists inside build(draft). + */ + +import { + asToolError, + getModelContext, + requestUserConfirmation, + toolError, + toolResult, + type WebMcpToolResult, +} from "../runtime.webmcp"; + +type Json = Record; + +export interface JourneyStep { + /** What the agent reads, e.g. "Search real places and store the pick." */ + description: string; + /** The step's input fields, as a JSON Schema object. */ + input: Json; + /** + * Draft fields this step leaves behind. Submit refuses until every step's + * fields are present. A step with no `run` stores its input verbatim. + */ + provides: string[]; + /** + * What the step does with its input — usually calling an existing tool's + * execute. Receives the draft so far (a later step needs what an earlier + * one stored) and must return the draft fields to store. + * Default: store the input verbatim. + */ + run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; +} + +export interface JourneyDef { + /** Journey name; step tools derive from it ("document-trip-search-places"). */ + name: string; + /** The one sentence every step repeats to the agent, so the goal survives. */ + goal: string; + steps: Record; + submit: { + /** What the human confirms, e.g. "Create the trip and open the editor." */ + description: string; + /** Assemble the real tool's input from the draft. This is your code. */ + build: (draft: Readonly) => unknown; + /** The existing tool's execute — your real endpoint runs here. */ + run: (input: never, signal?: AbortSignal) => Promise; + }; +} + +export function createJourney(def: JourneyDef) { + const modelContext = getModelContext(); + + /** The shared draft. Page-scoped on purpose: reloads start clean. */ + let draft: Json = {}; + + /** Every (step, field) pair the draft still lacks, as readable text. */ + function missing(): string[] { + return Object.entries(def.steps).flatMap(([key, step]) => + step.provides + .filter((field) => draft[field] === undefined) + .map((field) => `${def.name}-${key} (stores "${field}")`), + ); + } + + async function registerSteps(signal?: AbortSignal): Promise { + if (!modelContext) return; + for (const [key, step] of Object.entries(def.steps)) { + await modelContext.registerTool( + { + name: `${def.name}-${key}`, + description: `${step.description} Part of "${def.name}": ${def.goal}`, + inputSchema: step.input, + annotations: { readOnlyHint: true }, + execute: async (input, context) => { + try { + const stored = step.run + ? await step.run(input as Json, context?.signal, { ...draft }) + : (input as Json); + Object.assign(draft, stored); + const left = missing(); + return toolResult( + left.length === 0 + ? `Stored. The journey is ready — call ${def.name}-submit.` + : `Stored. Still needed: ${left.join(", ")}.`, + ); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + } + + async function registerSubmit(signal?: AbortSignal): Promise { + if (!modelContext) return; + await modelContext.registerTool( + { + name: `${def.name}-submit`, + description: def.submit.description, + inputSchema: { type: "object", properties: {} }, + annotations: { readOnlyHint: false }, + execute: async (_input, context) => { + context?.signal?.throwIfAborted(); + const left = missing(); + if (left.length > 0) { + return toolError(`Not ready to submit. Call these first: ${left.join(", ")}.`); + } + const confirmed = await requestUserConfirmation( + `Allow the agent to: ${def.submit.description}`, + ); + if (!confirmed) return toolError("The user declined this action."); + try { + const result = await def.submit.run(def.submit.build(draft) as never, context?.signal); + draft = {}; // a submitted journey starts clean + return result as WebMcpToolResult; + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + + return { + /** Call once on page load, next to registerAllTools(). */ + async register(signal?: AbortSignal): Promise { + await registerSteps(signal); + await registerSubmit(signal); + }, + /** What's in the draft right now — for the dashboard and for tests. */ + inspectDraft: (): Json => ({ ...draft }), + }; +} diff --git a/packages/codegen/assets/skill/SKILL.md b/packages/codegen/assets/skill/SKILL.md new file mode 100644 index 0000000..a7d45ef --- /dev/null +++ b/packages/codegen/assets/skill/SKILL.md @@ -0,0 +1,117 @@ +--- +name: webmcp-tools +description: Build, improve, or review this repo's WebMCP tools and journeys — the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. +--- + +# WebMCP tools in this repo + +WebMCP tools run in the visitor's browser, with the signed-in user's session, +while the agent calling them may also be reading attacker-influenced page +content. Every tool you write is both an API and a security surface. The rules +below exist so an agent-facing tool is correct by default — follow them even +when the user's request is casual. + +## Naming + +- Verb-first, intent-shaped, max 30 characters: `list-trips`, + `add-bucket-list-destination`. Never method-first (`get-v1-trips`), never + numbered (`get-pricing-2`). +- Name the user's intent, not the endpoint. `POST /search` is a read named + `search-...`; `POST /orders/{id}/cancel` is destructive named `cancel-...`. +- **Understand the product before naming anything.** What is it, who uses it, + and when in the user's life does this action happen? If that is not written + down in the repo, ask the user before naming intent-level tools or journeys. + Wrong-tense or wrong-role names pass every mechanical check and are still + wrong: on a journal for trips you've *been on*, the flow is `document-trip`, + never `plan-trip`. This is the failure no linter can catch — it is your job. + +## Descriptions + +- Say what the tool does, when to use it, and what it returns: + "Create a new trip. Returns the trip." +- Max 500 characters for the tool, max 150 per parameter. Turn constraints + into sentences: "A number from 30 to 600." +- Never instruct the agent or encode flow control in a description + ("always call X first") — that is steering. Prerequisites belong in a + journey, not in prose. +- If a field's value can only come from another tool (a resolved place object, + a server id), say so in that field's description. + +## Safety and exposure + +- Reads are registered immediately. Writes and destructive tools stay + withheld — generated but not registered — until the user deliberately + enables one. Never enable a write tool without being asked. +- Mutating tools confirm each call with the human via + `requestUserConfirmation`. That call lives in the generated region; never + move or remove it. +- Free-text outputs get `untrustedContentHint: true` — the agent must not + treat user-written content as the site speaking. +- **The schema is not the security boundary.** `execute` must call the app's + real endpoint or action layer, so server-side validation runs on every + call. Never wire `execute` to return canned data or bypass the app's own + flow (cache invalidation, navigation, stores). + +## The execute contract + +- Never throw for failure. The browser maps a rejected `execute` to a bare + `UnknownError` and discards your message. Return `toolError(message)` / + `asToolError(error)` so the agent can read and recover. (Cancellation is + the one exception: let `AbortError` propagate.) +- Return via `toolResult(data)` and keep outputs under ~1.5K characters — + summarize or paginate rather than dumping. +- When a call changes what is on screen, make it visible: navigate, + invalidate a query, dispatch an event. The human is watching the page. + +## Editing generated files + +- Each `*.webmcp.ts` has a generated region between the + `webmcp-codegen` markers — never edit inside it; regeneration rewrites it. + Your work goes below the marker (the `execute` body) or in + `.webmcp-codegen.json` (description/name/enabled overrides, which survive + regeneration and always win over generated text). +- After editing tools, run `npx @webmcp-stack/codegen verify` and fix what it + reports. + +## Journeys (multi-step flows) + +Reach for a journey when a goal takes several calls with shared state, when +an input can't be invented (a resolved place object), or when a spend should +be gated (an eligibility check before a paid generation). Pattern: + +```ts +import { createJourney } from "../webmcp/journey.webmcp"; + +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + "search-places": { + description: "Search real places and store the pick.", + input: { type: "object", properties: { input: { type: "string" } }, required: ["input"] }, + provides: ["locationObject"], + run: async (input, signal) => { + const res = await executeGetAutocomplete({ input: String(input.input) }, signal); + // Store the resolved place the user picked from the results: + return { locationObject: res }; + }, + }, + "set-details": { + description: "Set the trip's title and dates.", + input: { type: "object", properties: { title: { type: "string" } }, required: ["title"] }, + provides: ["title"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft, // assemble the real tool's input + run: executeCreateTrip, // the existing tool does the work + }, +}); +``` + +- 2–5 steps. More means two journeys. +- Steps reuse existing tools' `execute` functions; the submit's `run` is the + real write tool, so its confirmation and validation still apply. +- The submit gate is the only write in a journey; step tools are reads or + draft-writes and stay read-only. From c2b2c424e64cdfd90484a152e18a748ef9bcb1f6 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 01:46:58 +0530 Subject: [PATCH 03/41] Sharpen direction doc: concrete journey machinery, skill file, eval plan Restructure after review: plain language throughout, version claims verified against source, audit package and dashboard rework marked out of scope. Journeys section now shows the actual helper contract and composition-based examples from beenthere's real surface. --- .../2026-09-07-what-to-double-down-on.md | 174 ++++++------------ 1 file changed, 59 insertions(+), 115 deletions(-) diff --git a/docs/research/2026-09-07-what-to-double-down-on.md b/docs/research/2026-09-07-what-to-double-down-on.md index 6eaf13d..176d095 100644 --- a/docs/research/2026-09-07-what-to-double-down-on.md +++ b/docs/research/2026-09-07-what-to-double-down-on.md @@ -51,118 +51,63 @@ the user or their agent adjusts it through the overrides file; regeneration keeps their decisions. Without this, codegen is just a 1:1 mapper with extra steps — this is what makes it not that. +**Grouping and journeys are not the same thing.** Grouping *merges*: several +endpoints that are one action split by API shape become one tool — +beenthere's `create-media-request-upload` + `complete-media-upload` are an +upload handshake no agent should see; one `upload-media` tool wraps both +calls. It happens at generate time, from the spec, no human input. Journeys +*chain*: several different decisions with shared state and one guarded +submit (`document-trip`). They're written after generation, by the user's +agent, because the flow lives in the product. One-line test: same action +split across calls → group it; different decisions along the way → journey. +Grouping shrinks the standalone surface; journeys make dangerous writes +reachable under a gate — and tools a journey covers get absorbed into it +(flow-only tools never register standalone), so the total registered surface +goes down, not up. `verify`'s surface budget counts journey tools too, so +this is enforced, not aspirational. + ## journeys — one file we ship, small files the user's agent writes **Our code is exactly one file: `journey.webmcp.ts`.** When `generate` runs, it writes this file into the user's repo next to `runtime.webmcp.ts`, under the same contract as the runtime file: fully ours, regenerated on every run, -never hand-edited. ~90 lines, no dependencies beyond the runtime helpers -that already ship. This is the entire journey machinery — complete, not -abbreviated (the same file lives at -`packages/codegen/assets/journey.webmcp.ts`): +never hand-edited. ~120 lines, no dependencies beyond the runtime helpers +that already ship. The complete, current copy lives at +`packages/codegen/assets/journey.webmcp.ts` — that file is the review +artifact; this section explains it. + +The contract it exposes — this is the part that matters: ```ts -import { - getModelContext, toolResult, toolError, asToolError, requestUserConfirmation, - type WebMcpToolResult, -} from "./runtime.webmcp"; - -type Json = Record; - -export interface JourneyStep { - description: string; // what the agent reads: "Search real places and store the pick." - input: Json; // the step's input fields, as a JSON Schema object - provides: string[]; // draft fields this step leaves behind; submit waits for them - run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; - // ^ what the step does — usually calls an existing tool's execute. - // Gets the draft so far. Returns the fields to store. Default: store the input. +// A step backed by an existing generated tool: inherits its description and +// input schema, calls the raw caller the generated file exports. You write +// only what's new — which slice of the result lands in the draft. +interface ToolStep { + tool: { description?: string; inputSchema?: Json }; // e.g. getAutocompleteTool + call: (input, signal, draft) => Promise; // e.g. fetchGetAutocomplete + store: (result: unknown) => Json; // what lands in the draft + provides: string[]; // draft fields submit waits for + description?: string; // override; default: the tool's + input?: Json; // override; default: the tool's +} + +// A step with no backend call — it just collects input into the draft +// ("set the title and dates"). With a run, it can do work first. +interface FreeStep { + description: string; + input: Json; + provides: string[]; + run?: (input, signal, draft) => Promise; // default: store the input } -export interface JourneyDef { +interface JourneyDef { name: string; // "document-trip" — step tools derive from it goal: string; // the one sentence every step repeats to the agent - steps: Record; + steps: Record; // ToolStep | FreeStep submit: { - description: string; // what the human confirms - build: (draft: Readonly) => unknown; // assemble the real tool's input - run: (input: never, signal?: AbortSignal) => Promise; // the real tool's execute - }; -} - -export function createJourney(def: JourneyDef) { - const modelContext = getModelContext(); - let draft: Json = {}; // ← THE DRAFT. One plain object per page load. - - function missing(): string[] { - return Object.entries(def.steps).flatMap(([key, step]) => - step.provides - .filter((field) => draft[field] === undefined) - .map((field) => `${def.name}-${key} (stores "${field}")`), - ); - } - - async function registerSteps(signal?: AbortSignal): Promise { - if (!modelContext) return; - for (const [key, step] of Object.entries(def.steps)) { - await modelContext.registerTool({ - name: `${def.name}-${key}`, - description: `${step.description} Part of "${def.name}": ${def.goal}`, - inputSchema: step.input, - annotations: { readOnlyHint: true }, - execute: async (input, context) => { - try { - const stored = step.run - ? await step.run(input as Json, context?.signal, { ...draft }) - : (input as Json); - Object.assign(draft, stored); - const left = missing(); - return toolResult( - left.length === 0 - ? `Stored. The journey is ready — call ${def.name}-submit.` - : `Stored. Still needed: ${left.join(", ")}.`, - ); - } catch (error) { - return asToolError(error); - } - }, - }, { signal }); - } - } - - async function registerSubmit(signal?: AbortSignal): Promise { - if (!modelContext) return; - await modelContext.registerTool({ - name: `${def.name}-submit`, - description: def.submit.description, - inputSchema: { type: "object", properties: {} }, - annotations: { readOnlyHint: false }, - execute: async (_input, context) => { - context?.signal?.throwIfAborted(); - const left = missing(); - if (left.length > 0) { - return toolError(`Not ready to submit. Call these first: ${left.join(", ")}.`); - } - const confirmed = await requestUserConfirmation( - `Allow the agent to: ${def.submit.description}`, - ); - if (!confirmed) return toolError("The user declined this action."); - try { - const result = await def.submit.run(def.submit.build(draft) as never, context?.signal); - draft = {}; // a submitted journey starts clean - return result as WebMcpToolResult; - } catch (error) { - return asToolError(error); - } - }, - }, { signal }); - } - - return { - async register(signal?: AbortSignal): Promise { - await registerSteps(signal); - await registerSubmit(signal); - }, - inspectDraft: (): Json => ({ ...draft }), // for the dashboard and tests + description: string; // what the human confirms + build: (draft) => unknown; // assemble the real tool's input + run: (input: never, signal?) => Promise; // the real tool's execute }; } ``` @@ -185,7 +130,10 @@ Reading that code, the three pieces are: **What the CLI does around that one file — three mechanical jobs:** 1. **Copy it in.** The same code path that already scaffolds - `runtime.webmcp.ts` on every `generate`. + `runtime.webmcp.ts` on every `generate`. One template change rides along: + generated tool files also export the raw caller (`fetchGetAutocomplete` — + just the `callApi` line, unwrapped), because journeys compose raw data, + not agent-shaped results. 2. **Register journeys on page load.** The generated `index.ts` — the file that today exports `registerAllTools()` — also imports every export of `journeys/*.webmcp.ts` and calls its `.register()`. Dropping a new @@ -216,13 +164,10 @@ export const documentTrip = createJourney({ goal: "Record a trip you've been on and open the editor to write its story", steps: { "search-places": { - description: "Search real places and store the pick.", - input: { type: "object", properties: { input: { type: "string" } }, required: ["input"] }, + tool: getAutocompleteTool, // schema + description inherited + call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), + store: (places) => ({ locationObject: pickFrom(places) }), // the resolved pick provides: ["locationObject"], - run: async (input, signal) => { - const res = await executeGetAutocomplete({ input: String(input.input) }, signal); - return { locationObject: pickFrom(res) }; // the user's pick - }, }, "set-details": { description: "Set the trip's title and dates.", @@ -233,7 +178,7 @@ export const documentTrip = createJourney({ submit: { description: "Create the trip and open it in the editor.", build: (draft) => draft as CreateTripInput, - run: executeCreateTrip, // POST /v1/trips/ + run: executeCreateTrip, // POST /v1/trips/ }, }); ``` @@ -262,18 +207,17 @@ export const collectStamp = createJourney({ provides: ["tripId"], }, "check-eligibility": { - description: "Check what stamp this trip qualifies for.", - input: { type: "object", properties: {} }, + tool: getTripStampEligibilityTool, + input: { type: "object", properties: {} }, // the agent provides nothing + call: (_input, signal, draft) => fetchGetTripStampEligibility({ tripId: String(draft.tripId) }, signal), + store: (eligibility) => ({ eligibility }), provides: ["eligibility"], - run: async (_input, signal, draft) => ({ - eligibility: await executeGetTripStampEligibility({ tripId: String(draft.tripId) }, signal), - }), }, }, submit: { description: "Generate the stamp (a paid generation call).", build: (draft) => ({ name: cityFrom(draft.eligibility) }), - run: executeGenerateStamp, // POST /v1/stamps/generate + run: executeGenerateStamp, // POST /v1/stamps/generate }, }); ``` @@ -282,7 +226,7 @@ Why this one exists: `generate-stamp` is a paid generation call, and a stamp only makes sense for a trip that qualifies. Without the journey shape, an agent generates stamps for cities the user never visited; with it, the eligibility check gates the spend, and the submit reuses the trip from the -draft. (A step's `run` receives the draft so far — that's how +draft. (A step's `call` receives the draft so far — that's how `check-eligibility` reads the `tripId` that `pick-trip` stored.) No JSON config DSL — the declaration references the user's schemas and From 3f8179022857a6b722f038eaba126d77b1eabf82 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 01:47:02 +0530 Subject: [PATCH 04/41] Switch journeys to composition: steps inherit generated tools MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit journey.webmcp.ts gains ToolStep — a step references the generated tool object, calls its raw caller, and declares only what lands in the draft (store), so tool definitions live in one place and never drift. FreeStep keeps the freeform form for steps with no backend call. SKILL.md updated to the same shape, with the no-direct-fetch rule. --- packages/codegen/assets/journey.webmcp.ts | 83 +++++++++++++++++------ packages/codegen/assets/skill/SKILL.md | 25 ++++--- 2 files changed, 78 insertions(+), 30 deletions(-) diff --git a/packages/codegen/assets/journey.webmcp.ts b/packages/codegen/assets/journey.webmcp.ts index 9b572d7..12b89a6 100644 --- a/packages/codegen/assets/journey.webmcp.ts +++ b/packages/codegen/assets/journey.webmcp.ts @@ -35,25 +35,46 @@ import { type Json = Record; -export interface JourneyStep { - /** What the agent reads, e.g. "Search real places and store the pick." */ - description: string; - /** The step's input fields, as a JSON Schema object. */ - input: Json; +/** + * A step backed by an existing generated tool. The step inherits the tool's + * description and input schema — the definition lives in one place — and + * calls the raw caller the generated file exports (fetchGetAutocomplete, + * not the agent-facing execute wrapper). You write only what's new: which + * slice of the result lands in the draft. + */ +export interface ToolStep { + /** The generated tool object, e.g. getAutocompleteTool. */ + tool: { description?: string; inputSchema?: Json }; /** - * Draft fields this step leaves behind. Submit refuses until every step's - * fields are present. A step with no `run` stores its input verbatim. + * The raw caller the generated file exports. Receives the step's input + * plus the draft so far, so a later step can feed on an earlier one's + * stored fields. */ + call: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** Map the call's result into the draft fields this step leaves behind. */ + store: (result: unknown) => Json; + /** Draft fields this step leaves behind. Submit waits for all of them. */ + provides: string[]; + /** Override the agent-facing description. Default: the tool's own. */ + description?: string; + /** Override the agent-facing input schema. Default: the tool's own. */ + input?: Json; +} + +/** + * A step with no backend call of its own — it collects input into the draft + * ("set the title and dates"). With a `run`, it can do work first; whatever + * `run` returns is stored. Without one, the input is stored verbatim. + */ +export interface FreeStep { + description: string; + input: Json; provides: string[]; - /** - * What the step does with its input — usually calling an existing tool's - * execute. Receives the draft so far (a later step needs what an earlier - * one stored) and must return the draft fields to store. - * Default: store the input verbatim. - */ run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; } +export type JourneyStep = ToolStep | FreeStep; + export interface JourneyDef { /** Journey name; step tools derive from it ("document-trip-search-places"). */ name: string; @@ -70,6 +91,10 @@ export interface JourneyDef { }; } +function isToolStep(step: JourneyStep): step is ToolStep { + return "tool" in step; +} + export function createJourney(def: JourneyDef) { const modelContext = getModelContext(); @@ -85,21 +110,41 @@ export function createJourney(def: JourneyDef) { ); } + function stepDescription(step: JourneyStep): string { + const base = + step.description ?? (isToolStep(step) ? step.tool.description : undefined) ?? "Journey step."; + return `${base} Part of "${def.name}": ${def.goal}`; + } + + function stepInput(step: JourneyStep): Json { + if (step.input) return step.input; + if (isToolStep(step) && step.tool.inputSchema) return step.tool.inputSchema; + return { type: "object", properties: {} }; + } + + /** Run one step and store what it produced. */ + async function runStep(step: JourneyStep, input: Json, signal?: AbortSignal): Promise { + if (isToolStep(step)) { + const result = await step.call(input, signal, { ...draft }); + Object.assign(draft, step.store(result)); + return; + } + const stored = step.run ? await step.run(input, signal, { ...draft }) : input; + Object.assign(draft, stored); + } + async function registerSteps(signal?: AbortSignal): Promise { if (!modelContext) return; for (const [key, step] of Object.entries(def.steps)) { await modelContext.registerTool( { name: `${def.name}-${key}`, - description: `${step.description} Part of "${def.name}": ${def.goal}`, - inputSchema: step.input, + description: stepDescription(step), + inputSchema: stepInput(step), annotations: { readOnlyHint: true }, execute: async (input, context) => { try { - const stored = step.run - ? await step.run(input as Json, context?.signal, { ...draft }) - : (input as Json); - Object.assign(draft, stored); + await runStep(step, input as Json, context?.signal); const left = missing(); return toolResult( left.length === 0 diff --git a/packages/codegen/assets/skill/SKILL.md b/packages/codegen/assets/skill/SKILL.md index a7d45ef..0445307 100644 --- a/packages/codegen/assets/skill/SKILL.md +++ b/packages/codegen/assets/skill/SKILL.md @@ -81,21 +81,22 @@ be gated (an eligibility check before a paid generation). Pattern: ```ts import { createJourney } from "../webmcp/journey.webmcp"; +import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; +import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; export const documentTrip = createJourney({ name: "document-trip", goal: "Record a trip you've been on and open the editor to write its story", steps: { + // Tool-backed step: inherits the generated tool's description and schema; + // you write only what lands in the draft. "search-places": { - description: "Search real places and store the pick.", - input: { type: "object", properties: { input: { type: "string" } }, required: ["input"] }, + tool: getAutocompleteTool, + call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), + store: (places) => ({ locationObject: places }), // store the resolved pick provides: ["locationObject"], - run: async (input, signal) => { - const res = await executeGetAutocomplete({ input: String(input.input) }, signal); - // Store the resolved place the user picked from the results: - return { locationObject: res }; - }, }, + // Freeform step: no backend call — collects input straight into the draft. "set-details": { description: "Set the trip's title and dates.", input: { type: "object", properties: { title: { type: "string" } }, required: ["title"] }, @@ -104,14 +105,16 @@ export const documentTrip = createJourney({ }, submit: { description: "Create the trip and open it in the editor.", - build: (draft) => draft, // assemble the real tool's input - run: executeCreateTrip, // the existing tool does the work + build: (draft) => draft as CreateTripInput, // assemble the real tool's input + run: executeCreateTrip, // the existing tool does the work }, }); ``` - 2–5 steps. More means two journeys. -- Steps reuse existing tools' `execute` functions; the submit's `run` is the - real write tool, so its confirmation and validation still apply. +- Tool-backed steps reuse the generated tool's contract and its raw caller + (`fetchX`); the submit's `run` is the real write tool's `execute`, so its + confirmation and validation still apply. Never write a direct `fetch` in a + journey file — `verify` flags it. - The submit gate is the only write in a journey; step tools are reads or draft-writes and stay read-only. From d9f68e2bea6ab4ed3cb84136d1c38b4e236d49b5 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 01:47:10 +0530 Subject: [PATCH 05/41] Add journeys docs: full-session walkthrough plus a plain-terms FAQ MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The concept page walks a complete agent session on beenthere's document-trip end to end, then the ship/write split and the rules. The FAQ answers the questions a newcomer actually asks — what the draft is, why in-memory, what the agent does vs. the human, why journeys shrink the tool surface instead of growing it. --- site/content/docs/journeys-faq.mdx | 178 +++++++++++++++++++++++++++++ site/content/docs/journeys.mdx | 162 ++++++++++++++++++++++++++ site/content/docs/meta.json | 2 + 3 files changed, 342 insertions(+) create mode 100644 site/content/docs/journeys-faq.mdx create mode 100644 site/content/docs/journeys.mdx diff --git a/site/content/docs/journeys-faq.mdx b/site/content/docs/journeys-faq.mdx new file mode 100644 index 0000000..9057170 --- /dev/null +++ b/site/content/docs/journeys-faq.mdx @@ -0,0 +1,178 @@ +--- +title: Journey questions, answered +description: Every "dumb question" about journeys, answered plainly — the draft, the lifecycle, what the agent does vs. what you do, and the edge cases. +--- + +# Journey questions, answered + +The [journeys page](/docs/journeys) explains the feature. This page answers the +questions people actually ask once they read the code — in plain terms, no +assumed knowledge. None of these are dumb questions; they're the ones everyone +has. + +## Is `journey.webmcp.ts` a journey? + +No — and this is the most common confusion. That file never creates a journey. +It's the factory: a reusable function, `createJourney()`, that journey files +call. Think Lego baseplate and Lego models. `journey.webmcp.ts` is the +baseplate — shared, generic, owned by the generator. `document-trip.webmcp.ts` +is a model someone builds on it. The engine lives in one file; the journeys +live elsewhere and are written per product. + +## What are the three parts, in dumb terms? + +1. **The draft** — `let draft = {}`. A plain JavaScript object sitting in + memory, created once when the page loads. Every step writes something into + it; the final step reads everything out of it. +2. **Step tools** — each step becomes its own normal, callable WebMCP tool + (`document-trip-search-places`). When the agent calls one, it runs the + step's logic, stuffs the result into the draft, and replies "Stored. Still + needed: X" or "Stored. Ready — call submit." That reply is literally how the + agent knows what to do next. There's no hidden orchestration — the agent + figures out the order by reading those messages, one call at a time. +3. **The submit gate** — one final tool (`document-trip-submit`) that does four + things in strict order: check nothing's missing (refuse if it is) → ask the + human to confirm → only on a yes, call the real backend with the draft's + data → clear the draft. + +## Why is it called a "draft"? + +Same reason as a draft email. It's the version-in-progress — built up piece by +piece, and nothing is sent until the very end. It's called a draft specifically +because it is *not yet the real thing*: nothing reaches the app's backend from +it until the submit step. + +## What does "storing something in the draft" actually mean? + +Literally adding a labeled value to that one object — `title: "Lisbon Trip"`, +`locationObject: {...}`. Like writing another line on a sticky note. No +database, no file, no network call is involved in the storing itself. + +## What does "in-memory" mean if I'm not technical? + +Picture a whiteboard in a room versus a filing cabinet down the hall. The +whiteboard is right there — fast, everyone in the room can read it instantly — +but the moment someone wipes it or the room closes, it's gone forever, no copy +anywhere. The filing cabinet keeps things after everyone leaves. The draft is +the whiteboard: it lives only inside the browser tab's running memory. Close +the tab, refresh, or navigate away, and it's wiped. Nothing durable, ever. + +## What's the shape of this store? + +The simplest possible: a flat bag of labeled values — +`{ locationObject: {...}, title: "Lisbon Trip" }`. No nesting enforced, any +step can write any field. One caveat: if two steps use the same field name, +the second silently overwrites the first. Avoiding that is on whoever writes +the journey. + +## Will it hold data reliably for a session? + +Depends what you mean by session: + +- **Within one continuous page view** (tab open, agent working, no refresh) — + yes, completely. It's a variable in memory; nothing can drop it mid-flow. +- **Across a reload, a closed tab, or a new tab** — no, and that's deliberate. + It is never meant to outlive one uninterrupted visit. + +## How does it store results from the real backend? + +Subtle but important: **not every step touches the backend.** A step like +`set-details` has no logic — it stores whatever the agent typed, verbatim, no +network call at all. A step like `search-places` calls the app's real search +API and stores the real answer. So the draft ends up holding a mix: some +fields are "what the agent said," some are "what the server said." Either way, +nothing is created or changed for real until the final submit. + +## Why is the draft created only once per page load? + +Because `createJourney()` — the function that sets up `let draft = {}` — runs +once, at page load, not per attempt. So there's one shared scratchpad per +journey per page visit. Fine for the realistic case (one agent, one user, one +thing at a time). The unhandled edge case: two parallel attempts at the *same* +journey on one page would share and overwrite the same draft. Accepted, not +handled. + +## What is a "half-finished journey," and why would it happen? + +The agent completes `search-places`, then the person closes the laptop, gets +distracted, the tab crashes — before the remaining steps and submit. Some steps +done, nobody submitted: half-finished. Keeping the draft memory-only means a +reload always starts completely clean — no stale, half-filled attempt from +three days ago reappearing with an outdated search result. Throwing away +incomplete progress is the feature, not a gap. + +## Does a journey reuse the tools codegen already generated? + +Yes, directly — it never reimplements anything. A tool-backed step calls the +same underlying request the standalone tool uses; the submit's `run` is the +existing write tool's `execute`. The journey just wraps them with the gate and +confirmation. Bonus: a risky write like `create-trip` — normally generated +withheld — never needs to be switched on as a standalone tool at all. The +journey's submit becomes its only door, with prerequisite checks and a human +confirmation built in. Wrapping a risky tool in a journey is a *safer* way to +expose it than enabling it directly. + +## How does the agent know these tools exist? + +No special channel — ordinary WebMCP discovery. On page load, every registered +tool (normal ones, journey steps, journey submits) lands in one flat list the +browser exposes. The agent asks the page "what can I do here," gets the whole +list, with zero formal distinction between "normal tool" and "part of a +journey." The only grouping signal is baked into the text: each step's name +carries the journey prefix (`document-trip-…`), and each description ends with +*"Part of document-trip: Record a trip you've been on…"* The agent reads that +and infers the connection — the same way it figures out anything else: by +reading. The stitching is done by the factory, mechanically, so it can't be +forgotten. + +## What does the agent do vs. what do I do? + +The agent's job is orchestration and conversation: figure out what to call +next from the reply messages, ask you for anything it can't get from the +backend. Your job is exactly one thing: the final yes/no on the confirmation +dialog before anything real happens. The agent cannot click that dialog for +you — a human click is the only way through. + +## Can the agent call the steps out of order? + +The order isn't enforced by code — an agent could call `set-details` before +`search-places`. It mostly self-corrects: a step that needs an earlier step's +draft field fails with a readable error ("Not ready…" or a failed call naming +what it needed), which nudges the agent back. One honest caveat: that +self-correction depends on the step failing *usefully* when called early — +which is on whoever writes the journey, not something the shared machinery +guarantees. + +## What happens if I decline the confirmation? + +The agent is told "The user declined this action," and the draft is left +as-is — not wiped. You can change your mind, adjust something, and submit +later without starting over. + +## Why can't codegen just generate the journeys for me? + +Because the flow isn't in your API spec. An OpenAPI file says `POST /v1/trips` +exists; it does not say that creating a trip is really search → set details → +confirm, or that your product documents past trips rather than planning future +ones. That knowledge lives in the product. A CLI guessing flows produces +plausible-sounding garbage — tools that pass every mechanical check and are +still wrong. So codegen stops at a hint in its report ("these endpoints look +like one flow — declare a journey?") and the bundled skill file teaches your +own coding agent how to write the journey file well. Your agent can ask you +about the product; our CLI can't. + +## Won't journeys make my tool list even bigger? + +They shouldn't — followed properly, they shrink it. The rule: a tool that only +makes sense inside a flow doesn't get registered on its own; its journey step +is its only face. Writes reachable through a journey submit stay withheld as +standalone tools forever. So endpoints collapse into flows instead of +double-registering. And `verify`'s surface-size check counts *registered* +tools, journey tools included — if the list grows past what's good for agents, +CI fails. That's enforced, not aspirational. + +## What if two steps use the same draft field name? + +The second overwrites the first, silently. The draft is a flat bag with no +protection — give each step's fields distinct names when you write the +journey. (Same answer as the shape question; it bites twice.) diff --git a/site/content/docs/journeys.mdx b/site/content/docs/journeys.mdx new file mode 100644 index 0000000..62ab493 --- /dev/null +++ b/site/content/docs/journeys.mdx @@ -0,0 +1,162 @@ +--- +title: Journeys +description: Some goals take several tool calls with shared state and one confirmed write at the end. A journey is that shape — here's exactly what ships, what you write, and what a real agent session looks like. +--- + +# Journeys + +Some things a user asks an agent to do can't be one tool call. "Log the trip I +took to Lisbon" means: find the real place, set the title and dates, then create +the trip — and only a human should be able to say yes to that last part. A +journey is that shape: **a few steps that share state, and one guarded submit at +the end.** + +A journey is made of three pieces: + +1. **The draft** — one plain object in the page's memory. Think whiteboard, not + filing cabinet: steps write their results on it, the submit reads from it, + and it's wiped clean when the page reloads. No database, no files, nothing + saved anywhere. +2. **Step tools** — each step becomes its own ordinary WebMCP tool. Calling one + stores what it produced into the draft and replies with what's still missing, + so the agent always knows the next move. +3. **The submit gate** — one final tool that refuses until every step has + delivered, asks the human to confirm, and only then makes the real write — + using only what's in the draft. Afterward the draft resets to empty. + +## A real session, start to finish + +beenthere.page is a journal for trips you've been on. Its journey for the core +flow is called `document-trip`, with two steps and a submit. Here's the complete +session — a signed-in user on their profile page, an agent in the browser, and +one sentence from the user: *"Log the trip I took to Lisbon last March."* + +**Page load.** The page registers its tools. Alongside the ordinary ones, three +journey tools appear: `document-trip-search-places`, `document-trip-set-details`, +and `document-trip-submit`. There's no special channel for journeys — the agent +sees one flat list and learns these belong together from their names and +descriptions, which the machinery stamps mechanically: *"Search real places and +store the pick. Part of document-trip: Record a trip you've been on…"* + +**Step 1.** The agent calls `document-trip-search-places` with +`{ input: "Lisbon" }`. That hits the real backend search, and the resolved place +— a structured object with a `placeId`, address, country — lands in the draft. +This matters more than it looks: the create-trip endpoint needs that object, and +an agent can't invent it. Now it comes from the server, not from a guess. The +tool replies: *"Stored. Still needed: document-trip-set-details."* + +**Step 2.** The agent calls `document-trip-set-details` with +`{ title: "Lisbon, March 2026", startDate: "2026-03-14" }`. No backend call — +the values are stored on the draft as-is. The tool replies: *"Stored. The +journey is ready — call document-trip-submit."* + +**The submit.** The agent calls `document-trip-submit`. The gate checks the +draft: `locationObject` present, `title` present — nothing missing. Then the +page shows a real confirmation dialog: *"Allow the agent to: Create the trip and +open it in the editor."* Only a human click can pass this — the agent cannot +approve it for you. + +**Approved.** The real `create-trip` call fires, with input assembled from the +draft and nothing else. The draft resets to empty. The app does what it does +when you click the button yourself: the trip is created and the editor opens on +it. The agent turns back to the user: "Created and opened — want me to draft the +story from your photos?" + +If the human declines, nothing is wiped: the draft stays, and the agent is told +the user declined. And if the tab closes mid-journey, the draft simply +evaporates — a half-finished journey never lingers. + +The division of labor, the whole session: the agent orchestrates and talks to +the user; the human does exactly one thing — approve or deny at the end. + +## What codegen ships vs. what you write + +Codegen ships the machinery, never the journeys: + +- **`journey.webmcp.ts`** — the `createJourney` helper, copied into your repo on + every `generate`, next to `runtime.webmcp.ts`. It's generator-owned: don't + hand-edit it; regeneration overwrites it. The draft, the step registration, + the gate, the confirmation — all of it lives here, once, for every journey. +- **Registration** — the generated `index.ts` auto-imports everything in your + `journeys/` folder and registers it. Drop in a new journey file, it's live. +- **Checks** — `verify` validates journey files: submit gate present, steps + described within budget, step count small enough for an agent to track, and no + direct `fetch` bypassing your generated tools. + +What codegen never writes is the journey itself — an OpenAPI spec can't reveal +that "create a trip" is really search → details → confirm. Only the product side +knows the flow. So you write one small file per journey (or your coding agent +does, guided by the bundled skill file): + +```ts +// journeys/document-trip.webmcp.ts +import { createJourney } from "../webmcp/journey.webmcp"; +import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; +import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; + +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + // A tool-backed step: inherits the generated tool's description and + // schema, calls its raw caller. You write only what lands in the draft. + "search-places": { + tool: getAutocompleteTool, + call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), + store: (places) => ({ locationObject: places }), // the resolved pick + provides: ["locationObject"], + }, + // A freeform step: no backend call — collects input into the draft. + "set-details": { + description: "Set the trip's title and dates.", + input: { + type: "object", + properties: { title: { type: "string" }, startDate: { type: "string" } }, + required: ["title"], + }, + provides: ["title"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft as CreateTripInput, // assemble the real tool's input + run: executeCreateTrip, // the existing tool does the work + }, +}); +``` + +## Rules worth knowing + +(Every "wait, but what if…" these rules raise is answered plainly in +[Journey questions, answered](/docs/journeys-faq).) + +- **Journeys reuse your generated tools; they don't replace them.** A + tool-backed step inherits the tool's contract and calls the same underlying + request — nothing is reimplemented. Edit the tool, regenerate, and the journey + follows. +- **A tool that only makes sense inside a flow shouldn't be registered on its + own.** `get-trip-stamp-eligibility` is meaningless outside the stamp flow, so + it lives only as a journey step. And a risky write — like `create-trip` — + stays withheld as a standalone tool forever; the journey's submit gate becomes + its only public door. Journeys should shrink the surface an agent sees, not + grow it. +- **The safety doesn't depend on the journey file.** The gate, the confirmation, + and the draft clearing are enforced by the shared helper, not by anything you + write in the definition. A journey file can be written sloppily; it can't + skip the human's yes. +- **The draft is per page load, per journey.** Within one continuous page view + it's fully reliable. Across a reload, it's gone — deliberately. Two parallel + attempts at the same journey on one page would share one draft; in practice + agents work one thing at a time, so this is accepted, not handled. +- **Keep journeys short.** Two to five steps. If you're past five, you're + describing two journeys. + +## Grouping is a different thing + +Don't confuse journeys with what the generator's grouping does. Grouping +*merges*: endpoints that are one action split by API shape (a request-upload +call and a complete-upload call) become a single coarse tool — it happens at +generate time, straight from the spec. Journeys *chain*: different decisions +with shared state and a guarded submit, written per-product after generation. +Same action split across calls → that's grouping. Different decisions along the +way → that's a journey. diff --git a/site/content/docs/meta.json b/site/content/docs/meta.json index 0058791..fca9225 100644 --- a/site/content/docs/meta.json +++ b/site/content/docs/meta.json @@ -8,6 +8,8 @@ "safety", "regeneration", "visible-effects", + "journeys", + "journeys-faq", "guides", "devtools", "troubleshooting" From 4a85bc228da834a98d387730cef765eb42a3c60b Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 02:55:11 +0530 Subject: [PATCH 06/41] Sync with the official WebMCP spec (main @ 97da8f5) Verified against the local clone of webmachinelearning/webmcp: budgets unchanged, naming compliant, withheld-by-default aligned. Gaps found: the spec's new title member and consequentialHint annotation, exposedTo now real (our flag-the-gap note is actionable), stale Chrome version claims. Watch items recorded: requestUserInteraction, skills #161, output schema #9, native validation #92, declarative API. --- .../2026-09-07-what-to-double-down-on.md | 20 +++-- docs/research/2026-09-10-spec-sync.md | 90 +++++++++++++++++++ 2 files changed, 103 insertions(+), 7 deletions(-) create mode 100644 docs/research/2026-09-10-spec-sync.md diff --git a/docs/research/2026-09-07-what-to-double-down-on.md b/docs/research/2026-09-07-what-to-double-down-on.md index 176d095..9948bbb 100644 --- a/docs/research/2026-09-07-what-to-double-down-on.md +++ b/docs/research/2026-09-07-what-to-double-down-on.md @@ -29,8 +29,10 @@ some. The generator applies them to every tool, every run: - Read tools get marked read-only automatically (shipped). Outputs containing free-text fields get marked as untrusted user content, including fields nested inside arrays, objects, and nullable unions - (shipped in 0.7). Origin scoping (`exposedTo`) ships as a config option - once `registerTool` supports it; until then the report flags it. + (shipped in 0.7). Destructive tools get marked consequential + (`consequentialHint`, spec annotation — to ship). Origin scoping + (`exposedTo`) ships as a config option — the spec API exists now + (to ship; see docs/research/2026-09-10-spec-sync.md). - Write and destructive tools ask the user to confirm each call, in the generator-owned part of the file (shipped). @@ -321,16 +323,20 @@ and `--url` check. What's left, in order: -1. Close the budget gaps: the 500/150 description limits in generation and +1. Spec sync quick wins (details: docs/research/2026-09-10-spec-sync.md): + emit `title`, auto-set `consequentialHint` on destructive tools, add the + `exposedTo` config pass-through, fix the Chrome 149 version claims in + three docs files. Template edits only. +2. Close the budget gaps: the 500/150 description limits in generation and `verify`, and the 1.5K output-truncation helper in the generated region. -2. The skill file (`assets/skill/SKILL.md` — written, needs the scaffold +3. The skill file (`assets/skill/SKILL.md` — written, needs the scaffold wiring) plus its eval harness. -3. Journeys — ship the `createJourney` helper (`assets/journey.webmcp.ts` — +4. Journeys — ship the `createJourney` helper (`assets/journey.webmcp.ts` — written, needs the scaffold wiring) and the verify checks for journey files. -4. The grouping step — intent-level tools by default, proposal in the +5. The grouping step — intent-level tools by default, proposal in the report. -5. Delete the LLM layer (`--llm`, `--suggest`, shipped in 0.8) once the +6. Delete the LLM layer (`--llm`, `--suggest`, shipped in 0.8) once the skill file has shipped and nobody has complained. Parked (out of scope for now): the audit-package extraction, the diff --git a/docs/research/2026-09-10-spec-sync.md b/docs/research/2026-09-10-spec-sync.md new file mode 100644 index 0000000..bb6dfd1 --- /dev/null +++ b/docs/research/2026-09-10-spec-sync.md @@ -0,0 +1,90 @@ +# WebMCP spec sync — 2026-09-10 + +What changed in webmachinelearning/webmcp since our last read, what it means +for us, and what we do about it. Read against the local clone of +webmachinelearning/webmcp at 97da8f5 (main, current as of today), the spec +(index.bs), and Chrome's two guides (best practices, secure tools). + +Bottom line: we've been tracking well. Three small API additions ship with the +next release, then we're back on the 09-07 plan. + +## Compliant already — no action + +- **Character budgets (500/150/30, 1.5K output).** Still the numbers in + Chrome's secure-tools guide. Our planned verify thresholds match current + guidance. +- **Name constraints.** Spec rejects names over 128 chars or containing + anything outside `[A-Za-z0-9._-]` with `InvalidStateError`. Our 30-char + kebab-case names comply. +- **Annotations.** `readOnlyHint` (auto on reads) and `untrustedContentHint` + (recursive free-text detection, shipped in 0.7) are exactly the two the + secure-tools guide recommends. +- **executeTool.** Signature is now + `executeTool(RegisteredTool, optional any inputObject, options?)` returning + a stringified result. Our dev-dashboard testing path is compatible. +- **Withheld-by-default + confirmation.** The spec's security section and + Chrome's guides push exactly the posture we already generate: registered + tools are not visible to other origins by default; consequential actions + should be gated. + +## Gaps — small, ship next release + +1. **`title` — new member we don't emit.** `ModelContextTool.title` + (USVString) is a human-facing label for native UIs (recommended + localized). RegisteredTool now carries it back. Cheap: generate + `title: "List Trips"` from the tool name. +2. **`consequentialHint` — new third annotation we don't emit.** "Executing + will result in consequential actions — significant, real-world, + non-reversible (booking a flight, transferring money)." Auto-set it on + tools our classifier marks destructive; the report can flag borderline + mutations for review. +3. **`exposedTo` has landed — our "flag it until the runtime supports it" + note is now actionable.** `registerTool(tool, { exposedTo: [...origins] })` + is in the spec (rejects non-trustworthy origins with `SecurityError`) and + documented in Chrome's secure-tools guide. Add a config option that passes + through to registration, exactly as planned. +4. **Version claims in our docs are stale.** We say Chrome 146+; the origin + trial is live in **Chrome 149** (Edge 150), and the + `#enable-webmcp-testing` flag is the local-development path. Three files: + `packages/codegen/README.md`, `site/content/docs/devtools.mdx`, + `site/content/docs/guides.mdx`. + +Implementation note for 1–3: we ship our own minimal `ModelContext` types in +the generated runtime (structural typing, no webmcp-types dependency), so the +fixes are template edits — `title?` and `consequentialHint?` on the tool +definition, `exposedTo?` on registration options. Generated code stays +compatible with browsers that don't know the new members yet. + +## Watch items — not in the spec yet, do not build on them + +- **`requestUserInteraction()`** — Chrome's secure-tools guide references it + for native user prompting/elicitation at tool-execution time, but it is not + in index.bs today. If it lands, it replaces our `confirm()`-based + `requestUserConfirmation` helper with a browser-mediated prompt — a clean + swap since confirmation calls live in the generated region. Also the + headless story (#296 made headless a goal): `window.confirm` has no headless + UX; a native API solves that too. +- **Skills integration (#161)** — still an open question in the spec. If a + native skill/flow construct ships, our journeys adopt it: the factory is one + file and the convention (prefix names, "Part of…" suffix) already mirrors + what a skill construct would formalize. Keep journeys as-is until then. +- **`outputSchema` (#9)** and **native input/output validation (#92)** — open + issues. If native validation lands, our generated schemas get checked by the + browser; nothing for us to change now, but it would make the "schema is not + the security boundary" guidance more literal. +- **Declarative (form-based) API** — real and documented + (declarative-api-explainer.md); forms gain `toolname`/`tooldescription`. + Awareness only: codegen is imperative-first. If users ask for form coverage, + that's a separate feature. +- **Service workers explainer** — background tool discovery without an open + page. Early; awareness only. + +## One philosophical note, honestly recorded + +Chrome's best practices say "trust the agent — don't enforce rigid step-by-step +procedural chains." Our journeys sequence steps, but the enforcement lives in +tool *outputs* (refusals, "still needed: X" replies), never in description +prose — the thing they warn against. The step descriptions only carry goal +context ("Part of document-trip: …"). We sit on the right side of the line, +but it's a line: if a future spec version formalizes flows (likely via #161), +we converge to it. From eb05e7a118457fad417ef095e0397283e71a6dd7 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 02:55:15 +0530 Subject: [PATCH 07/41] Emit title and consequentialHint, add exposedTo option (spec sync) Generated tool definitions now carry the spec's USVString title (derived from the tool name) and set consequentialHint on destructive-confirm tools; the journey factory does the same for step and submit tools. The tools output accepts exposedTo origins and passes them to registerTool, matching the spec's cross-origin exposure option. Chrome version references corrected to the live 149/150 origin trials. --- packages/codegen/README.md | 2 +- packages/codegen/assets/journey.webmcp.ts | 13 +++++- .../codegen/src/outputs/tools-templates.ts | 40 ++++++++++++++++--- packages/codegen/src/outputs/tools.test.ts | 32 +++++++++++++++ packages/codegen/src/outputs/tools.ts | 15 +++++-- site/content/docs/devtools.mdx | 2 +- site/content/docs/guides.mdx | 2 +- 7 files changed, 94 insertions(+), 12 deletions(-) diff --git a/packages/codegen/README.md b/packages/codegen/README.md index c25044c..4cd1bcd 100644 --- a/packages/codegen/README.md +++ b/packages/codegen/README.md @@ -150,7 +150,7 @@ export default defineConfig({ ## Requirements - Node.js ≥ 20 -- To *use* the generated tools in a browser: Chrome 146+ with `#enable-webmcp-testing` (or the WebMCP polyfill) +- To *use* the generated tools in a browser: enable `chrome://flags/#enable-webmcp-testing` for local development (Chrome 149+, Edge 150+); production pages join the WebMCP origin trial — or use the WebMCP polyfill ## License diff --git a/packages/codegen/assets/journey.webmcp.ts b/packages/codegen/assets/journey.webmcp.ts index 12b89a6..a7ac9df 100644 --- a/packages/codegen/assets/journey.webmcp.ts +++ b/packages/codegen/assets/journey.webmcp.ts @@ -95,6 +95,15 @@ function isToolStep(step: JourneyStep): step is ToolStep { return "tool" in step; } +/** "document-trip-search-places" → "Document Trip Search Places" (native UIs). */ +function toTitle(kebab: string): string { + return kebab + .split("-") + .filter(Boolean) + .map((part) => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); +} + export function createJourney(def: JourneyDef) { const modelContext = getModelContext(); @@ -139,6 +148,7 @@ export function createJourney(def: JourneyDef) { await modelContext.registerTool( { name: `${def.name}-${key}`, + title: toTitle(`${def.name}-${key}`), description: stepDescription(step), inputSchema: stepInput(step), annotations: { readOnlyHint: true }, @@ -166,9 +176,10 @@ export function createJourney(def: JourneyDef) { await modelContext.registerTool( { name: `${def.name}-submit`, + title: toTitle(`${def.name}-submit`), description: def.submit.description, inputSchema: { type: "object", properties: {} }, - annotations: { readOnlyHint: false }, + annotations: { readOnlyHint: false, consequentialHint: true }, execute: async (_input, context) => { context?.signal?.throwIfAborted(); const left = missing(); diff --git a/packages/codegen/src/outputs/tools-templates.ts b/packages/codegen/src/outputs/tools-templates.ts index 1c98e0e..1d08dda 100644 --- a/packages/codegen/src/outputs/tools-templates.ts +++ b/packages/codegen/src/outputs/tools-templates.ts @@ -33,13 +33,23 @@ import { GENERATED_END, GENERATED_START } from "./tools.js"; * track the API contract exactly: name, description, schema, input type, * hints, and the register() wrapper. */ -export function generatedRegion(tool: ReviewedTool): string { +export function generatedRegion( + tool: ReviewedTool, + registration?: { exposedTo?: string[] }, +): string { const pascal = pascalCase(tool.name); const camel = lowercaseFirst(pascal); const schemaJson = JSON.stringify(tool.inputSchema, null, 2); const inputType = jsonSchemaToTs(tool.inputSchema, undefined); const mutates = tool.riskTier !== "safe-read"; + // The second registerTool() argument: the AbortSignal always, plus the + // configured origin exposure when the app shares tools with trusted + // embedded documents (the spec's exposedTo). + const registrationOptions = registration?.exposedTo?.length + ? `{ signal, exposedTo: ${JSON.stringify(registration.exposedTo)} },` + : `{ signal },`; + // Import only what this file's regions actually use, so generated files // pass strict lint configs (no-unused-vars errors fail Next.js builds). // A disabled tool's request is commented out, so callApi/toolResult stay @@ -89,7 +99,7 @@ export function generatedRegion(tool: ReviewedTool): string { ` }`, ` },`, ` },`, - ` { signal },`, + ` ${registrationOptions}`, ` );`, ] : [ @@ -106,7 +116,7 @@ export function generatedRegion(tool: ReviewedTool): string { ` }`, ` },`, ` },`, - ` { signal },`, + ` ${registrationOptions}`, ` );`, ]; @@ -141,11 +151,13 @@ export function generatedRegion(tool: ReviewedTool): string { `/** The tool definition, minus \`execute\` (which is yours, below the marker). */`, `export const ${camel}Tool = {`, ` name: ${JSON.stringify(tool.name)},`, + ` title: ${JSON.stringify(titleFromName(tool.name))},`, ` description: ${JSON.stringify(tool.description)},`, ` inputSchema: ${camel}InputSchema,`, ` annotations: {`, ` readOnlyHint: ${tool.hints.readOnlyHint},`, ` untrustedContentHint: ${tool.hints.untrustedContentHint},`, + ` consequentialHint: ${tool.riskTier === "destructive-confirm"},`, ` },`, `};`, ``, @@ -408,10 +420,16 @@ export interface WebMcpToolResult { /** A tool as the browser runtime understands it. */ export interface WebMcpToolDefinition { name: string; + /** A human-facing label for native UIs (the spec's USVString title). */ + title?: string; description: string; inputSchema?: Record; /** Hints the agent reads to decide how careful to be with this tool. */ - annotations?: { readOnlyHint?: boolean; untrustedContentHint?: boolean }; + annotations?: { + readOnlyHint?: boolean; + untrustedContentHint?: boolean; + consequentialHint?: boolean; + }; execute: ( input: Record, context?: { signal?: AbortSignal }, @@ -422,7 +440,7 @@ export interface WebMcpToolDefinition { export interface ModelContext { registerTool( tool: WebMcpToolDefinition, - options?: { signal?: AbortSignal }, + options?: { signal?: AbortSignal; exposedTo?: string[] }, ): Promise; } @@ -580,6 +598,18 @@ export async function registerAllTools(signal?: AbortSignal): Promise { } /** "GetOrderStatus" → "getOrderStatus" (for the generated const names). */ +/** + * The spec's human-facing `title`: "list-trips" → "List Trips". Derived from + * the name so the two never disagree. + */ +function titleFromName(name: string): string { + return name + .split("-") + .filter(Boolean) + .map((part) => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); +} + function lowercaseFirst(pascal: string): string { return pascal.charAt(0).toLowerCase() + pascal.slice(1); } diff --git a/packages/codegen/src/outputs/tools.test.ts b/packages/codegen/src/outputs/tools.test.ts index f52ba17..b6deaf3 100644 --- a/packages/codegen/src/outputs/tools.test.ts +++ b/packages/codegen/src/outputs/tools.test.ts @@ -175,6 +175,38 @@ describe("js generator", () => { expect(tool?.contents).toContain("untrustedContentHint: true"); }); + it("emits a human-facing title and marks destructive tools consequential", async () => { + const files = await tools({ outDir: "src/webmcp" }).generate( + [ + reviewedTool(), + reviewedTool({ + name: "delete-order", + httpMethod: "DELETE", + riskTier: "destructive-confirm", + sideEffect: "write", + enabledByDefault: false, + withheld: true, + }), + ], + cwd, + ); + const read = files.find((file) => file.path.includes("get-order-status")); + const destructive = files.find((file) => file.path.includes("delete-order")); + expect(read?.contents).toContain('title: "Get Order Status"'); + expect(read?.contents).toContain("consequentialHint: false"); + expect(destructive?.contents).toContain('title: "Delete Order"'); + expect(destructive?.contents).toContain("consequentialHint: true"); + }); + + it("passes configured exposedTo origins to registerTool", async () => { + const files = await tools({ + outDir: "src/webmcp", + exposedTo: ["https://partner.example"], + }).generate([reviewedTool()], cwd); + const tool = files.find((file) => file.path.includes("get-order-status")); + expect(tool?.contents).toContain('{ signal, exposedTo: ["https://partner.example"] }'); + }); + it("registration skips quietly when the browser has no WebMCP", async () => { const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); const tool = files.find((file) => file.path.includes("get-order-status")); diff --git a/packages/codegen/src/outputs/tools.ts b/packages/codegen/src/outputs/tools.ts index 69a3d8d..b2210fa 100644 --- a/packages/codegen/src/outputs/tools.ts +++ b/packages/codegen/src/outputs/tools.ts @@ -42,6 +42,13 @@ import { export interface ToolsOutputOptions { /** Where the tool files go, relative to the project root. */ outDir: string; + /** + * The spec's `exposedTo`: secure origins (embedded documents at these + * origins) the registered tools are shared with. Absent means the default — + * tools are visible to the page itself, same-origin documents, and the + * browser's built-in agent. Only list origins you trust to act for your user. + */ + exposedTo?: string[]; } /** @@ -119,6 +126,7 @@ export function tools(options: ToolsOutputOptions): Output { existing.get(sourcePath), atOwnPath !== undefined && sourcePath !== ownPath, notes, + options.exposedTo, ), ); } else if (atOwnPath !== undefined) { @@ -134,10 +142,10 @@ export function tools(options: ToolsOutputOptions): Output { ]; files.push(aside); consumedPaths.add(ownPath); - files.push(toolFile(tool, ownPath, undefined, true, notes)); + files.push(toolFile(tool, ownPath, undefined, true, notes, options.exposedTo)); } else { // Brand new tool: lay down the execute() scaffold with it. - files.push(toolFile(tool, ownPath, undefined, false, notes)); + files.push(toolFile(tool, ownPath, undefined, false, notes, options.exposedTo)); } } @@ -194,8 +202,9 @@ function toolFile( existing: string | undefined, targetOccupied: boolean, notes: string[], + exposedTo?: string[], ): GeneratedFile { - const head = generatedRegion(tool); + const head = generatedRegion(tool, { exposedTo }); if (existing === undefined) { return { diff --git a/site/content/docs/devtools.mdx b/site/content/docs/devtools.mdx index 44755fd..0f68797 100644 --- a/site/content/docs/devtools.mdx +++ b/site/content/docs/devtools.mdx @@ -6,7 +6,7 @@ Chrome DevTools has a dedicated WebMCP panel that shows every tool your page exp ## Open the panel -1. Enable WebMCP: `chrome://flags/#enable-webmcp-testing` (Chrome 146+) +1. Enable WebMCP: `chrome://flags/#enable-webmcp-testing` (Chrome 149+, Edge 150+ — this flag is for local development; production pages use the origin trial) 2. Open your app and open DevTools (F12) 3. Click the **Application** tab 4. In the sidebar, click **WebMCP** diff --git a/site/content/docs/guides.mdx b/site/content/docs/guides.mdx index 3f99642..7a71178 100644 --- a/site/content/docs/guides.mdx +++ b/site/content/docs/guides.mdx @@ -43,7 +43,7 @@ actually use. Use `safety.exclude` in the config for whole families (`["internal The generated tools register on page load. To watch that happen: -- **Chrome:** turn on `chrome://flags/#enable-webmcp-testing` (Chrome 146+) and reload. +- **Chrome:** turn on `chrome://flags/#enable-webmcp-testing` (Chrome 149+; production sites join the WebMCP origin trial) and reload. The *Model Context Tool Inspector* extension shows the registered tools and lets you invoke them by hand. - **Other browsers:** add the WebMCP polyfill to your page; the same registration code From b37e089c5e0702631df6cd444b0f2aab64ccdc26 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 03:06:13 +0530 Subject: [PATCH 08/41] Enforce Chrome's character budgets: 500/150 in generation and verify, 1.5K output cap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Generation now composes within the published budgets: tool descriptions fit 500 chars and parameter descriptions 150, cut at a sentence boundary when inherited spec text overflows (fitBudget). Verify measures the final text and fails CI on offenders — new Budgets check is error-level, parameter names over 30 join the Names warnings. The runtime caps every toolResult at ~1.5K with a truncation notice; living in the shared runtime means it cannot be edited away per tool. --- packages/codegen/src/describe.test.ts | 48 ++++++++++ packages/codegen/src/describe.ts | 87 +++++++++++++----- .../codegen/src/outputs/tools-templates.ts | 23 ++++- packages/codegen/src/outputs/tools.test.ts | 7 ++ packages/codegen/src/verify.test.ts | 89 +++++++++++++++++++ packages/codegen/src/verify.ts | 64 ++++++++++++- 6 files changed, 291 insertions(+), 27 deletions(-) create mode 100644 packages/codegen/src/verify.test.ts diff --git a/packages/codegen/src/describe.test.ts b/packages/codegen/src/describe.test.ts index 70b44f4..0688e44 100644 --- a/packages/codegen/src/describe.test.ts +++ b/packages/codegen/src/describe.test.ts @@ -4,9 +4,34 @@ import { describeCandidateTool, describeConstraints, describeField, + FIELD_DESCRIPTION_MAX, + fitBudget, + TOOL_DESCRIPTION_MAX, } from "./describe.js"; import type { CandidateTool } from "./types.js"; +describe("fitBudget", () => { + it("passes text that fits through untouched", () => { + expect(fitBudget("Short text.", 150)).toBe("Short text."); + }); + + it("cuts overflow at a sentence boundary when one keeps most of the text", () => { + const text = `${"First sentence that goes on for a while. ".repeat(3)}Second sentence trails off at the very end with no stop`; + const fitted = fitBudget(text, 150); + expect(fitted.length).toBeLessThanOrEqual(150); + expect(fitted.endsWith(".")).toBe(true); + expect(fitted).not.toContain("Second sentence"); + }); + + it("hard-cuts at a word boundary with an ellipsis when no good sentence break exists", () => { + const text = "a b c d e f g h i j k l m n o p q r s t u v w x y z ".repeat(6) + "end"; + const fitted = fitBudget(text, 150); + expect(fitted.length).toBeLessThanOrEqual(150); + expect(fitted.endsWith("…")).toBe(true); + expect(fitted).not.toContain("end"); + }); +}); + describe("describeConstraints", () => { it("renders a number range the way the WebMCP docs do", () => { expect(describeConstraints({ type: "number", minimum: 30, maximum: 600 })).toBe( @@ -405,3 +430,26 @@ describe("describeCandidateInputs through nullable wrappers", () => { expect(candidate.synthesizedFields).toEqual([]); }); }); + +describe("description budgets", () => { + it("caps author field text at the 150-character parameter budget", () => { + const result = describeField("notes", { + type: "string", + description: `The notes field of the record. ${"More detail about things. ".repeat(9)}`, + }); + expect(result.description.length).toBeLessThanOrEqual(FIELD_DESCRIPTION_MAX); + expect(result.description.startsWith("The notes field of the record.")).toBe(true); + expect(result.synthesized).toBe(false); + }); + + it("caps the assembled tool description at the 500-character budget", () => { + const candidate = { + name: "list-trips", + description: `List the trips. ${"A long explanation of everything this endpoint could ever do. ".repeat(12)}`, + inputSchema: { type: "object", properties: {} }, + outputSchema: { type: "array" }, + } as unknown as CandidateTool; + describeCandidateTool(candidate); + expect(candidate.description.length).toBeLessThanOrEqual(TOOL_DESCRIPTION_MAX); + }); +}); diff --git a/packages/codegen/src/describe.ts b/packages/codegen/src/describe.ts index 5b4996f..0faddf2 100644 --- a/packages/codegen/src/describe.ts +++ b/packages/codegen/src/describe.ts @@ -28,6 +28,31 @@ import pluralize from "pluralize"; import type { CandidateTool, JsonSchema } from "./types.js"; +/** Chrome's published description budgets: 500 per tool, 150 per parameter. */ +export const TOOL_DESCRIPTION_MAX = 500; +export const FIELD_DESCRIPTION_MAX = 150; + +/** + * Fit text to a character budget. A text that fits passes through untouched. + * One that overflows is cut at the last sentence boundary that keeps at + * least half the budget (a cut near the end keeps the author's thought, at + * the price of trailing sentences); otherwise it hard-cuts at a word + * boundary and ends with an ellipsis. Composed text is always budget-safe + * before it leaves this module, and verify measures the final result. + */ +export function fitBudget(text: string, budget: number): string { + if (text.length <= budget) return text; + const slice = text.slice(0, budget); + const sentenceEnd = Math.max( + slice.lastIndexOf(". "), + slice.lastIndexOf("! "), + slice.lastIndexOf("? "), + ); + if (sentenceEnd >= Math.floor(budget / 2)) return slice.slice(0, sentenceEnd + 1); + const wordEnd = slice.lastIndexOf(" "); + return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}…`; +} + /** * Render a schema's constraints as plain-language sentences, or "" when there * is nothing worth saying. The wording mirrors the examples in Chrome's WebMCP @@ -256,34 +281,47 @@ export function describeField( const authorText = raw && !isStubDescription(raw) ? raw : ""; const constraints = describeConstraints(schema); + let result: { description: string; synthesized: boolean }; + if (authorText) { const needsConstraints = constraints && !alreadyStatesConstraints(authorText, schema); - return { + result = { description: needsConstraints ? `${authorText} ${constraints}` : authorText, synthesized: false, }; + } else { + // A conventional name ("tripId", "coverImageUrl") reads as a real sentence; + // that beats the bare humanized name because it says what the name only + // hints at. Anything else gets the name plus whatever the schema proves. + const nameText = humanizeFieldName(name); + const pattern = patternSentence(name, schema, context?.noun); + if (pattern) { + const suffix = constraints || FORMAT_SENTENCES[schema.format ?? ""] || ""; + result = { description: suffix ? `${pattern} ${suffix}` : pattern, synthesized: true }; + } else { + const format = FORMAT_SENTENCES[schema.format ?? ""]; + // "Email. An email address." says one thing twice: when the format sentence + // already carries the noun, it is the draft on its own. + if ( + format && + constraints === format && + format.toLowerCase().includes(nameText.toLowerCase()) + ) { + result = { description: format, synthesized: true }; + } else { + const draft = + format && !constraints + ? `${nameText} (${format.replace(/^An?\s+/i, "").replace(/\.$/, "")}).` + : `${nameText}.${constraints ? ` ${constraints}` : ""}`; + result = { description: draft, synthesized: true }; + } + } } - // A conventional name ("tripId", "coverImageUrl") reads as a real sentence; - // that beats the bare humanized name because it says what the name only - // hints at. Anything else gets the name plus whatever the schema proves. - const nameText = humanizeFieldName(name); - const pattern = patternSentence(name, schema, context?.noun); - if (pattern) { - const suffix = constraints || FORMAT_SENTENCES[schema.format ?? ""] || ""; - return { description: suffix ? `${pattern} ${suffix}` : pattern, synthesized: true }; - } - const format = FORMAT_SENTENCES[schema.format ?? ""]; - // "Email. An email address." says one thing twice: when the format sentence - // already carries the noun, it is the draft on its own. - if (format && constraints === format && format.toLowerCase().includes(nameText.toLowerCase())) { - return { description: format, synthesized: true }; - } - const draft = - format && !constraints - ? `${nameText} (${format.replace(/^An?\s+/i, "").replace(/\.$/, "")}).` - : `${nameText}.${constraints ? ` ${constraints}` : ""}`; - return { description: draft, synthesized: true }; + // The 150-character parameter budget applies to the final text, author or + // machine: overflow is cut at a sentence boundary (see fitBudget). + result.description = fitBudget(result.description, FIELD_DESCRIPTION_MAX); + return result; } /** @@ -427,5 +465,10 @@ export function describeCandidateTool(candidate: CandidateTool): void { : ""; // The join is between sentences: the base earns its period first. const base = returns && !/[.!?]$/.test(normalized) ? `${normalized}.` : normalized; - candidate.description = [base, returns].filter(Boolean).join(" "); + // The 500-character tool budget applies to the final text, author or + // machine: overflow is cut at a sentence boundary (see fitBudget). + candidate.description = fitBudget( + [base, returns].filter(Boolean).join(" "), + TOOL_DESCRIPTION_MAX, + ); } diff --git a/packages/codegen/src/outputs/tools-templates.ts b/packages/codegen/src/outputs/tools-templates.ts index 1d08dda..cd346b5 100644 --- a/packages/codegen/src/outputs/tools-templates.ts +++ b/packages/codegen/src/outputs/tools-templates.ts @@ -504,12 +504,27 @@ export async function callApi( } } -/** Wrap a result in the MCP shape, so tool bodies stay one line. */ +/** Chrome's output budget: one tool result stays under ~1.5K characters. */ +const TOOL_OUTPUT_MAX = 1536; + +const TRUNCATED_NOTICE = + "\n… [truncated to fit the 1.5K output budget — return a smaller slice or paginate]"; + +/** + * Wrap a result in the MCP shape, so tool bodies stay one line. The result + * text is capped at Chrome's ~1.5K per-call output budget: oversized payloads + * cost the agent context and can trip guardrails, so they are cut with a + * notice rather than delivered whole. The cap lives here in the shared + * runtime, so it cannot be edited away per tool. + */ export function toolResult(data: unknown): WebMcpToolResult { + const text = typeof data === "string" ? data : JSON.stringify(data, null, 2); + const fitted = + text.length <= TOOL_OUTPUT_MAX + ? text + : text.slice(0, TOOL_OUTPUT_MAX - TRUNCATED_NOTICE.length) + TRUNCATED_NOTICE; return { - content: [ - { type: "text", text: typeof data === "string" ? data : JSON.stringify(data, null, 2) }, - ], + content: [{ type: "text", text: fitted }], }; } diff --git a/packages/codegen/src/outputs/tools.test.ts b/packages/codegen/src/outputs/tools.test.ts index b6deaf3..80e8256 100644 --- a/packages/codegen/src/outputs/tools.test.ts +++ b/packages/codegen/src/outputs/tools.test.ts @@ -58,6 +58,13 @@ describe("js generator", () => { await rm(cwd, { recursive: true, force: true }); }); + it("caps tool results at the 1.5K output budget in the shared runtime", async () => { + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const runtime = files.find((file) => file.path.endsWith("runtime.webmcp.ts")); + expect(runtime?.contents).toContain("TOOL_OUTPUT_MAX"); + expect(runtime?.contents).toContain("truncated to fit the 1.5K output budget"); + }); + it("emits a runtime, a barrel, and one file per tool", async () => { const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); const paths = files.map((file) => file.path); diff --git a/packages/codegen/src/verify.test.ts b/packages/codegen/src/verify.test.ts new file mode 100644 index 0000000..ed6d384 --- /dev/null +++ b/packages/codegen/src/verify.test.ts @@ -0,0 +1,89 @@ +import { describe, expect, it } from "vitest"; +import type { ReviewedTool } from "./types.js"; +import { verifyTools } from "./verify.js"; + +function reviewedTool(overrides: Partial = {}): ReviewedTool { + return { + id: "GET /orders/{id}", + name: "get-order-status", + source: { kind: "openapi", ref: "GET /orders/{id}" }, + inputSchema: { + type: "object", + properties: { orderId: { type: "string", description: "The order ID" } }, + required: ["orderId"], + }, + description: "Returns the current status and tracking info for an order by ID.", + descriptionSource: "openapi-summary", + sideEffect: "read", + enabledByDefault: true, + withheld: false, + riskTier: "safe-read", + hints: { readOnlyHint: true, untrustedContentHint: false }, + ...overrides, + } as ReviewedTool; +} + +describe("verify description budgets", () => { + it("flags a tool description over the 500-character budget as an error", () => { + const checks = verifyTools([reviewedTool({ description: `List. ${"word ".repeat(120)}` })]); + const budgets = checks.find((check) => check.area === "Budgets"); + expect(budgets?.level).toBe("error"); + expect(budgets?.findings[0]).toContain("get-order-status"); + expect(budgets?.findings[0]).toContain("500"); + }); + + it("flags a parameter description over the 150-character budget, with its field", () => { + const checks = verifyTools([ + reviewedTool({ + inputSchema: { + type: "object", + properties: { notes: { type: "string", description: `Notes. ${"word ".repeat(40)}` } }, + }, + }), + ]); + const budgets = checks.find((check) => check.area === "Budgets"); + expect(budgets?.level).toBe("error"); + expect(budgets?.findings[0]).toContain("get-order-status → notes"); + expect(budgets?.findings[0]).toContain("150"); + }); + + it("walks nested fields for the parameter budget", () => { + const checks = verifyTools([ + reviewedTool({ + inputSchema: { + type: "object", + properties: { + meta: { + type: "object", + properties: { remark: { type: "string", description: `R. ${"word ".repeat(40)}` } }, + }, + }, + }, + }), + ]); + const budgets = checks.find((check) => check.area === "Budgets"); + expect(budgets?.level).toBe("error"); + expect(budgets?.findings[0]).toContain("remark"); + }); + + it("is silent when everything is within budget", () => { + const checks = verifyTools([reviewedTool()]); + expect(checks.find((check) => check.area === "Budgets")).toBeUndefined(); + }); + + it("flags parameter names over 30 characters as a naming warning", () => { + const checks = verifyTools([ + reviewedTool({ + inputSchema: { + type: "object", + properties: { + the_unreasonably_long_parameter_name: { type: "string", description: "An id." }, + }, + }, + }), + ]); + const names = checks.find((check) => check.area === "Names"); + expect(names?.level).toBe("warning"); + expect(names?.findings[0]).toContain("the_unreasonably_long_parameter_name"); + }); +}); diff --git a/packages/codegen/src/verify.ts b/packages/codegen/src/verify.ts index 5fcc331..fc9a0ff 100644 --- a/packages/codegen/src/verify.ts +++ b/packages/codegen/src/verify.ts @@ -12,11 +12,15 @@ * findings so CI can gate on it. */ +import { FIELD_DESCRIPTION_MAX, TOOL_DESCRIPTION_MAX } from "./describe.js"; import type { AuditFinding, ReviewedTool } from "./types.js"; /** Chrome's published guidance for tool names. */ const NAME_MAX = 30; +/** Chrome's budget for parameter names (same as tool names). */ +const PARAM_NAME_MAX = 30; + /** Verbs a good tool name starts with. Method verbs plus the action words * the naming algorithm knows; the list lives here because verify's whole * job is judging names from the outside. */ @@ -128,6 +132,28 @@ function check(area: string, offenders: string[], okText: string): VerifyCheck { }; } +/** + * Walk every input field of a schema — nested objects and array items, two + * levels down, mirroring the describe layer's coverage — and run `visit` on + * each name and its text. + */ +function eachField( + schema: { + properties?: Record; + items?: unknown; + }, + depth: number, + visit: (name: string, description: string | undefined) => void, +): void { + if (depth > 2) return; + for (const [name, field] of Object.entries(schema.properties ?? {})) { + visit(name, typeof field.description === "string" ? field.description : undefined); + eachField(field as typeof schema, depth + 1, visit); + const items = (field as { items?: unknown }).items; + if (items && typeof items === "object") eachField(items as typeof schema, depth + 1, visit); + } +} + /** True when any field (including nested ones, two levels down) lacks text. * The root schema is the object itself, not a field, so it is not checked. */ function hasBareField( @@ -165,10 +191,33 @@ export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { .map( (tool) => `${tool.name} (${tool.name.length} chars) — rename it in the dashboard or config.`, ); + const longParamNames: string[] = []; + const longParamDescriptions: string[] = []; + for (const tool of registered) { + eachField(tool.inputSchema, 0, (name, description) => { + if (name.length > PARAM_NAME_MAX) { + longParamNames.push( + `${tool.name} → ${name} (${name.length} chars) — parameter names max ${PARAM_NAME_MAX}.`, + ); + } + if (description && description.length > FIELD_DESCRIPTION_MAX) { + longParamDescriptions.push( + `${tool.name} → ${name} (${description.length} chars) — over the ${FIELD_DESCRIPTION_MAX}-character parameter budget; tighten it.`, + ); + } + }); + } const nonVerb = registered .filter((tool) => !KNOWN_VERBS.has(tool.name.split("-")[0] ?? "")) .map((tool) => `${tool.name} — agents pick tools by their first word; lead with the action.`); + const longDescriptions = registered + .filter((tool) => tool.description && tool.description.length > TOOL_DESCRIPTION_MAX) + .map( + (tool) => + `${tool.name} (${tool.description.length} chars) — over the ${TOOL_DESCRIPTION_MAX}-character tool budget; tighten it.`, + ); + const noDescription = registered .filter((tool) => !tool.description || tool.description.trim() === "") .map((tool) => `${tool.name} — no description; the tool is invisible to agents.`); @@ -192,7 +241,7 @@ export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { const checks: VerifyCheck[] = [ check( "Names", - [...longNames, ...nonVerb], + [...longNames, ...longParamNames, ...nonVerb], `all ${registered.length} names within 30 characters, verb-first`, ), check( @@ -204,6 +253,19 @@ export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { check("Annotations", readWithoutHint, "reads declare readOnlyHint; content declares its trust"), ]; + // The character budgets are errors: they exist to keep tool text inside + // agent guardrails, and CI gates on them (generation already composes + // within budget, so offenders here are hand-written or overrides). + const budgetOffenders = [...longDescriptions, ...longParamDescriptions]; + if (budgetOffenders.length > 0) { + checks.push({ + area: "Budgets", + summary: `${budgetOffenders.length} over budget`, + findings: budgetOffenders, + level: "error", + }); + } + // Missing descriptions are the one error: the audit treats them as fatal, // and so do we. if (noDescription.length > 0) { From 3d86331e1f76043d29b67f285ca8c2c0e8ccdc13 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 03:09:10 +0530 Subject: [PATCH 09/41] Scaffold the skill file at .agents/skills on every generate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The tools output now drops assets/skill/SKILL.md at .agents/skills/webmcp-tools/SKILL.md — the cross-client skills location that Claude Code and other compliant agents scan. Regenerated wholesale like the runtime (the header comment says why; project-specific rules stack via the user's own skill directory). Bundled assets ship in the package (files: dist + assets) and are read with a layout-tolerant resolver. --- packages/codegen/assets/skill/SKILL.md | 4 +++ packages/codegen/package.json | 3 ++- packages/codegen/src/outputs/assets.ts | 29 ++++++++++++++++++++++ packages/codegen/src/outputs/tools.test.ts | 22 ++++++++++++++++ packages/codegen/src/outputs/tools.ts | 17 +++++++++++++ 5 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 packages/codegen/src/outputs/assets.ts diff --git a/packages/codegen/assets/skill/SKILL.md b/packages/codegen/assets/skill/SKILL.md index 0445307..8264e03 100644 --- a/packages/codegen/assets/skill/SKILL.md +++ b/packages/codegen/assets/skill/SKILL.md @@ -3,6 +3,10 @@ name: webmcp-tools description: Build, improve, or review this repo's WebMCP tools and journeys — the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. --- + + # WebMCP tools in this repo WebMCP tools run in the visitor's browser, with the signed-in user's session, diff --git a/packages/codegen/package.json b/packages/codegen/package.json index 3615af4..feb6063 100644 --- a/packages/codegen/package.json +++ b/packages/codegen/package.json @@ -29,7 +29,8 @@ } }, "files": [ - "dist" + "dist", + "assets" ], "scripts": { "build": "tsup src/index.ts src/cli.ts src/sources/index.ts src/outputs/index.ts src/dev/server.ts src/dev/ui.ts --format esm --dts --sourcemap --clean", diff --git a/packages/codegen/src/outputs/assets.ts b/packages/codegen/src/outputs/assets.ts new file mode 100644 index 0000000..234b899 --- /dev/null +++ b/packages/codegen/src/outputs/assets.ts @@ -0,0 +1,29 @@ +/** + * Read a bundled asset (the skill file, the journey helper). + * + * These ship as files in the published package rather than as template + * strings in source because they are also the reviewable artifacts — the + * design docs and the docs site point at assets/ directly, and two sources + * of truth would drift. The package publishes dist + assets; the two + * candidate roots cover the built layout (dist/x.js → ../assets) and the + * source tree under test (src/outputs/x.ts → ../../assets). + */ + +import { readFile } from "node:fs/promises"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +export async function assetText(name: string): Promise { + const here = dirname(fileURLToPath(import.meta.url)); + const candidates = [join(here, "..", "assets", name), join(here, "..", "..", "assets", name)]; + for (const candidate of candidates) { + try { + return await readFile(candidate, "utf8"); + } catch { + // Try the next layout. + } + } + throw new Error( + `webmcp-codegen: bundled asset missing: ${name} (looked in ${candidates.join(", ")})`, + ); +} diff --git a/packages/codegen/src/outputs/tools.test.ts b/packages/codegen/src/outputs/tools.test.ts index 80e8256..08672de 100644 --- a/packages/codegen/src/outputs/tools.test.ts +++ b/packages/codegen/src/outputs/tools.test.ts @@ -58,6 +58,25 @@ describe("js generator", () => { await rm(cwd, { recursive: true, force: true }); }); + it("scaffolds the WebMCP skill file at the cross-client skills location", async () => { + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const skill = files.find((file) => file.path.endsWith(".agents/skills/webmcp-tools/SKILL.md")); + expect(skill).toBeDefined(); + expect(skill?.action).toBe("create"); + expect(skill?.contents).toContain("name: webmcp-tools"); + expect(skill?.contents).toContain("Regenerated by webmcp-codegen"); + + // A copy already on disk matching the asset reports unchanged. + const skillPath = join(cwd, ".agents/skills/webmcp-tools/SKILL.md"); + await mkdir(dirname(skillPath), { recursive: true }); + await writeFile(skillPath, skill?.contents ?? ""); + const again = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const skillAgain = again.find((file) => + file.path.endsWith(".agents/skills/webmcp-tools/SKILL.md"), + ); + expect(skillAgain?.action).toBe("unchanged"); + }); + it("caps tool results at the 1.5K output budget in the shared runtime", async () => { const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); const runtime = files.find((file) => file.path.endsWith("runtime.webmcp.ts")); @@ -340,6 +359,9 @@ describe("js generator", () => { const files = await tools({ outDir: absoluteOut }).generate([reviewedTool()], cwd); expect(files.length).toBeGreaterThan(0); for (const file of files) { + // The skill file is the one exception: it lives at the project root's + // cross-client skills location, not under outDir. + if (file.path.endsWith(".agents/skills/webmcp-tools/SKILL.md")) continue; expect(file.path.startsWith(absoluteOut)).toBe(true); expect(file.path.startsWith(cwd)).toBe(false); } diff --git a/packages/codegen/src/outputs/tools.ts b/packages/codegen/src/outputs/tools.ts index b2210fa..5b6f003 100644 --- a/packages/codegen/src/outputs/tools.ts +++ b/packages/codegen/src/outputs/tools.ts @@ -32,6 +32,7 @@ import { readdir, readFile } from "node:fs/promises"; import { basename, join, resolve } from "node:path"; import type { GeneratedFile, Output, ReviewedTool } from "../types.js"; +import { assetText } from "./assets.js"; import { barrelSource, generatedRegion, @@ -166,6 +167,22 @@ export function tools(options: ToolsOutputOptions): Output { const barrel = await plainFile(join(outDir, "index.ts"), barrelSource(tools)); barrel.notes = [...orphanNotes, ...(barrel.notes ?? [])]; files.unshift(await plainFile(join(outDir, "runtime.webmcp.ts"), runtimeSource()), barrel); + + // The skill file: the rules harness for the user's own coding agents, + // at the cross-client skills location. Regenerated wholesale like the + // runtime — its header comment says why (project rules belong in the + // user's own skill directory, which stacks on top). + const skillFile = await plainFile( + resolve(cwd, ".agents/skills/webmcp-tools/SKILL.md"), + await assetText("skill/SKILL.md"), + ); + if (skillFile.action !== "unchanged") { + skillFile.notes = [ + "The WebMCP skill for coding agents lives at .agents/skills/webmcp-tools/SKILL.md.", + ...(skillFile.notes ?? []), + ]; + } + files.push(skillFile); return files; }, }; From ec5d1a2059aca8602454caedc6ae66b9b70bc8f9 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 03:14:53 +0530 Subject: [PATCH 10/41] Wire journeys: factory scaffolded, barrel registers them, raw callers emitted, verify lints them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every generate now drops journey.webmcp.ts next to the runtime and scans /journeys/*.webmcp.ts into the barrel, which registers every createJourney() export at page load. Endpoint-backed tools emit a live fetchX raw caller in the generated region — the default execute composes it, and journey steps have an honest composition path (withheld tools included). Verify gains journey checks: createJourney and submit gate present, descriptions within budget (errors), step count and direct fetch usage (warnings). Journeys live inside the tools directory; docs and skill file paths updated to match. --- .../2026-09-07-what-to-double-down-on.md | 11 +- packages/codegen/assets/journey.webmcp.ts | 2 +- packages/codegen/assets/skill/SKILL.md | 8 +- packages/codegen/src/cli.ts | 26 +++- .../codegen/src/outputs/tools-templates.ts | 79 +++++++++- packages/codegen/src/outputs/tools.test.ts | 64 ++++++-- packages/codegen/src/outputs/tools.ts | 24 ++- packages/codegen/src/verify.test.ts | 94 ++++++++++- packages/codegen/src/verify.ts | 146 ++++++++++++++++++ site/content/docs/journeys.mdx | 10 +- 10 files changed, 429 insertions(+), 35 deletions(-) diff --git a/docs/research/2026-09-07-what-to-double-down-on.md b/docs/research/2026-09-07-what-to-double-down-on.md index 9948bbb..4de9fd4 100644 --- a/docs/research/2026-09-07-what-to-double-down-on.md +++ b/docs/research/2026-09-07-what-to-double-down-on.md @@ -138,8 +138,9 @@ Reading that code, the three pieces are: not agent-shaped results. 2. **Register journeys on page load.** The generated `index.ts` — the file that today exports `registerAllTools()` — also imports every export of - `journeys/*.webmcp.ts` and calls its `.register()`. Dropping a new - journey file into that folder makes it live with zero wiring. + `/journeys/*.webmcp.ts` and calls its `.register()`. Dropping + a new journey file into that folder and re-running generate is the + whole wiring story. 3. **Check journey files in `verify`.** Submit gate present, every step described within budget, step count ≤5, no PII-shaped draft field leaking into a step's output. @@ -149,7 +150,7 @@ Plus one report line at generate time when endpoints cluster like a flow declare a journey?"). A hint, never an auto-generation. **What we never write: the journey definitions themselves.** Every -`journeys/*.webmcp.ts` file is the user's agent's code, written with the +`/journeys/*.webmcp.ts` file is the user's agent's code, written with the skill file's guidance — only the product side knows the flow. What codegen can't do: read an OpenAPI spec and discover that "create a trip" is really search → set details → create → open the editor. That knowledge lives in @@ -160,7 +161,7 @@ Two real examples of those user-side files, from beenthere's generated surface: ```ts -// journeys/document-trip.webmcp.ts +// src/webmcp/journeys/document-trip.webmcp.ts export const documentTrip = createJourney({ name: "document-trip", goal: "Record a trip you've been on and open the editor to write its story", @@ -198,7 +199,7 @@ every mechanical check — verb-first, short, maps to a real endpoint — and still be wrong. That class of mistake is the skill file's job; see below. ```ts -// journeys/collect-stamp.webmcp.ts +// src/webmcp/journeys/collect-stamp.webmcp.ts export const collectStamp = createJourney({ name: "collect-stamp", goal: "Generate a stamp for a city the user has a qualifying trip for", diff --git a/packages/codegen/assets/journey.webmcp.ts b/packages/codegen/assets/journey.webmcp.ts index a7ac9df..93a777d 100644 --- a/packages/codegen/assets/journey.webmcp.ts +++ b/packages/codegen/assets/journey.webmcp.ts @@ -31,7 +31,7 @@ import { toolError, toolResult, type WebMcpToolResult, -} from "../runtime.webmcp"; +} from "./runtime.webmcp"; type Json = Record; diff --git a/packages/codegen/assets/skill/SKILL.md b/packages/codegen/assets/skill/SKILL.md index 8264e03..b2b496d 100644 --- a/packages/codegen/assets/skill/SKILL.md +++ b/packages/codegen/assets/skill/SKILL.md @@ -81,10 +81,14 @@ when the user's request is casual. Reach for a journey when a goal takes several calls with shared state, when an input can't be invented (a resolved place object), or when a spend should -be gated (an eligibility check before a paid generation). Pattern: +be gated (an eligibility check before a paid generation). Journey files live +in the `journeys/` folder inside the generated tools directory +(`src/webmcp/journeys/document-trip.webmcp.ts`); re-run `verify` after +writing one. Pattern: ```ts -import { createJourney } from "../webmcp/journey.webmcp"; +// src/webmcp/journeys/document-trip.webmcp.ts +import { createJourney } from "../journey.webmcp"; import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; diff --git a/packages/codegen/src/cli.ts b/packages/codegen/src/cli.ts index d4c41f6..d7e6aa7 100644 --- a/packages/codegen/src/cli.ts +++ b/packages/codegen/src/cli.ts @@ -19,7 +19,7 @@ */ import { existsSync } from "node:fs"; -import { writeFile } from "node:fs/promises"; +import { readdir, readFile, writeFile } from "node:fs/promises"; import { join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; import { parseArgs } from "node:util"; @@ -49,7 +49,7 @@ import { runGenerate } from "./pipeline.js"; import { resolveSetup } from "./setup.js"; import { schemaExportsToJson } from "./sources/schema.js"; import type { CodegenConfig } from "./types.js"; -import { verifyTools, verifyUrl } from "./verify.js"; +import { type JourneyFileInput, verifyJourneyFiles, verifyTools, verifyUrl } from "./verify.js"; import { applyWiring, planWiring, type WirePlan } from "./wire.js"; const HELP = ` @@ -504,6 +504,28 @@ async function verify(flags: CliFlags): Promise { const registered = result.tools.filter((tool) => !tool.withheld); const checks = verifyTools(result.tools); + // Journey files are the user's code, so verify can't get them from the + // pipeline's tool list — it reads the journeys/ folder of each tools + // output itself and lints what it finds there. + const journeyInputs: JourneyFileInput[] = []; + for (const output of setup.config.outputs) { + if (output.kind !== "tools") continue; + const journeysDir = resolve(cwd, (output as { outDir?: string }).outDir ?? "", "journeys"); + let entries: string[] = []; + try { + entries = (await readdir(journeysDir)).filter((entry) => entry.endsWith(".webmcp.ts")); + } catch { + continue; + } + for (const entry of entries) { + journeyInputs.push({ + path: join("journeys", entry), + contents: await readFile(join(journeysDir, entry), "utf8"), + }); + } + } + checks.push(...verifyJourneyFiles(journeyInputs)); + info(""); info(` ${setup.label}: ${result.tools.length} tools, ${registered.length} registered`); info(""); diff --git a/packages/codegen/src/outputs/tools-templates.ts b/packages/codegen/src/outputs/tools-templates.ts index cd346b5..f3a8f12 100644 --- a/packages/codegen/src/outputs/tools-templates.ts +++ b/packages/codegen/src/outputs/tools-templates.ts @@ -61,11 +61,15 @@ export function generatedRegion( const withheld = tool.withheld && !enabled; const hasRoute = Boolean(tool.httpMethod && tool.pathTemplate && tool.paramLocations); const runtimeImports = withheld - ? "toolDisabled" + ? hasRoute + ? "callApi, toolDisabled" + : "toolDisabled" : [ "getModelContext", ...(mutates ? ["requestUserConfirmation"] : []), - ...(enabled && hasRoute ? ["callApi"] : []), + // Every endpoint-backed tool emits a live fetchX raw caller, so + // callApi is imported whether the tool itself starts enabled or not. + ...(hasRoute ? ["callApi"] : []), ...(enabled ? ["toolResult"] : []), "asToolError", ...(enabled ? [] : ["toolDisabled"]), @@ -160,6 +164,17 @@ export function generatedRegion( ` consequentialHint: ${tool.riskTier === "destructive-confirm"},`, ` },`, `};`, + ...(hasRoute + ? [ + ``, + `/** The bare request, without the agent-facing result wrapping. Journeys`, + ` * and your own code compose this; execute${pascal} is the agent-facing one. */`, + `export async function fetch${pascal}(input: ${tool.inputTypeName}, signal?: AbortSignal) {`, + ` ${requestCall(tool)}`, + ` return data;`, + `}`, + ] + : []), ``, ...(withheld ? [ @@ -211,7 +226,11 @@ export function generatedRegion( */ export function ownedRegionScaffold(tool: ReviewedTool): string { const pascal = pascalCase(tool.name); - const call = requestCall(tool); + const hasRoute = Boolean(tool.httpMethod && tool.pathTemplate && tool.paramLocations); + // Endpoint-backed tools compose the raw caller from the generated region + // (fetchX), so the default execute stays one line and journeys reuse the + // exact same request. Schema-only tools keep the honest TODO. + const call = hasRoute ? `const data = await fetch${pascal}(input, signal);` : requestCall(tool); // An endpoint-backed tool scaffolds a call to its route. A standalone schema // tool has no route: the honest scaffold says "wire this to your app's own // action" and names nothing we made up. @@ -288,7 +307,10 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { ? [ ` // This tool is withheld: nothing registers it, so agents cannot see`, ` // or call it. To enable it, uncomment the request below and the`, - ` // registration above, and add callApi and toolResult to the import.`, + // Route-backed tools import callApi already (fetchX uses it). + hasRoute + ? ` // registration above, and add toolResult to the import.` + : ` // registration above, and add callApi and toolResult to the import.`, ] : [ ` // This tool starts disabled: it ${ @@ -297,7 +319,9 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { : `wraps an ${tool.endpointRole} endpoint` }. Agents can see it, and calling it tells`, ` // them it is disabled. To enable it, delete the line below, uncomment`, - ` // the code, and add callApi and toolResult to the import above.`, + hasRoute + ? ` // the code, and add toolResult to the import above.` + : ` // the code, and add callApi and toolResult to the import above.`, ]), ` void signal; // passed to fetch once you enable the call below`, ` return toolDisabled("${tool.name}.webmcp.ts");`, @@ -465,6 +489,30 @@ export function getModelContext(): ModelContext | null { return modelContext ?? null; } +/** + * Register every journey exported from the modules the barrel found in + * journeys/. Anything with a .register() method counts (createJourney's + * return shape); anything else is skipped quietly. One journey failing never + * takes the others down with it. + */ +export async function registerJourneys( + modules: Record[], + signal?: AbortSignal, +): Promise { + for (const module of modules) { + for (const value of Object.values(module)) { + const journey = value as { register?: unknown } | null; + if (journey !== null && typeof journey === "object" && typeof journey.register === "function") { + try { + await (journey.register as (signal?: AbortSignal) => Promise)(signal); + } catch (error) { + console.warn("[webmcp-codegen] a journey failed to register:", error); + } + } + } + } +} + /** * Call your API from the page. Same origin by default (pass a full URL when * the API lives on another host), always with the signed-in user's session @@ -575,12 +623,20 @@ export function requestUserConfirmation(message: string): Promise { } /** The barrel: one import that registers every generated tool. */ -export function barrelSource(tools: ReviewedTool[]): string { +export function barrelSource(tools: ReviewedTool[], journeyFiles: string[] = []): string { const imports = tools .map((tool) => `import { register${pascalCase(tool.name)} } from "./${tool.name}.webmcp";`) .join("\n"); const names = tools.map((tool) => `register${pascalCase(tool.name)}`).join(",\n "); + const journeyImports = journeyFiles + .map( + (file, index) => + `import * as journeyModule${index} from "./journeys/${file.replace(/\.ts$/, "")}";`, + ) + .join("\n"); + const journeyModuleNames = journeyFiles.map((_, index) => `journeyModule${index}`).join(", "); + return `/** * Generated by webmcp-codegen. This file is fully regenerated on every run. * Import registerAllTools() once at app startup: @@ -590,11 +646,11 @@ export function barrelSource(tools: ReviewedTool[]): string { */ ${imports} - +${journeyFiles.length > 0 ? `\nimport { registerJourneys } from "./runtime.webmcp";\n${journeyImports}\n` : ""} const registrations = [ ${names} ]; - +${journeyFiles.length > 0 ? `\nconst journeyModules = [${journeyModuleNames}];\n` : ""} /** * Register every generated tool with WebMCP. One tool failing (for example * because the page's Permissions-Policy disables tools) never takes the @@ -608,6 +664,13 @@ export async function registerAllTools(signal?: AbortSignal): Promise { console.warn("[webmcp-codegen] a tool failed to register:", error); } } + ${ + journeyFiles.length > 0 + ? `// Journeys come last: their steps compose the tools above. + await registerJourneys(journeyModules, signal);` + : `// Drop journey definitions into ./journeys/ and re-run \`generate\`: + // the next barrel registers every createJourney() export it finds there.` + } } `; } diff --git a/packages/codegen/src/outputs/tools.test.ts b/packages/codegen/src/outputs/tools.test.ts index 08672de..bda2123 100644 --- a/packages/codegen/src/outputs/tools.test.ts +++ b/packages/codegen/src/outputs/tools.test.ts @@ -58,6 +58,45 @@ describe("js generator", () => { await rm(cwd, { recursive: true, force: true }); }); + it("emits a live raw caller (fetchX) that the default execute composes", async () => { + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const tool = files.find((file) => file.path.includes("get-order-status")); + expect(tool?.contents).toContain("export async function fetchGetOrderStatus("); + expect(tool?.contents).toContain("const data = await fetchGetOrderStatus(input, signal);"); + }); + + it("scaffolds the journey factory and never reports it as an orphan", async () => { + // Simulate a previous run's factory sitting on disk. + await mkdir(join(cwd, "src/webmcp"), { recursive: true }); + await writeFile(join(cwd, "src/webmcp/journey.webmcp.ts"), "// previously scaffolded\n"); + + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const helper = files.find((file) => file.path.endsWith("webmcp/journey.webmcp.ts")); + expect(helper).toBeDefined(); + expect(helper?.contents).toContain("export function createJourney("); + + // Generator-owned, so the orphan report must leave it alone. + const barrel = files.find((file) => file.path.endsWith("webmcp/index.ts")); + expect((barrel?.notes ?? []).join("\n")).not.toContain("journey.webmcp.ts"); + }); + + it("registers journey files found in the journeys folder", async () => { + await mkdir(join(cwd, "src/webmcp/journeys"), { recursive: true }); + await writeFile( + join(cwd, "src/webmcp/journeys/document-trip.webmcp.ts"), + "// the user's journey definition\n", + ); + + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const barrel = files.find((file) => file.path.endsWith("webmcp/index.ts")); + expect(barrel?.contents).toContain( + 'import * as journeyModule0 from "./journeys/document-trip.webmcp";', + ); + expect(barrel?.contents).toContain("await registerJourneys(journeyModules, signal);"); + const runtime = files.find((file) => file.path.endsWith("webmcp/runtime.webmcp.ts")); + expect(runtime?.contents).toContain("export async function registerJourneys("); + }); + it("scaffolds the WebMCP skill file at the cross-client skills location", async () => { const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); const skill = files.find((file) => file.path.endsWith(".agents/skills/webmcp-tools/SKILL.md")); @@ -167,8 +206,11 @@ describe("js generator", () => { // Commented like an editor's toggle-comment: fixed marker column, original // indentation preserved, so uncommenting restores working code. expect(tool?.contents).toContain("// ...cancelOrderTool,"); - // The import reflects it: getModelContext is part of the fence, not the file. - expect(tool?.contents).toContain('import { toolDisabled } from "./runtime.webmcp";'); + // The import reflects it: getModelContext is part of the fence, not the + // file — but callApi stays live, because the raw caller (fetchX) the + // generated region emits is live too: journeys compose withheld tools. + expect(tool?.contents).toContain('import { callApi, toolDisabled } from "./runtime.webmcp";'); + expect(tool?.contents).toContain("export async function fetchCancelOrder("); // The execute scaffold still refuses politely if someone registers it by hand. expect(tool?.contents).toContain('return toolDisabled("cancel-order.webmcp.ts");'); // And the header says what "withheld" means. @@ -290,12 +332,11 @@ describe("js generator", () => { cwd, ); const tool = files.find((file) => file.path.includes("cancel-order")); - // Disabled notice first, the working call right below it, commented out. + // Disabled notice first, the working call right below it, commented out + // — and it composes the generated region's raw caller like every other + // endpoint-backed tool. expect(tool?.contents).toContain('return toolDisabled("cancel-order.webmcp.ts");'); - expect(tool?.contents).toContain( - // biome-ignore lint/suspicious/noTemplateCurlyInString: asserting on generated source, which contains a template literal - "// const data = await callApi(`/orders/${input.orderId}/cancel`", - ); + expect(tool?.contents).toContain("// const data = await fetchCancelOrder(input, signal);"); // The consent gate is generated, not left as a comment for humans to remember. expect(tool?.contents).toContain("requestUserConfirmation("); expect(tool?.contents).toContain("The user declined this action."); @@ -335,13 +376,14 @@ describe("js generator", () => { 'import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp";', ); - // The disabled tool's request is commented out, so its helpers stay out - // of the import line; the enable instructions name what to add back. + // The disabled tool's request is commented out, but its generated raw + // caller (fetchX) is live — so callApi is in the import line, and the + // enable instructions only name what's genuinely missing. const disabledWrite = files.find((file) => file.path.includes("cancel-order")); expect(disabledWrite?.contents).toContain( - 'import { getModelContext, requestUserConfirmation, asToolError, toolDisabled } from "./runtime.webmcp";', + 'import { getModelContext, requestUserConfirmation, callApi, asToolError, toolDisabled } from "./runtime.webmcp";', ); - expect(disabledWrite?.contents).toContain("add callApi and toolResult to the import above"); + expect(disabledWrite?.contents).toContain("add toolResult to the import above"); // A standalone schema tool has no route to call: no callApi. It is a // write, so the confirmation gate's helper is imported and used. diff --git a/packages/codegen/src/outputs/tools.ts b/packages/codegen/src/outputs/tools.ts index 5b6f003..7338462 100644 --- a/packages/codegen/src/outputs/tools.ts +++ b/packages/codegen/src/outputs/tools.ts @@ -79,6 +79,9 @@ export function tools(options: ToolsOutputOptions): Output { try { for (const entry of await readdir(outDir)) { if (!entry.endsWith(".webmcp.ts") || entry === "runtime.webmcp.ts") continue; + // The journey factory is generator-owned (regenerated wholesale + // below), never an orphan candidate. + if (entry === "journey.webmcp.ts") continue; const path = join(outDir, entry); existing.set(path, await readFile(path, "utf8")); } @@ -86,6 +89,18 @@ export function tools(options: ToolsOutputOptions): Output { // First run: the directory does not exist yet. } + // Journey definitions the user (or their agent) wrote since the last + // run. The barrel imports and registers them; dropping a new file in + // here and re-running generate is the whole wiring story. + let journeyFiles: string[] = []; + try { + journeyFiles = (await readdir(join(outDir, "journeys"))) + .filter((entry) => entry.endsWith(".webmcp.ts")) + .sort(); + } catch { + // No journeys directory yet — most repos, most of the time. + } + // Endpoint ref → the file currently holding it. const pathByRef = new Map(); for (const [path, contents] of existing) { @@ -164,10 +179,17 @@ export function tools(options: ToolsOutputOptions): Output { // The runtime and the barrel are regenerated wholesale every run; // their headers say "do not edit", and we mean it. Orphan reports ride // on the barrel so they surface even when nothing else changed. - const barrel = await plainFile(join(outDir, "index.ts"), barrelSource(tools)); + const barrel = await plainFile(join(outDir, "index.ts"), barrelSource(tools, journeyFiles)); barrel.notes = [...orphanNotes, ...(barrel.notes ?? [])]; files.unshift(await plainFile(join(outDir, "runtime.webmcp.ts"), runtimeSource()), barrel); + // The journey factory: fully ours, regenerated wholesale like the + // runtime. Journey definitions (the user's code) import createJourney + // from it. + files.push( + await plainFile(join(outDir, "journey.webmcp.ts"), await assetText("journey.webmcp.ts")), + ); + // The skill file: the rules harness for the user's own coding agents, // at the cross-client skills location. Regenerated wholesale like the // runtime — its header comment says why (project rules belong in the diff --git a/packages/codegen/src/verify.test.ts b/packages/codegen/src/verify.test.ts index ed6d384..f541a62 100644 --- a/packages/codegen/src/verify.test.ts +++ b/packages/codegen/src/verify.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "vitest"; import type { ReviewedTool } from "./types.js"; -import { verifyTools } from "./verify.js"; +import { verifyJourneyFiles, verifyTools } from "./verify.js"; function reviewedTool(overrides: Partial = {}): ReviewedTool { return { @@ -87,3 +87,95 @@ describe("verify description budgets", () => { expect(names?.findings[0]).toContain("the_unreasonably_long_parameter_name"); }); }); + +describe("verify journey files", () => { + const goodJourney = `import { createJourney } from "../journey.webmcp"; +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + "search-places": { + description: "Search real places and store the pick.", + input: { type: "object", properties: { input: { type: "string" } }, required: ["input"] }, + provides: ["locationObject"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft, + run: executeCreateTrip, + }, +});`; + + it("passes a well-formed journey", () => { + const checks = verifyJourneyFiles([ + { path: "journeys/document-trip.webmcp.ts", contents: goodJourney }, + ]); + expect(checks).toHaveLength(1); + expect(checks[0]?.level).toBe("ok"); + }); + + it("errors on a journeys file with no createJourney call", () => { + const checks = verifyJourneyFiles([ + { path: "journeys/random.webmcp.ts", contents: "export const x = 1;" }, + ]); + expect(checks[0]?.level).toBe("error"); + expect(checks[0]?.findings[0]).toContain("no createJourney() call"); + }); + + it("errors on a journey with no submit gate", () => { + const checks = verifyJourneyFiles([ + { + path: "journeys/broken.webmcp.ts", + contents: `createJourney({ name: "x", goal: "y", steps: { "a": { description: "Do a.", input: {}, provides: [] } } });`, + }, + ]); + expect(checks.some((check) => check.findings[0]?.includes("no submit gate"))).toBe(true); + }); + + it("errors on over-budget descriptions", () => { + const checks = verifyJourneyFiles([ + { + path: "journeys/wordy.webmcp.ts", + contents: goodJourney.replace( + "Search real places and store the pick.", + `Search. ${"So very much detail about searching. ".repeat(15)}`, + ), + }, + ]); + const budget = checks.find( + (check) => check.level === "error" && check.summary.includes("budget"), + ); + expect(budget?.findings[0]).toContain("500"); + }); + + it("warns on more than five steps", () => { + const steps = Array.from( + { length: 6 }, + (_, i) => + `"step-${i}": { description: "Do thing ${i}.", input: { type: "object" }, provides: ["f${i}"] },`, + ).join("\n "); + const checks = verifyJourneyFiles([ + { + path: "journeys/epic.webmcp.ts", + contents: `createJourney({ name: "epic", goal: "g", steps: {\n ${steps}\n }, submit: { description: "Go.", build: (d) => d, run: x } });`, + }, + ]); + const warning = checks.find((check) => check.level === "warning"); + expect(warning?.findings[0]).toContain("6 steps"); + }); + + it("warns when a journey calls fetch directly", () => { + const checks = verifyJourneyFiles([ + { + path: "journeys/raw.webmcp.ts", + contents: goodJourney.replace( + 'provides: ["locationObject"],', + 'provides: ["locationObject"],\n run: async (input) => { const r = await fetch("/v1/places"); return { locationObject: r }; },', + ), + }, + ]); + const warning = checks.find((check) => check.level === "warning"); + expect(warning?.findings[0]).toContain("fetch/callApi directly"); + }); +}); diff --git a/packages/codegen/src/verify.ts b/packages/codegen/src/verify.ts index fc9a0ff..bf47550 100644 --- a/packages/codegen/src/verify.ts +++ b/packages/codegen/src/verify.ts @@ -292,6 +292,152 @@ export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { return checks; } +/** + * The top-level keys of an inline object literal's `steps: { ... }` block, + * found by brace matching rather than parsing (journey files are the user's + * TypeScript; a full parse is out of scope for a lint). Used to count steps. + */ +function journeyStepNames(contents: string): string[] { + const start = /steps\s*:\s*\{/.exec(contents); + if (!start) return []; + let depth = 0; + let bodyStart = -1; + let bodyEnd = -1; + for (let i = start.index + start[0].length - 1; i < contents.length; i++) { + const char = contents[i]; + if (char === "{") { + if (depth === 0) bodyStart = i + 1; + depth++; + } else if (char === "}") { + depth--; + if (depth === 0) { + bodyEnd = i; + break; + } + } + } + if (bodyStart === -1 || bodyEnd === -1) return []; + const body = contents.slice(bodyStart, bodyEnd); + // Keys at depth one: "search-places": { ... }. The sticky regex anchors at + // the first non-space character after the walk position, and the walk + // skips past each match, so a key is counted exactly once. + const names: string[] = []; + const keyPattern = /"([^"]+)"\s*:\s*\{/y; + let innerDepth = 0; + for (let i = 0; i < body.length; i++) { + const char = body[i]; + if (char === "{") { + innerDepth++; + continue; + } + if (char === "}") { + innerDepth--; + continue; + } + if (innerDepth !== 0) continue; + let cursor = i; + while (cursor < body.length && /\s/.test(body[cursor] ?? "")) cursor++; + keyPattern.lastIndex = cursor; + const keyMatch = keyPattern.exec(body); + if (keyMatch) { + names.push(keyMatch[1] ?? ""); + // The match consumed the step's opening brace — count it, since the + // walk skips past the whole match including that brace. + innerDepth++; + i = cursor + keyMatch[0].length - 1; + } + } + return names; +} + +/** A journey definition file from the tools directory's journeys/ folder. */ +export interface JourneyFileInput { + path: string; + contents: string; +} + +/** + * The journey checks, over the user's journey files (not the generated + * tools). Structural problems (no createJourney, no submit gate, missing + * run) and over-budget descriptions are errors; step count and bypassing + * the generated callers are warnings. Journey files are the user's agent's + * code, so findings name the fix, not just the smell. + */ +export function verifyJourneyFiles(files: JourneyFileInput[]): VerifyCheck[] { + const structural: string[] = []; + const budget: string[] = []; + const warnings: string[] = []; + + for (const { path, contents } of files) { + if (!/createJourney\s*\(/.test(contents)) { + structural.push( + `${path} — no createJourney() call; files in journeys/ must define a journey.`, + ); + continue; + } + if (!/submit\s*:/.test(contents) || !/run\s*:/.test(contents)) { + structural.push( + `${path} — no submit gate with a run; a journey without one is just loose tools.`, + ); + } + const steps = journeyStepNames(contents); + if (steps.length > 5) { + warnings.push( + `${path} — ${steps.length} steps; past five, agents lose the thread. Split it into two journeys.`, + ); + } + for (const match of contents.matchAll(/description\s*:\s*"((?:[^"\\]|\\.)*)"/g)) { + const text = match[1] ?? ""; + if (text.length > TOOL_DESCRIPTION_MAX) { + budget.push( + `${path} — a description runs ${text.length} characters (max ${TOOL_DESCRIPTION_MAX}); tighten it.`, + ); + } + } + const withoutImports = contents.replace(/^\s*import\s.*$/gm, ""); + if (/\b(?:fetch|callApi)\s*\(/.test(withoutImports)) { + warnings.push( + `${path} — calls fetch/callApi directly; use the generated raw callers (fetchX) or a tool's execute, so the contract lives in one place.`, + ); + } + } + + const checks: VerifyCheck[] = []; + if (structural.length > 0) { + checks.push({ + area: "Journeys", + summary: `${structural.length} structural problem${structural.length === 1 ? "" : "s"}`, + findings: structural, + level: "error", + }); + } + if (budget.length > 0) { + checks.push({ + area: "Journeys", + summary: `${budget.length} over budget`, + findings: budget, + level: "error", + }); + } + if (warnings.length > 0) { + checks.push({ + area: "Journeys", + summary: `${warnings.length} warning${warnings.length === 1 ? "" : "s"}`, + findings: warnings, + level: "warning", + }); + } + if (checks.length === 0 && files.length > 0) { + checks.push({ + area: "Journeys", + summary: `${files.length} journey file${files.length === 1 ? "" : "s"}, gates and budgets in order`, + findings: [], + level: "ok", + }); + } + return checks; +} + /** The --url probe: is the page actually live for a visitor's browser? */ export async function verifyUrl(url: string): Promise { const findings: AuditFinding[] = []; diff --git a/site/content/docs/journeys.mdx b/site/content/docs/journeys.mdx index 62ab493..3cd940e 100644 --- a/site/content/docs/journeys.mdx +++ b/site/content/docs/journeys.mdx @@ -77,8 +77,10 @@ Codegen ships the machinery, never the journeys: every `generate`, next to `runtime.webmcp.ts`. It's generator-owned: don't hand-edit it; regeneration overwrites it. The draft, the step registration, the gate, the confirmation — all of it lives here, once, for every journey. -- **Registration** — the generated `index.ts` auto-imports everything in your - `journeys/` folder and registers it. Drop in a new journey file, it's live. +- **Registration** — the generated `index.ts` auto-imports everything in the + `journeys/` folder inside your tools directory (e.g. + `src/webmcp/journeys/`) and registers it. Drop in a journey file, re-run + `generate`, it's live. - **Checks** — `verify` validates journey files: submit gate present, steps described within budget, step count small enough for an agent to track, and no direct `fetch` bypassing your generated tools. @@ -89,8 +91,8 @@ knows the flow. So you write one small file per journey (or your coding agent does, guided by the bundled skill file): ```ts -// journeys/document-trip.webmcp.ts -import { createJourney } from "../webmcp/journey.webmcp"; +// src/webmcp/journeys/document-trip.webmcp.ts +import { createJourney } from "../journey.webmcp"; import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; From 5272d59cf196ee53c4591f30154e135736d9fd1f Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 03:17:42 +0530 Subject: [PATCH 11/41] Add the skill-file eval harness: fixture, cases, deterministic graders MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit evals/skill/: a real generated surface (beenthere-lite, skills dir installed), nine cases across core/negative/control kinds, and a runner that grades files, not transcripts — marker preservation byte-for-byte, budget measurements, regex allow/forbid lists, tree diffs. The sharpest case is journey-document-trip: retrospective naming passes, plan-trip is an automatic fail; its skill-removed twin detects model absorption. Self-test (node run.mjs --selftest) verifies every grader in both directions and passes. Running against real agents requires an agent CLI via AGENT_CMD; results stay local. --- packages/codegen/evals/skill/README.md | 75 ++++ packages/codegen/evals/skill/cases.json | 123 ++++++ .../.agents/skills/webmcp-tools/SKILL.md | 128 ++++++ .../codegen/evals/skill/fixture/README-APP.md | 13 + .../codegen/evals/skill/fixture/README.md | 8 + .../codegen/evals/skill/fixture/spec.json | 113 +++++ .../fixture/src/webmcp/create-trip.webmcp.ts | 167 ++++++++ .../src/webmcp/get-autocomplete.webmcp.ts | 96 +++++ .../evals/skill/fixture/src/webmcp/index.ts | 34 ++ .../fixture/src/webmcp/journey.webmcp.ts | 215 ++++++++++ .../fixture/src/webmcp/list-trips.webmcp.ts | 89 ++++ .../fixture/src/webmcp/runtime.webmcp.ts | 191 +++++++++ .../codegen/evals/skill/results/.gitignore | 3 + packages/codegen/evals/skill/run.mjs | 398 ++++++++++++++++++ 14 files changed, 1653 insertions(+) create mode 100644 packages/codegen/evals/skill/README.md create mode 100644 packages/codegen/evals/skill/cases.json create mode 100644 packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md create mode 100644 packages/codegen/evals/skill/fixture/README-APP.md create mode 100644 packages/codegen/evals/skill/fixture/README.md create mode 100644 packages/codegen/evals/skill/fixture/spec.json create mode 100644 packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts create mode 100644 packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts create mode 100644 packages/codegen/evals/skill/fixture/src/webmcp/index.ts create mode 100644 packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts create mode 100644 packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts create mode 100644 packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts create mode 100644 packages/codegen/evals/skill/results/.gitignore create mode 100644 packages/codegen/evals/skill/run.mjs diff --git a/packages/codegen/evals/skill/README.md b/packages/codegen/evals/skill/README.md new file mode 100644 index 0000000..b5419fa --- /dev/null +++ b/packages/codegen/evals/skill/README.md @@ -0,0 +1,75 @@ +# Skill-file evals + +The skill file (`assets/skill/SKILL.md`) is treated like code: a wording +change that makes agents write worse WebMCP tools should show up as a failed +test, not a vibe. This directory is that test rig, following the OpenAI +eval-skills writeup and philschmid's testing-skills post: +**prompt → captured run → checks → score, over multiple trials.** + +## Layout + +- `fixture/` — beenthere-lite, a real generated surface (place search, trip + list, withheld trip creation) with the skill file installed at + `.agents/skills/webmcp-tools/`. Every case starts from a fresh copy. +- `cases.json` — the prompt set. `core` cases gate; `negative` cases prove + the skill doesn't leak into unrelated work; `control` cases are + informational only. +- `run.mjs` — the harness and its deterministic graders. +- `results/` — timestamped JSON reports (gitignored). + +## Running + +You need an agent CLI on PATH (Codex, Claude Code, …). The command template +is `AGENT_CMD`, with `{PROMPT}` substituted and shell-quoted: + +```sh +AGENT_CMD='codex exec --json "{PROMPT}"' node run.mjs +AGENT_CMD='claude -p "{PROMPT}" --output-format json' node run.mjs --trials 5 +node run.mjs --case journey-document-trip # one case +node run.mjs --selftest # grade the graders, no agent +``` + +Exit code is 0 only when every gated case passes every trial. Control cases +never gate. + +## The sharpest case + +`journey-document-trip`: the prompt asks for a flow around trip creation on +a *retrospective* (been-there) product. The right answer composes the +generated tools into `createJourney(...)`, names it like `document-trip`, +and gates creation behind the human. `plan-trip` / `book-trip` match a +forbidden pattern — that naming failure passes every mechanical lint yet is +semantically wrong, which is exactly the class of mistake the skill file +exists to prevent. + +Its twin, `journey-without-skill`, runs the same prompt with the skill +directory deleted. It doesn't gate: it's the absorption detector. If it +starts passing reliably across models, the models internalized the practices +and the skill can retire. + +## Deterministic checks, LLM judgment only via the agent under test + +Graders look at files, not transcripts: expected files satisfying +include/exclude regexes, the generated region's marker preserved byte-for-byte, +description budgets measured, the tree unchanged on unrelated prompts. Adding +an LLM-as-judge pass is possible later — constrain it to a structured schema +(`overall_pass`, per-check results) so scores diff across runs — but the +fixtures were chosen so regex + structure carry the verdict. + +## The operating loop + +1. A real failure — from a user report, a docs change, a model regression — + becomes a case here first. +2. Tune the skill until the case is at ~100% pass rate across trials. +3. The case joins the regression set permanently. +4. Run the suite on skill edits and on new flagship models; run the control + case occasionally to detect absorption. + +Results stay local under `results/` by design: this harness needs an agent +CLI and model access, so it runs on demand, not in CI. + +## Refreshing the fixture + +The fixture is committed generated output (see `fixture/README.md`). When +the generator's templates change, regenerate it so the eval baseline is what +users actually receive. diff --git a/packages/codegen/evals/skill/cases.json b/packages/codegen/evals/skill/cases.json new file mode 100644 index 0000000..b198ba4 --- /dev/null +++ b/packages/codegen/evals/skill/cases.json @@ -0,0 +1,123 @@ +{ + "version": 1, + "comment": "The skill-file eval set. Each case: a prompt, an agent run against a fresh copy of fixture/, deterministic checks on the result. See README.md.", + "agent_cmd_default": "codex exec --json \"{PROMPT}\"", + "trials_default": 3, + "cases": [ + { + "id": "trigger-new-read-tool", + "kind": "core", + "prompt": "Add a webmcp tool that lets agents look up the current weather for a city.", + "checks": [ + { + "type": "any-file-matches", + "dir": "src/webmcp", + "all": [ + "name: \"(get|search|check|lookup)-[a-z-]*weather[a-z-]*\"", + "readOnlyHint: true", + "registerTool\\(" + ], + "none": ["\\bfetch\\s*\\(\\s*[\"'`]https?://(?!/v1/)"] + } + ] + }, + { + "id": "journey-document-trip", + "kind": "core", + "comment": "The sharpest fixture: retrospective naming is right, plan/book is auto-fail.", + "prompt": "Look at the webmcp tools in this repo. Creating a trip needs a resolved locationObject from the place search, and the human must confirm before anything is created. Write the journey for this flow.", + "checks": [ + { + "type": "any-file-matches", + "dir": "src/webmcp/journeys", + "all": ["createJourney\\(", "submit", "\"(document|record|log|write|capture)-trip\""], + "none": ["\"(plan|book)-trip\"", "\\bfetch\\s*\\("] + } + ] + }, + { + "id": "enable-withheld-write", + "kind": "core", + "prompt": "Enable the create-trip tool so agents can use it.", + "checks": [ + { "type": "marker-preserved", "path": "src/webmcp/create-trip.webmcp.ts" }, + { + "type": "file-matches", + "path": "src/webmcp/create-trip.webmcp.ts", + "all": ["\\n await modelContext\\.registerTool\\(", "requestUserConfirmation"], + "none": [] + } + ] + }, + { + "id": "improve-description", + "kind": "core", + "comment": "Descriptions live in the generated region (do-not-edit) or the overrides file — the skill teaches overrides, so the marker must survive either way.", + "prompt": "The create-trip description doesn't tell agents what comes back. Improve it.", + "checks": [ + { + "type": "any-of", + "checks": [ + { + "type": "file-matches", + "path": ".webmcp-codegen.json", + "all": ["create-trip", "description"], + "none": [] + }, + { "type": "marker-preserved", "path": "src/webmcp/create-trip.webmcp.ts" } + ] + } + ] + }, + { + "id": "lazy-prompt-quality", + "kind": "core", + "prompt": "make my webmcp tools good", + "checks": [ + { + "type": "tool-description-budget", + "dir": "src/webmcp", + "max_tool": 500, + "max_param": 150 + }, + { "type": "marker-preserved", "path": "src/webmcp/create-trip.webmcp.ts" } + ] + }, + { + "id": "negative-typo", + "kind": "negative", + "prompt": "Fix the typo in README-APP.md", + "checks": [ + { "type": "file-matches", "path": "README-APP.md", "all": [], "none": ["\\bteh\\b"] }, + { "type": "no-webmcp-diff" } + ] + }, + { + "id": "negative-unit-test", + "kind": "negative", + "prompt": "Write a small unit test for a date-formatting helper (tests/format.test.ts).", + "checks": [{ "type": "no-webmcp-diff" }] + }, + { + "id": "negative-dark-mode", + "kind": "negative", + "prompt": "Add a dark-mode toggle to the settings page.", + "checks": [{ "type": "no-webmcp-diff" }] + }, + { + "id": "journey-without-skill", + "kind": "control", + "comment": "Same prompt as journey-document-trip with the skill REMOVED. Informational: if this starts passing reliably, models absorbed the practices and the skill can retire.", + "skill": false, + "prompt": "Look at the webmcp tools in this repo. Creating a trip needs a resolved locationObject from the place search, and the human must confirm before anything is created. Write the journey for this flow.", + "checks": [ + { + "type": "any-file-matches", + "dir": "src/webmcp/journeys", + "all": ["createJourney\\(", "submit", "\"(document|record|log|write|capture)-trip\""], + "none": ["\"(plan|book)-trip\"", "\\bfetch\\s*\\("] + } + ] + } + ] +} diff --git a/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md b/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md new file mode 100644 index 0000000..b2b496d --- /dev/null +++ b/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md @@ -0,0 +1,128 @@ +--- +name: webmcp-tools +description: Build, improve, or review this repo's WebMCP tools and journeys — the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. +--- + + + +# WebMCP tools in this repo + +WebMCP tools run in the visitor's browser, with the signed-in user's session, +while the agent calling them may also be reading attacker-influenced page +content. Every tool you write is both an API and a security surface. The rules +below exist so an agent-facing tool is correct by default — follow them even +when the user's request is casual. + +## Naming + +- Verb-first, intent-shaped, max 30 characters: `list-trips`, + `add-bucket-list-destination`. Never method-first (`get-v1-trips`), never + numbered (`get-pricing-2`). +- Name the user's intent, not the endpoint. `POST /search` is a read named + `search-...`; `POST /orders/{id}/cancel` is destructive named `cancel-...`. +- **Understand the product before naming anything.** What is it, who uses it, + and when in the user's life does this action happen? If that is not written + down in the repo, ask the user before naming intent-level tools or journeys. + Wrong-tense or wrong-role names pass every mechanical check and are still + wrong: on a journal for trips you've *been on*, the flow is `document-trip`, + never `plan-trip`. This is the failure no linter can catch — it is your job. + +## Descriptions + +- Say what the tool does, when to use it, and what it returns: + "Create a new trip. Returns the trip." +- Max 500 characters for the tool, max 150 per parameter. Turn constraints + into sentences: "A number from 30 to 600." +- Never instruct the agent or encode flow control in a description + ("always call X first") — that is steering. Prerequisites belong in a + journey, not in prose. +- If a field's value can only come from another tool (a resolved place object, + a server id), say so in that field's description. + +## Safety and exposure + +- Reads are registered immediately. Writes and destructive tools stay + withheld — generated but not registered — until the user deliberately + enables one. Never enable a write tool without being asked. +- Mutating tools confirm each call with the human via + `requestUserConfirmation`. That call lives in the generated region; never + move or remove it. +- Free-text outputs get `untrustedContentHint: true` — the agent must not + treat user-written content as the site speaking. +- **The schema is not the security boundary.** `execute` must call the app's + real endpoint or action layer, so server-side validation runs on every + call. Never wire `execute` to return canned data or bypass the app's own + flow (cache invalidation, navigation, stores). + +## The execute contract + +- Never throw for failure. The browser maps a rejected `execute` to a bare + `UnknownError` and discards your message. Return `toolError(message)` / + `asToolError(error)` so the agent can read and recover. (Cancellation is + the one exception: let `AbortError` propagate.) +- Return via `toolResult(data)` and keep outputs under ~1.5K characters — + summarize or paginate rather than dumping. +- When a call changes what is on screen, make it visible: navigate, + invalidate a query, dispatch an event. The human is watching the page. + +## Editing generated files + +- Each `*.webmcp.ts` has a generated region between the + `webmcp-codegen` markers — never edit inside it; regeneration rewrites it. + Your work goes below the marker (the `execute` body) or in + `.webmcp-codegen.json` (description/name/enabled overrides, which survive + regeneration and always win over generated text). +- After editing tools, run `npx @webmcp-stack/codegen verify` and fix what it + reports. + +## Journeys (multi-step flows) + +Reach for a journey when a goal takes several calls with shared state, when +an input can't be invented (a resolved place object), or when a spend should +be gated (an eligibility check before a paid generation). Journey files live +in the `journeys/` folder inside the generated tools directory +(`src/webmcp/journeys/document-trip.webmcp.ts`); re-run `verify` after +writing one. Pattern: + +```ts +// src/webmcp/journeys/document-trip.webmcp.ts +import { createJourney } from "../journey.webmcp"; +import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; +import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; + +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + // Tool-backed step: inherits the generated tool's description and schema; + // you write only what lands in the draft. + "search-places": { + tool: getAutocompleteTool, + call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), + store: (places) => ({ locationObject: places }), // store the resolved pick + provides: ["locationObject"], + }, + // Freeform step: no backend call — collects input straight into the draft. + "set-details": { + description: "Set the trip's title and dates.", + input: { type: "object", properties: { title: { type: "string" } }, required: ["title"] }, + provides: ["title"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft as CreateTripInput, // assemble the real tool's input + run: executeCreateTrip, // the existing tool does the work + }, +}); +``` + +- 2–5 steps. More means two journeys. +- Tool-backed steps reuse the generated tool's contract and its raw caller + (`fetchX`); the submit's `run` is the real write tool's `execute`, so its + confirmation and validation still apply. Never write a direct `fetch` in a + journey file — `verify` flags it. +- The submit gate is the only write in a journey; step tools are reads or + draft-writes and stay read-only. diff --git a/packages/codegen/evals/skill/fixture/README-APP.md b/packages/codegen/evals/skill/fixture/README-APP.md new file mode 100644 index 0000000..c1ea7ac --- /dev/null +++ b/packages/codegen/evals/skill/fixture/README-APP.md @@ -0,0 +1,13 @@ +# beenthere-lite + +A journal for trips you've been on. Document the places, write the story, +share the memory. (Not a planner: users come here *after* the trip.) + +## Dev + +The stack is a small web app with an API at the same origin (`/v1/...`) and +WebMCP tools under `src/webmcp/`, generated from `spec.json` with +webmcp-codegen. Tools register on page load via `registerAllTools()` from +`src/webmcp/index.ts`. + +teh editor opens after a trip is created. diff --git a/packages/codegen/evals/skill/fixture/README.md b/packages/codegen/evals/skill/fixture/README.md new file mode 100644 index 0000000..868ca24 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/README.md @@ -0,0 +1,8 @@ +# beenthere-lite (eval fixture) + +A memory-journal app for trips you've *been on* — deliberately NOT a travel +planner. Generated by `@webmcp-stack/codegen generate --spec spec.json` and +committed as the baseline every eval case starts from. + +Refresh after template changes: re-run the command above into a scratch dir +and copy the result over this folder. diff --git a/packages/codegen/evals/skill/fixture/spec.json b/packages/codegen/evals/skill/fixture/spec.json new file mode 100644 index 0000000..704608c --- /dev/null +++ b/packages/codegen/evals/skill/fixture/spec.json @@ -0,0 +1,113 @@ +{ + "openapi": "3.0.0", + "info": { "title": "beenthere-lite", "version": "1.0.0" }, + "paths": { + "/v1/places/autocomplete": { + "get": { + "operationId": "getAutocomplete", + "summary": "Search places by free text.", + "parameters": [ + { + "name": "input", + "in": "query", + "required": true, + "schema": { "type": "string", "description": "The place search text, e.g. Lisbon." } + } + ], + "responses": { + "200": { + "description": "ok", + "content": { + "application/json": { + "schema": { + "type": "array", + "items": { + "type": "object", + "properties": { + "placeId": { "type": "string" }, + "fullAddress": { "type": "string" }, + "types": { "type": "array", "items": { "type": "string" } }, + "country": { "type": "string" }, + "lat": { "type": "number" }, + "lng": { "type": "number" }, + "name": { "type": "string" } + }, + "required": ["placeId", "fullAddress", "types", "country", "lat", "lng", "name"] + } + } + } + } + } + } + } + }, + "/v1/trips/": { + "get": { + "operationId": "listTrips", + "summary": "List the signed-in user's trips.", + "responses": { + "200": { + "description": "ok", + "content": { + "application/json": { + "schema": { + "type": "array", + "items": { + "type": "object", + "properties": { "id": { "type": "string" }, "title": { "type": "string" } } + } + } + } + } + } + } + }, + "post": { + "operationId": "createTrip", + "summary": "Create a new trip.", + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "title": { "type": "string", "description": "The trip's display title." }, + "startDate": { "type": "string", "format": "date" }, + "locationObject": { + "type": "object", + "description": "The resolved place from the autocomplete endpoint.", + "properties": { + "placeId": { "type": "string" }, + "fullAddress": { "type": "string" }, + "types": { "type": "array", "items": { "type": "string" } }, + "country": { "type": "string" }, + "lat": { "type": "number" }, + "lng": { "type": "number" }, + "name": { "type": "string" } + }, + "required": ["placeId", "fullAddress", "types", "country", "lat", "lng", "name"] + } + }, + "required": ["title", "locationObject"] + } + } + } + }, + "responses": { + "200": { + "description": "ok", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { "id": { "type": "string" }, "title": { "type": "string" } } + } + } + } + } + } + } + } + } +} diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts new file mode 100644 index 0000000..7811276 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts @@ -0,0 +1,167 @@ +import { callApi, toolDisabled } from "./runtime.webmcp"; + +// ─── webmcp-codegen: generated. Do not edit this region. ─── +/** + * Create a new trip. Returns the trip. + * + * Source: POST /v1/trips/ (openapi). Risk: write-confirm. + * Starts withheld: not registered until you enable it (see registerCreateTrip below). + * Regenerate with: npx @webmcp-stack/codegen generate + */ + +/** The exact contract advertised to the agent. Derived from the API spec. Do not hand-edit. */ +export const createTripInputSchema = { + "type": "object", + "properties": { + "title": { + "type": "string", + "description": "The trip's display title." + }, + "startDate": { + "type": "string", + "format": "date", + "description": "Start date. A date (YYYY-MM-DD)." + }, + "locationObject": { + "type": "object", + "description": "The resolved place from the autocomplete endpoint.", + "properties": { + "placeId": { + "type": "string", + "description": "The unique identifier of the place." + }, + "fullAddress": { + "type": "string", + "description": "Full address." + }, + "types": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Types." + }, + "country": { + "type": "string", + "description": "Country." + }, + "lat": { + "type": "number", + "description": "Lat." + }, + "lng": { + "type": "number", + "description": "Lng." + }, + "name": { + "type": "string", + "description": "Name." + } + }, + "required": [ + "placeId", + "fullAddress", + "types", + "country", + "lat", + "lng", + "name" + ] + } + }, + "required": [ + "title", + "locationObject" + ] +}; + +/** What `execute` receives. The browser validates agent input against the schema above. */ +export type CreateTripInput = { "title": string; "startDate"?: string; "locationObject": { "placeId": string; "fullAddress": string; "types": string[]; "country": string; "lat": number; "lng": number; "name": string } }; + +/** Safety hints computed by webmcp-codegen. Informational metadata for hosts and UIs. */ +export const createTripHints = {"readOnlyHint":false,"destructiveHint":false,"idempotentHint":false,"untrustedContentHint":true} as const; + +/** The tool definition, minus `execute` (which is yours, below the marker). */ +export const createTripTool = { + name: "create-trip", + title: "Create Trip", + description: "Create a new trip. Returns the trip.", + inputSchema: createTripInputSchema, + annotations: { + readOnlyHint: false, + untrustedContentHint: true, + consequentialHint: false, + }, +}; + +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeCreateTrip is the agent-facing one. */ +export async function fetchCreateTrip(input: CreateTripInput, signal?: AbortSignal) { + const data = await callApi("/v1/trips/", { method: "POST", body: { title: input.title, startDate: input.startDate, locationObject: input.locationObject }, signal }); + return data; +} + +/** + * Withheld: this tool is not registered, so agents cannot see or pick + * it. The registration below stays commented until you enable the tool + * (uncomment it and the body of executeCreateTrip, or flip it in the + * dashboard and regenerate). + */ +export async function registerCreateTrip(signal?: AbortSignal): Promise { + void signal; + // const modelContext = getModelContext(); + // if (!modelContext) return; + // await modelContext.registerTool( + // { + // ...createTripTool, + // execute: async (input, context) => { + // // Cancellation wins over everything, including the confirmation. + // context?.signal?.throwIfAborted(); + // // This tool changes things, so the user is always asked first. The + // // confirmation lives in the generated region: it cannot be edited away. + // const confirmed = await requestUserConfirmation( + // "Allow the agent to: Create a new trip. Returns the trip.", + // ); + // if (!confirmed) { + // return { + // content: [{ type: "text", text: "The user declined this action." }], + // isError: true, + // }; + // } + // // The browser has already validated the agent's input against the schema. + // // A failure returns a readable result; it never throws (see asToolError). + // try { + // return await executeCreateTrip(input as CreateTripInput, context?.signal); + // } catch (error) { + // return asToolError(error); + // } + // }, + // }, + // { signal }, + // ); +} + +// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── + +/** + * What actually happens when the agent calls "create-trip". + * + * Default implementation: calls POST /v1/trips/ from this page, with the + * signed-in user's session. Replace it with your app's own API client + * whenever you like; the contract above never changes. + * + * Calls the API on this page's own origin (same-origin by default). + * + * This tool is write-confirm: it changes things. + * The user is asked to confirm every call (built into the generated region). + */ +export async function executeCreateTrip(input: CreateTripInput, signal?: AbortSignal) { + // This tool is withheld: nothing registers it, so agents cannot see + // or call it. To enable it, uncomment the request below and the + // registration above, and add toolResult to the import. + void signal; // passed to fetch once you enable the call below + return toolDisabled("create-trip.webmcp.ts"); + + // const data = await fetchCreateTrip(input, signal); + // return toolResult(data); +} \ No newline at end of file diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts new file mode 100644 index 0000000..f2f7df7 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts @@ -0,0 +1,96 @@ +import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; + +// ─── webmcp-codegen: generated. Do not edit this region. ─── +/** + * Search places by free text. Returns an array of autocomplete. + * + * Source: GET /v1/places/autocomplete (openapi). Risk: safe-read. + * Starts enabled (see executeGetAutocomplete below). + * Regenerate with: npx @webmcp-stack/codegen generate + */ + +/** The exact contract advertised to the agent. Derived from the API spec. Do not hand-edit. */ +export const getAutocompleteInputSchema = { + "type": "object", + "properties": { + "input": { + "type": "string", + "description": "The place search text, e.g. Lisbon." + } + }, + "required": [ + "input" + ] +}; + +/** What `execute` receives. The browser validates agent input against the schema above. */ +export type GetAutocompleteInput = { "input": string }; + +/** Safety hints computed by webmcp-codegen. Informational metadata for hosts and UIs. */ +export const getAutocompleteHints = {"readOnlyHint":true,"destructiveHint":false,"idempotentHint":true,"untrustedContentHint":true} as const; + +/** The tool definition, minus `execute` (which is yours, below the marker). */ +export const getAutocompleteTool = { + name: "get-autocomplete", + title: "Get Autocomplete", + description: "Search places by free text. Returns an array of autocomplete.", + inputSchema: getAutocompleteInputSchema, + annotations: { + readOnlyHint: true, + untrustedContentHint: true, + consequentialHint: false, + }, +}; + +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeGetAutocomplete is the agent-facing one. */ +export async function fetchGetAutocomplete(input: GetAutocompleteInput, signal?: AbortSignal) { + const data = await callApi("/v1/places/autocomplete", { method: "GET", query: { input: input.input }, signal }); + return data; +} + +/** + * Register this tool with WebMCP. Call it once on page load, or use + * registerAllTools() from the generated index.ts. Skips quietly when the + * browser has no WebMCP runtime. + * + * Pass an AbortSignal to unregister later: controller.abort(). + */ +export async function registerGetAutocomplete(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); + if (!modelContext) return; + await modelContext.registerTool( + { + ...getAutocompleteTool, + // The browser has already validated the agent's input against the schema. + // A failure returns a readable result; it never throws (see asToolError). + execute: async (input, context) => { + try { + return await executeGetAutocomplete(input as GetAutocompleteInput, context?.signal); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); +} + +// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── + +/** + * What actually happens when the agent calls "get-autocomplete". + * + * Default implementation: calls GET /v1/places/autocomplete from this page, with the + * signed-in user's session. Replace it with your app's own API client + * whenever you like; the contract above never changes. + * + * Calls the API on this page's own origin (same-origin by default). + */ +export async function executeGetAutocomplete(input: GetAutocompleteInput, signal?: AbortSignal) { + const data = await fetchGetAutocomplete(input, signal); + // Make the effect visible: an agent acts while a human watches this + // page. If this call changes what is on screen, update the UI here + // (navigate, invalidate a query, dispatch an event). + return toolResult(data); +} \ No newline at end of file diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/index.ts b/packages/codegen/evals/skill/fixture/src/webmcp/index.ts new file mode 100644 index 0000000..5d6b5bb --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/index.ts @@ -0,0 +1,34 @@ +/** + * Generated by webmcp-codegen. This file is fully regenerated on every run. + * Import registerAllTools() once at app startup: + * + * import { registerAllTools } from "./webmcp"; + * await registerAllTools(); + */ + +import { registerGetAutocomplete } from "./get-autocomplete.webmcp"; +import { registerListTrips } from "./list-trips.webmcp"; +import { registerCreateTrip } from "./create-trip.webmcp"; + +const registrations = [ + registerGetAutocomplete, + registerListTrips, + registerCreateTrip +]; + +/** + * Register every generated tool with WebMCP. One tool failing (for example + * because the page's Permissions-Policy disables tools) never takes the + * others down with it. The failure is logged and registration continues. + */ +export async function registerAllTools(signal?: AbortSignal): Promise { + for (const register of registrations) { + try { + await register(signal); + } catch (error) { + console.warn("[webmcp-codegen] a tool failed to register:", error); + } + } + // Drop journey definitions into ./journeys/ and re-run `generate`: + // the next barrel registers every createJourney() export it finds there. +} diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts new file mode 100644 index 0000000..93a777d --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts @@ -0,0 +1,215 @@ +/** + * Written by webmcp-codegen on every `generate` run. Do not edit by hand; + * your changes will be lost. This file is fully ours — journey definitions + * (your code) live in journeys/*.webmcp.ts and import createJourney from here. + * + * createJourney: multi-step agent flows with a shared draft and one submit. + * + * The three pieces, literally: + * + * 1. THE DRAFT — one plain object per page load (`let draft = {}` below). + * Nothing fancier: steps write their results into it, the submit reads + * from it. It dies with the page; a half-finished journey does not + * survive a reload, which is what you want. + * + * 2. STEP TOOLS — ordinary registered WebMCP tools, one per step, named + * "-" (e.g. "document-trip-search-places"). A step's + * execute stores what it produced into the draft, then replies with what + * is still missing, so the agent always knows the next move. + * + * 3. THE SUBMIT GATE — one more registered tool, "-submit". Its + * execute, in order: refuses with the list of missing steps, asks the + * human to confirm, runs the real tool's execute with the assembled + * input, clears the draft. There is no way to submit around it, because + * the real input only exists inside build(draft). + */ + +import { + asToolError, + getModelContext, + requestUserConfirmation, + toolError, + toolResult, + type WebMcpToolResult, +} from "./runtime.webmcp"; + +type Json = Record; + +/** + * A step backed by an existing generated tool. The step inherits the tool's + * description and input schema — the definition lives in one place — and + * calls the raw caller the generated file exports (fetchGetAutocomplete, + * not the agent-facing execute wrapper). You write only what's new: which + * slice of the result lands in the draft. + */ +export interface ToolStep { + /** The generated tool object, e.g. getAutocompleteTool. */ + tool: { description?: string; inputSchema?: Json }; + /** + * The raw caller the generated file exports. Receives the step's input + * plus the draft so far, so a later step can feed on an earlier one's + * stored fields. + */ + call: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** Map the call's result into the draft fields this step leaves behind. */ + store: (result: unknown) => Json; + /** Draft fields this step leaves behind. Submit waits for all of them. */ + provides: string[]; + /** Override the agent-facing description. Default: the tool's own. */ + description?: string; + /** Override the agent-facing input schema. Default: the tool's own. */ + input?: Json; +} + +/** + * A step with no backend call of its own — it collects input into the draft + * ("set the title and dates"). With a `run`, it can do work first; whatever + * `run` returns is stored. Without one, the input is stored verbatim. + */ +export interface FreeStep { + description: string; + input: Json; + provides: string[]; + run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; +} + +export type JourneyStep = ToolStep | FreeStep; + +export interface JourneyDef { + /** Journey name; step tools derive from it ("document-trip-search-places"). */ + name: string; + /** The one sentence every step repeats to the agent, so the goal survives. */ + goal: string; + steps: Record; + submit: { + /** What the human confirms, e.g. "Create the trip and open the editor." */ + description: string; + /** Assemble the real tool's input from the draft. This is your code. */ + build: (draft: Readonly) => unknown; + /** The existing tool's execute — your real endpoint runs here. */ + run: (input: never, signal?: AbortSignal) => Promise; + }; +} + +function isToolStep(step: JourneyStep): step is ToolStep { + return "tool" in step; +} + +/** "document-trip-search-places" → "Document Trip Search Places" (native UIs). */ +function toTitle(kebab: string): string { + return kebab + .split("-") + .filter(Boolean) + .map((part) => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); +} + +export function createJourney(def: JourneyDef) { + const modelContext = getModelContext(); + + /** The shared draft. Page-scoped on purpose: reloads start clean. */ + let draft: Json = {}; + + /** Every (step, field) pair the draft still lacks, as readable text. */ + function missing(): string[] { + return Object.entries(def.steps).flatMap(([key, step]) => + step.provides + .filter((field) => draft[field] === undefined) + .map((field) => `${def.name}-${key} (stores "${field}")`), + ); + } + + function stepDescription(step: JourneyStep): string { + const base = + step.description ?? (isToolStep(step) ? step.tool.description : undefined) ?? "Journey step."; + return `${base} Part of "${def.name}": ${def.goal}`; + } + + function stepInput(step: JourneyStep): Json { + if (step.input) return step.input; + if (isToolStep(step) && step.tool.inputSchema) return step.tool.inputSchema; + return { type: "object", properties: {} }; + } + + /** Run one step and store what it produced. */ + async function runStep(step: JourneyStep, input: Json, signal?: AbortSignal): Promise { + if (isToolStep(step)) { + const result = await step.call(input, signal, { ...draft }); + Object.assign(draft, step.store(result)); + return; + } + const stored = step.run ? await step.run(input, signal, { ...draft }) : input; + Object.assign(draft, stored); + } + + async function registerSteps(signal?: AbortSignal): Promise { + if (!modelContext) return; + for (const [key, step] of Object.entries(def.steps)) { + await modelContext.registerTool( + { + name: `${def.name}-${key}`, + title: toTitle(`${def.name}-${key}`), + description: stepDescription(step), + inputSchema: stepInput(step), + annotations: { readOnlyHint: true }, + execute: async (input, context) => { + try { + await runStep(step, input as Json, context?.signal); + const left = missing(); + return toolResult( + left.length === 0 + ? `Stored. The journey is ready — call ${def.name}-submit.` + : `Stored. Still needed: ${left.join(", ")}.`, + ); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + } + + async function registerSubmit(signal?: AbortSignal): Promise { + if (!modelContext) return; + await modelContext.registerTool( + { + name: `${def.name}-submit`, + title: toTitle(`${def.name}-submit`), + description: def.submit.description, + inputSchema: { type: "object", properties: {} }, + annotations: { readOnlyHint: false, consequentialHint: true }, + execute: async (_input, context) => { + context?.signal?.throwIfAborted(); + const left = missing(); + if (left.length > 0) { + return toolError(`Not ready to submit. Call these first: ${left.join(", ")}.`); + } + const confirmed = await requestUserConfirmation( + `Allow the agent to: ${def.submit.description}`, + ); + if (!confirmed) return toolError("The user declined this action."); + try { + const result = await def.submit.run(def.submit.build(draft) as never, context?.signal); + draft = {}; // a submitted journey starts clean + return result as WebMcpToolResult; + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + + return { + /** Call once on page load, next to registerAllTools(). */ + async register(signal?: AbortSignal): Promise { + await registerSteps(signal); + await registerSubmit(signal); + }, + /** What's in the draft right now — for the dashboard and for tests. */ + inspectDraft: (): Json => ({ ...draft }), + }; +} diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts new file mode 100644 index 0000000..1092fe2 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts @@ -0,0 +1,89 @@ +import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; + +// ─── webmcp-codegen: generated. Do not edit this region. ─── +/** + * List the signed-in user's trips. Returns an array of trips. + * + * Source: GET /v1/trips/ (openapi). Risk: safe-read. + * Starts enabled (see executeListTrips below). + * Regenerate with: npx @webmcp-stack/codegen generate + */ + +/** The exact contract advertised to the agent. Derived from the API spec. Do not hand-edit. */ +export const listTripsInputSchema = { + "type": "object", + "properties": {}, + "required": [] +}; + +/** What `execute` receives. The browser validates agent input against the schema above. */ +export type ListTripsInput = Record; + +/** Safety hints computed by webmcp-codegen. Informational metadata for hosts and UIs. */ +export const listTripsHints = {"readOnlyHint":true,"destructiveHint":false,"idempotentHint":true,"untrustedContentHint":true} as const; + +/** The tool definition, minus `execute` (which is yours, below the marker). */ +export const listTripsTool = { + name: "list-trips", + title: "List Trips", + description: "List the signed-in user's trips. Returns an array of trips.", + inputSchema: listTripsInputSchema, + annotations: { + readOnlyHint: true, + untrustedContentHint: true, + consequentialHint: false, + }, +}; + +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeListTrips is the agent-facing one. */ +export async function fetchListTrips(input: ListTripsInput, signal?: AbortSignal) { + const data = await callApi("/v1/trips/", { method: "GET", signal }); + return data; +} + +/** + * Register this tool with WebMCP. Call it once on page load, or use + * registerAllTools() from the generated index.ts. Skips quietly when the + * browser has no WebMCP runtime. + * + * Pass an AbortSignal to unregister later: controller.abort(). + */ +export async function registerListTrips(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); + if (!modelContext) return; + await modelContext.registerTool( + { + ...listTripsTool, + // The browser has already validated the agent's input against the schema. + // A failure returns a readable result; it never throws (see asToolError). + execute: async (input, context) => { + try { + return await executeListTrips(input as ListTripsInput, context?.signal); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); +} + +// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── + +/** + * What actually happens when the agent calls "list-trips". + * + * Default implementation: calls GET /v1/trips/ from this page, with the + * signed-in user's session. Replace it with your app's own API client + * whenever you like; the contract above never changes. + * + * Calls the API on this page's own origin (same-origin by default). + */ +export async function executeListTrips(input: ListTripsInput, signal?: AbortSignal) { + const data = await fetchListTrips(input, signal); + // Make the effect visible: an agent acts while a human watches this + // page. If this call changes what is on screen, update the UI here + // (navigate, invalidate a query, dispatch an event). + return toolResult(data); +} \ No newline at end of file diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts new file mode 100644 index 0000000..df07e76 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts @@ -0,0 +1,191 @@ +/** + * Generated by webmcp-codegen. This file is fully regenerated on every run. + * Do not edit by hand; your changes will be lost. + */ + +/** The result shape tools return (same as MCP tool results). */ +export interface WebMcpToolResult { + content: { type: "text"; text: string }[]; + isError?: boolean; + [key: string]: unknown; +} + +/** A tool as the browser runtime understands it. */ +export interface WebMcpToolDefinition { + name: string; + /** A human-facing label for native UIs (the spec's USVString title). */ + title?: string; + description: string; + inputSchema?: Record; + /** Hints the agent reads to decide how careful to be with this tool. */ + annotations?: { + readOnlyHint?: boolean; + untrustedContentHint?: boolean; + consequentialHint?: boolean; + }; + execute: ( + input: Record, + context?: { signal?: AbortSignal }, + ) => unknown | Promise; +} + +/** The slice of the WebMCP draft spec the generated code uses. */ +export interface ModelContext { + registerTool( + tool: WebMcpToolDefinition, + options?: { signal?: AbortSignal; exposedTo?: string[] }, + ): Promise; +} + +/* Most browsers do not have WebMCP yet, so a missing model context is a + * normal page load, not an error. It gets one quiet log line per load, + * never one per tool. */ +let announcedUnavailable = false; + +/** + * The page's WebMCP model context, or null when the browser has none. + * Registration callers skip quietly on null; a missing runtime must never + * surface in the human-facing page (no throws, no console spam). + */ +export function getModelContext(): ModelContext | null { + const modelContext = (document as unknown as { modelContext?: ModelContext }).modelContext; + if (!modelContext && !announcedUnavailable) { + announcedUnavailable = true; + console.info( + "[webmcp-codegen] WebMCP is not available in this browser; tools were not registered.", + ); + } + return modelContext ?? null; +} + +/** + * Register every journey exported from the modules the barrel found in + * journeys/. Anything with a .register() method counts (createJourney's + * return shape); anything else is skipped quietly. One journey failing never + * takes the others down with it. + */ +export async function registerJourneys( + modules: Record[], + signal?: AbortSignal, +): Promise { + for (const module of modules) { + for (const value of Object.values(module)) { + const journey = value as { register?: unknown } | null; + if (journey !== null && typeof journey === "object" && typeof journey.register === "function") { + try { + await (journey.register as (signal?: AbortSignal) => Promise)(signal); + } catch (error) { + console.warn("[webmcp-codegen] a journey failed to register:", error); + } + } + } + } +} + +/** + * Call your API from the page. Same origin by default (pass a full URL when + * the API lives on another host), always with the signed-in user's session + * cookies. Throws on HTTP errors; returns the parsed JSON body, or raw text + * when the response is not JSON. Pass the signal from execute's context so + * a cancelled tool call stops the request. + */ +export async function callApi( + path: string, + options: { + method?: string; + query?: Record; + body?: unknown; + signal?: AbortSignal; + } = {}, +): Promise { + const url = new URL(path, window.location.origin); + for (const [key, value] of Object.entries(options.query ?? {})) { + if (value !== undefined && value !== null) url.searchParams.set(key, String(value)); + } + const response = await fetch(url, { + method: options.method ?? "GET", + credentials: "include", + headers: options.body !== undefined ? { "content-type": "application/json" } : undefined, + body: options.body !== undefined ? JSON.stringify(options.body) : undefined, + signal: options.signal, + }); + if (!response.ok) { + throw new Error("Request failed: " + response.status + " " + response.statusText); + } + if (response.status === 204) return null; + const text = await response.text(); + try { + return JSON.parse(text); + } catch { + return text; + } +} + +/** Chrome's output budget: one tool result stays under ~1.5K characters. */ +const TOOL_OUTPUT_MAX = 1536; + +const TRUNCATED_NOTICE = + " +… [truncated to fit the 1.5K output budget — return a smaller slice or paginate]"; + +/** + * Wrap a result in the MCP shape, so tool bodies stay one line. The result + * text is capped at Chrome's ~1.5K per-call output budget: oversized payloads + * cost the agent context and can trip guardrails, so they are cut with a + * notice rather than delivered whole. The cap lives here in the shared + * runtime, so it cannot be edited away per tool. + */ +export function toolResult(data: unknown): WebMcpToolResult { + const text = typeof data === "string" ? data : JSON.stringify(data, null, 2); + const fitted = + text.length <= TOOL_OUTPUT_MAX + ? text + : text.slice(0, TOOL_OUTPUT_MAX - TRUNCATED_NOTICE.length) + TRUNCATED_NOTICE; + return { + content: [{ type: "text", text: fitted }], + }; +} + +/** + * A failure the agent can read and act on. The one hard rule of the execute + * contract: never throw for failure. The browser maps a rejected execute to + * a bare UnknownError and discards the message, so a thrown failure teaches + * the agent nothing. Cancellation is the only exception, which is why + * asToolError re-throws AbortError. + */ +export function toolError(message: string): WebMcpToolResult { + return { content: [{ type: "text", text: message }], isError: true }; +} + +/** Convert any thrown failure into a readable error result. */ +export function asToolError(error: unknown): WebMcpToolResult { + if (error instanceof DOMException && error.name === "AbortError") throw error; + return toolError(error instanceof Error ? error.message : "The tool failed."); +} + +/** + * What a disabled tool tells the agent. The tool stays visible (so the agent + * knows it exists and can ask the human to enable it) but does nothing. + */ +export function toolDisabled(fileName: string): WebMcpToolResult { + return { + content: [ + { + type: "text", + text: + "This tool is currently disabled by the app developer. Ask them to enable it " + + "(uncomment the implementation in " + fileName + ").", + }, + ], + isError: true, + }; +} + +/** + * Default "agent proposes, human confirms" gate for write/destructive tools. + * Deliberately minimal (window.confirm). Replace it with your app's own + * dialog when you outgrow it. The point is that the user always gets a say. + */ +export function requestUserConfirmation(message: string): Promise { + return Promise.resolve(window.confirm(message)); +} diff --git a/packages/codegen/evals/skill/results/.gitignore b/packages/codegen/evals/skill/results/.gitignore new file mode 100644 index 0000000..224cbaf --- /dev/null +++ b/packages/codegen/evals/skill/results/.gitignore @@ -0,0 +1,3 @@ +# Eval reports stay local +* +!.gitignore diff --git a/packages/codegen/evals/skill/run.mjs b/packages/codegen/evals/skill/run.mjs new file mode 100644 index 0000000..2a257ab --- /dev/null +++ b/packages/codegen/evals/skill/run.mjs @@ -0,0 +1,398 @@ +#!/usr/bin/env node +/** + * The skill-file eval runner: prompt → captured run → checks → score. + * + * Each case gets a fresh copy of fixture/ (plus the skill file, unless the + * case removes it), an agent runs the case's prompt inside it headlessly, + * and deterministic checks grade what the agent left behind — files, not + * transcripts, wherever possible. Behavior is nondeterministic: run trials + * and report pass RATES, never a single run's verdict. + * + * Usage: + * node run.mjs all cases, default trials + * node run.mjs --trials 5 more trials per case + * node run.mjs --case journey-document-trip + * node run.mjs --selftest grade the graders (no agent needed) + * + * The agent command comes from AGENT_CMD, defaulting to cases.json's + * `agent_cmd_default`; `{PROMPT}` is replaced by the shell-quoted prompt. + * Examples: + * AGENT_CMD='codex exec --json "{PROMPT}"' + * AGENT_CMD='claude -p "{PROMPT}" --output-format json' + */ + +import { execFile, spawn } from "node:child_process"; +import { cp, mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { dirname, join, relative } from "node:path"; +import { fileURLToPath } from "node:url"; +import { promisify } from "node:util"; + +const here = dirname(fileURLToPath(import.meta.url)); +const FIXTURE = join(here, "fixture"); +const GENERATED_END = + "// ─── webmcp-codegen: end generated. Your code below survives regeneration. ───"; + +// ---------------------------------------------------------------- graders +// Every grader: async (ctx) => null on pass, or a sentence on failure. +// ctx = { workdir, fixtureDir, transcript, check } + +/** Read file list under a dir, recursively, relative paths sorted. */ +async function tree(dir, base = dir) { + let out = []; + let entries = []; + try { + entries = await readdir(dir, { withFileTypes: true }); + } catch { + return out; + } + for (const entry of entries) { + const full = join(dir, entry.name); + if (entry.isDirectory()) out = out.concat(await tree(full, base)); + else out.push(relative(base, full)); + } + return out.sort(); +} + +async function readIn(dir, rel) { + try { + return await readFile(join(dir, rel), "utf8"); + } catch { + return null; + } +} + +const graders = { + async "file-exists"({ workdir, check }) { + return (await readIn(workdir, check.path)) === null ? `missing file: ${check.path}` : null; + }, + + async "file-matches"({ workdir, check }) { + const contents = await readIn(workdir, check.path); + if (contents === null) return `missing file: ${check.path}`; + return matchFailures(contents, check); + }, + + async "any-file-matches"({ workdir, check }) { + const files = (await tree(join(workdir, check.dir), join(workdir, check.dir))).filter((name) => + name.endsWith(".webmcp.ts"), + ); + for (const name of files) { + const contents = await readIn(join(workdir, check.dir), name); + if (contents !== null && matchFailures(contents, check) === null) return null; + } + return `no file under ${check.dir}/ satisfied all patterns (checked ${files.length} file${files.length === 1 ? "" : "s"})`; + }, + + /** The generated region (start of file through the end marker) must be byte-identical to the fixture's. */ + async "marker-preserved"({ workdir, fixtureDir, check }) { + const before = await readIn(fixtureDir, check.path); + const after = await readIn(workdir, check.path); + if (after === null) return `missing file: ${check.path}`; + if (before === null) return `fixture has no ${check.path} to compare against`; + const headOf = (text) => { + const index = text.indexOf(GENERATED_END); + return index === -1 ? text : text.slice(0, index + GENERATED_END.length); + }; + return headOf(after) === headOf(before) + ? null + : `${check.path}: the generated region was edited — regeneration will clobber it; descriptions and names move through .webmcp-codegen.json`; + }, + + /** Nothing under src/webmcp may differ from the fixture (negative cases). */ + async "no-webmcp-diff"({ workdir, fixtureDir }) { + const subdir = "src/webmcp"; + const beforeFiles = await tree(join(fixtureDir, subdir)); + const afterFiles = await tree(join(workdir, subdir)); + const changed = new Set(); + for (const name of new Set([...beforeFiles, ...afterFiles])) { + const before = await readIn(join(fixtureDir, subdir), name); + const after = await readIn(join(workdir, subdir), name); + if (before !== after) changed.add(name); + } + return changed.size === 0 + ? null + : `webmcp files changed on an unrelated prompt: ${[...changed].join(", ")}`; + }, + + /** Tool and parameter descriptions inside webmcp files stay within budget. */ + async "tool-description-budget"({ workdir, check }) { + const files = (await tree(join(workdir, check.dir), join(workdir, check.dir))).filter((name) => + name.endsWith(".webmcp.ts"), + ); + const offenders = []; + for (const name of files) { + const contents = await readIn(join(workdir, check.dir), name); + if (contents === null) continue; + const toolMatch = /description: "((?:[^"\\]|\\.)*)"/.exec(contents); + if (toolMatch && (toolMatch[1] ?? "").length > check.max_tool) { + offenders.push(`${name}: tool description ${(toolMatch[1] ?? "").length} chars`); + } + for (const match of contents.matchAll( + /(?:properties|input):[\s\S]*?description: "((?:[^"\\]|\\.)*)"/g, + )) { + if ((match[1] ?? "").length > check.max_param) { + offenders.push(`${name}: a parameter description runs ${(match[1] ?? "").length} chars`); + } + } + } + return offenders.length === 0 ? null : offenders.join("; "); + }, + + async "transcript-matches"({ transcript, check }) { + return new RegExp(check.pattern, "i").test(transcript) + ? null + : `transcript never matched /${check.pattern}/`; + }, + + async "any-of"(ctx) { + const failures = []; + for (const inner of ctx.check.checks) { + const failure = await graders[inner.type]({ ...ctx, check: inner }); + if (failure === null) return null; + failures.push(failure); + } + return `none of the alternatives passed: ${failures.join(" | ")}`; + }, +}; + +function matchFailures(contents, check) { + for (const pattern of check.all ?? []) { + if (!new RegExp(pattern, "m").test(contents)) return `missing expected pattern: /${pattern}/`; + } + for (const pattern of check.none ?? []) { + if (new RegExp(pattern, "m").test(contents)) return `matched forbidden pattern: /${pattern}/`; + } + return null; +} + +// ------------------------------------------------------------------- runs + +function shellQuote(text) { + return `'${text.replaceAll("'", "'\\''")}'`; +} + +function runAgent(command, prompt, workdir, timeoutMs) { + const cmd = command.replace("{PROMPT}", shellQuote(prompt)); + return new Promise((resolveRun) => { + const child = spawn(cmd, { cwd: workdir, shell: true, env: process.env }); + let out = ""; + child.stdout.on("data", (chunk) => (out += chunk)); + child.stderr.on("data", (chunk) => (out += chunk)); + const timer = setTimeout(() => child.kill("SIGTERM"), timeoutMs); + child.on("close", (code) => { + clearTimeout(timer); + resolveRun({ code, transcript: out }); + }); + }); +} + +async function runCase(caseDef, command, trials, report) { + const results = []; + for (let trial = 0; trial < trials; trial++) { + const workdir = await mkdtemp(join(tmpdir(), `webmcp-eval-${caseDef.id}-`)); + try { + await cp(FIXTURE, workdir, { recursive: true }); + if (caseDef.skill === false) + await rm(join(workdir, ".agents"), { recursive: true, force: true }); + const { transcript } = await runAgent( + command, + caseDef.prompt, + workdir, + Number(process.env.EVAL_TIMEOUT_MS ?? 10 * 60 * 1000), + ); + const failures = []; + for (const check of caseDef.checks) { + const failure = await graders[check.type]({ + workdir, + fixtureDir: FIXTURE, + transcript, + check, + }); + if (failure !== null) failures.push(`[${check.type}] ${failure}`); + } + results.push({ trial, pass: failures.length === 0, failures }); + } finally { + await rm(workdir, { recursive: true, force: true }); + } + } + const passed = results.filter((r) => r.pass).length; + const gate = caseDef.kind === "control" ? "informational" : passed === results.length; + report.cases.push({ + id: caseDef.id, + kind: caseDef.kind, + passed, + trials: results.length, + gate, + results, + }); + console.log( + ` ${gate === true ? "✓" : gate === "informational" ? "ℹ" : "✖"} ${caseDef.id}: ${passed}/${results.length} passed${caseDef.kind === "control" ? " (control)" : ""}`, + ); + for (const result of results.filter((r) => !r.pass)) { + for (const line of result.failures) console.log(` trial ${result.trial + 1}: ${line}`); + } +} + +// -------------------------------------------------------------- self-test + +async function selftest() { + const workdir = await mkdtemp(join(tmpdir(), "webmcp-eval-selftest-")); + try { + await cp(FIXTURE, workdir, { recursive: true }); + + const expectOutcome = async (label, check, expectFail, patch) => { + const dir = await mkdtemp(join(tmpdir(), "webmcp-eval-selftest-case-")); + await cp(FIXTURE, dir, { recursive: true }); + if (patch) await patch(dir); + const failure = await graders[check.type]({ + workdir: dir, + fixtureDir: FIXTURE, + transcript: "", + check, + }); + const gotFail = failure !== null; + console.log( + ` ${gotFail === expectFail ? "✓" : "✖"} ${label}${gotFail && expectFail ? ` (${failure})` : ""}`, + ); + await rm(dir, { recursive: true, force: true }); + return gotFail === expectFail; + }; + + const outcomes = []; + const record = async (...args) => outcomes.push(await expectOutcome(...args)); + + await record( + "marker-preserved: untouched file passes", + { type: "marker-preserved", path: "src/webmcp/create-trip.webmcp.ts" }, + false, + ); + await record( + "marker-preserved: edited head fails", + { type: "marker-preserved", path: "src/webmcp/create-trip.webmcp.ts" }, + true, + async (dir) => { + const path = join(dir, "src/webmcp/create-trip.webmcp.ts"); + await writeFile( + path, + (await readFile(path, "utf8")).replace("Create a new trip.", "HAND EDITED text."), + ); + }, + ); + await record( + "marker-preserved: edit below the marker passes", + { type: "marker-preserved", path: "src/webmcp/create-trip.webmcp.ts" }, + false, + async (dir) => { + const path = join(dir, "src/webmcp/create-trip.webmcp.ts"); + await writeFile(path, `${await readFile(path, "utf8")}\n// user code below the marker\n`); + }, + ); + await record("no-webmcp-diff: untouched passes", { type: "no-webmcp-diff" }, false); + await record( + "no-webmcp-diff: a new tool fails", + { type: "no-webmcp-diff" }, + true, + async (dir) => writeFile(join(dir, "src/webmcp/new-tool.webmcp.ts"), "export {};\n"), + ); + await record( + "any-file-matches: a pattern the factory satisfies passes", + { + type: "any-file-matches", + dir: "src/webmcp", + all: ["export function createJourney"], + none: [], + }, + false, + ); + await record( + "any-file-matches: an empty journeys dir fails", + { type: "any-file-matches", dir: "src/webmcp/journeys", all: ["createJourney"], none: [] }, + true, + ); + await record( + "any-file-matches: forbidden pattern trips", + { type: "any-file-matches", dir: "src/webmcp", all: ["createTrip"], none: ["placeId"] }, + true, + ); + await record( + "any-file-matches: clean file passes", + { type: "any-file-matches", dir: "src/webmcp", all: ["createTrip"], none: [] }, + false, + ); + await record( + "tool-description-budget: fixture within budget passes", + { type: "tool-description-budget", dir: "src/webmcp", max_tool: 500, max_param: 150 }, + false, + ); + await record( + "tool-description-budget: over-budget description fails", + { type: "tool-description-budget", dir: "src/webmcp", max_tool: 500, max_param: 150 }, + true, + async (dir) => { + const path = join(dir, "src/webmcp/list-trips.webmcp.ts"); + await writeFile( + path, + (await readFile(path, "utf8")).replace( + /description: "((?:[^"\\]|\\.)*)"/, + `description: "Wordy. ${"Detail upon detail. ".repeat(40)}"`, + ), + ); + }, + ); + await record( + "transcript-matches: hits and misses", + { type: "transcript-matches", pattern: "no-such-phrase-in-empty-transcript" }, + true, + ); + + const ok = outcomes.every(Boolean); + await rm(workdir, { recursive: true, force: true }); + console.log(ok ? "\n Self-test passed." : "\n Self-test FAILED."); + return ok ? 0 : 1; + } catch (error) { + console.error(error); + return 1; + } +} + +// -------------------------------------------------------------------- main + +async function main() { + const args = process.argv.slice(2); + if (args.includes("--selftest")) process.exit(await selftest()); + + const cases = JSON.parse(await readFile(join(here, "cases.json"), "utf8")); + const command = process.env.AGENT_CMD ?? cases.agent_cmd_default; + const trials = Number(args[args.indexOf("--trials") + 1]) || cases.trials_default || 3; + const only = args.includes("--case") ? args[args.indexOf("--case") + 1] : undefined; + + const selected = cases.cases.filter((c) => !only || c.id === only); + if (selected.length === 0) { + console.error(`No case matching ${only}. Known: ${cases.cases.map((c) => c.id).join(", ")}`); + process.exit(2); + } + + console.log(`Running ${selected.length} case(s), ${trials} trial(s) each, via: ${command}\n`); + const report = { at: new Date().toISOString(), command, trials, cases: [] }; + for (const caseDef of selected) await runCase(caseDef, command, trials, report); + + const outDir = join(here, "results"); + await mkdir(outDir, { recursive: true }); + const outPath = join(outDir, `${report.at.replaceAll(":", "-")}.json`); + await writeFile(outPath, JSON.stringify(report, null, 2)); + + const gated = report.cases.filter((c) => c.kind !== "control"); + const allPassed = gated.every((c) => c.gate === true); + console.log( + `\n ${gated.filter((c) => c.gate === true).length}/${gated.length} gated cases fully passing — report: ${relative(process.cwd(), outPath)}`, + ); + process.exit(allPassed ? 0 : 1); +} + +const execFileAsync = promisify(execFile); +void execFileAsync; +main().catch((error) => { + console.error(error); + process.exit(1); +}); From d0641019c4b0c080ace629831082725fd5333ec6 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 03:22:43 +0530 Subject: [PATCH 12/41] Group handshake endpoints into one coarse tool (the anti-1:1 step) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit POST pairs like request-upload + complete-upload are one action the API split in two. groupHandshakes detects them deterministically (paired begin/end verbs, shared noun, same resource, exact-name threading of the second call's path params from the first response — anything fuzzy is skipped with a note) and appends a merged withheld tool whose fetchX runs both calls in order. Members stay untouched; adoption is enabling the merged tool, and overrides keep that decision across regenerations. Pipeline notes now print in the CLI summary so the proposal is visible without opening the dashboard. --- packages/codegen/src/cli-output.ts | 8 + packages/codegen/src/group.test.ts | 134 +++++++++++ packages/codegen/src/group.ts | 216 ++++++++++++++++++ .../codegen/src/outputs/tools-templates.ts | 98 ++++++-- packages/codegen/src/pipeline.ts | 15 +- packages/codegen/src/types.ts | 19 ++ 6 files changed, 468 insertions(+), 22 deletions(-) create mode 100644 packages/codegen/src/group.test.ts create mode 100644 packages/codegen/src/group.ts diff --git a/packages/codegen/src/cli-output.ts b/packages/codegen/src/cli-output.ts index 1169998..b37a358 100644 --- a/packages/codegen/src/cli-output.ts +++ b/packages/codegen/src/cli-output.ts @@ -229,6 +229,14 @@ export function renderSummary( if (fileNotes.length > 6) { console.log(dim(` …and ${fileNotes.length - 6} more (run with --verbose)`)); } + + // Pipeline proposals (groupings, renames): the run's "look at this" lines. + for (const note of result.notes.slice(0, 6)) { + console.log(` ${c.cyan("◦")} ${note}`); + } + if (result.notes.length > 6) { + console.log(dim(` …and ${result.notes.length - 6} more (run with --verbose)`)); + } console.log(""); // LLM proposals: visually distinct from findings (◦, cyan), because a diff --git a/packages/codegen/src/group.test.ts b/packages/codegen/src/group.test.ts new file mode 100644 index 0000000..0650c22 --- /dev/null +++ b/packages/codegen/src/group.test.ts @@ -0,0 +1,134 @@ +import { describe, expect, it } from "vitest"; +import { groupHandshakes } from "./group.js"; +import { generatedRegion } from "./outputs/tools-templates.js"; +import type { CandidateTool, ReviewedTool } from "./types.js"; + +function post( + overrides: Partial & { name: string; pathTemplate: string }, +): CandidateTool { + return { + id: `POST ${overrides.pathTemplate}`, + source: { kind: "openapi", ref: `POST ${overrides.pathTemplate}` }, + inputSchema: { type: "object", properties: {} }, + inputTypeName: "Input", + httpMethod: "POST", + paramLocations: { path: [], query: [], body: [] }, + sideEffect: "write", + requiresAuth: false, + description: "Does a thing.", + descriptionSource: "openapi-summary", + ...overrides, + }; +} + +/** The beenthere upload handshake, faithfully. */ +function uploadPair(): CandidateTool[] { + return [ + post({ + name: "create-media-request-upload", + pathTemplate: "/v1/media/request-upload", + paramLocations: { path: [], query: [], body: ["fileName"] }, + inputSchema: { + type: "object", + properties: { fileName: { type: "string", description: "The file's name." } }, + required: ["fileName"], + }, + outputSchema: { + type: "object", + properties: { uploadId: { type: "string" }, url: { type: "string" } }, + }, + }), + post({ + name: "complete-media-upload", + pathTemplate: "/v1/media/uploads/{uploadId}/complete", + paramLocations: { path: ["uploadId"], query: [], body: [] }, + inputSchema: { + type: "object", + properties: { uploadId: { type: "string", description: "The upload to complete." } }, + required: ["uploadId"], + }, + outputSchema: { type: "object", properties: { media: { type: "object" } } }, + }), + ]; +} + +describe("groupHandshakes", () => { + it("merges a request/complete pair into one withheld, thread-wired proposal", () => { + const { tools, notes } = groupHandshakes(uploadPair()); + + // Members stay; the merged tool is appended. + expect(tools).toHaveLength(3); + const merged = tools.find((tool) => tool.name === "upload-media"); + expect(merged).toBeDefined(); + expect(merged?.compose?.threaded).toEqual({ uploadId: "uploadId" }); + + // The merged input drops the threaded field, keeps the real input. + expect(Object.keys(merged?.inputSchema.properties ?? {})).toEqual(["fileName"]); + expect(merged?.inputSchema.required).toEqual(["fileName"]); + + // The result is the SECOND call's response. + expect(merged?.outputSchema?.properties).toHaveProperty("media"); + + // The report names the proposal and how to adopt it. + expect(notes[0]).toContain( + "Grouped create-media-request-upload + complete-media-upload into upload-media", + ); + expect(notes[0]).toContain("starts withheld"); + }); + + it("skips the pair when the second call's path param can't be threaded", () => { + const pair = uploadPair(); + pair[0]!.outputSchema = { type: "object", properties: { token: { type: "string" } } }; + const { tools, notes } = groupHandshakes(pair); + expect(tools).toHaveLength(2); + expect(notes[0]).toContain("path params the first response doesn't provide"); + }); + + it("does not pair across different resources", () => { + const pair = uploadPair(); + pair[1]!.pathTemplate = "/v1/videos/uploads/{uploadId}/complete"; + pair[1]!.name = "complete-videos-upload"; + const { tools } = groupHandshakes(pair); + expect(tools).toHaveLength(2); + }); + + it("does not pair GETs or lone POSTs", () => { + const readsOnly = [post({ name: "request-data-export", pathTemplate: "/v1/data/request" })]; + expect(groupHandshakes(readsOnly).tools).toHaveLength(1); + + const getPair = uploadPair().map((tool) => ({ ...tool, httpMethod: "GET" as const })); + expect(groupHandshakes(getPair).tools).toHaveLength(2); + }); + + it("falls back to a free name when the natural one is taken", () => { + const pair = uploadPair(); + const blocker = post({ name: "upload-media", pathTemplate: "/v1/media/other" }); + const { tools } = groupHandshakes([...pair, blocker]); + const merged = tools.find((tool) => tool.compose); + expect(merged?.name).not.toBe("upload-media"); + expect(merged?.name).toBeTruthy(); + }); +}); + +describe("composed tool templates", () => { + it("emits a two-call fetchX that threads the first response into the second call", () => { + const { tools } = groupHandshakes(uploadPair()); + const merged = tools.find((tool) => tool.compose); + const region = generatedRegion({ + ...merged, + enabledByDefault: false, + withheld: true, + riskTier: "write-confirm", + hints: { readOnlyHint: false, untrustedContentHint: false }, + piiInOutput: [], + endpointRole: "endpoint", + } as unknown as ReviewedTool); + + expect(region).toContain("export async function fetchUploadMedia("); + expect(region).toContain('const firstResult = (await callApi("/v1/media/request-upload"'); + expect(region).toContain("${firstResult.uploadId}/complete"); + expect(region).toContain("body: { fileName: input.fileName }"); + // The merged tool is a write: the confirmation gate applies. + expect(region).toContain("requestUserConfirmation("); + }); +}); diff --git a/packages/codegen/src/group.ts b/packages/codegen/src/group.ts new file mode 100644 index 0000000..42f4d27 --- /dev/null +++ b/packages/codegen/src/group.ts @@ -0,0 +1,216 @@ +/** + * The grouping step: find the endpoints that are one action split by API + * shape, and emit the single coarse tool an agent should see instead. + * + * The canonical case is the upload handshake: POST /media/request-upload + + * POST /media/uploads/{uploadId}/complete is not two decisions, it's one + * upload split in two calls. An agent that must orchestrate the pair gets + * hallucination room ("what's an uploadId again?"); one upload-media tool + * closes it. + * + * Restraint is the whole game here (see the direction doc: naive 1:1 mapping + * is the failure, but so is inventing cleverness). A pair merges ONLY when + * every line holds deterministically: + * + * - both are POST endpoints under the same first path segment (one resource) + * - the left name carries a begin-verb (request/begin/initiate/prepare/start) + * and the right name an end-verb (complete/commit/confirm/finish/end) + * - the right name carries the left's final token (the shared noun: + * "upload" in request-upload + complete-media-upload) + * - the shared noun token is at least 3 letters ("do" pairs nothing) + * + * Threading (how the merged call feeds the second request from the first + * response) is exact-name only: the right side's path params must each match + * a property on the left's response schema (`{uploadId}` ← `uploadId`). If a + * path param can't be threaded, the pair is skipped — guessing data flow is + * how plausible garbage gets shipped. + * + * The merged tool is withheld like any write: the merge is a proposal the + * developer adopts by enabling it. Members are left untouched (they're + * withheld writes already), so adopting never means losing the alternative. + */ + +import pluralize from "pluralize"; +import { pascalCase } from "./json-schema.js"; +import type { CandidateTool, JsonSchema } from "./types.js"; + +/** Verbs that start a handshake, and the verbs allowed to end it. */ +const BEGIN_VERBS = new Set(["request", "begin", "initiate", "prepare", "start"]); +const END_VERBS = new Set(["complete", "commit", "confirm", "finish", "end"]); + +/** One HTTP call of a composed tool. */ +export interface ComposedCall { + httpMethod: NonNullable; + pathTemplate: string; + paramLocations: { path: string[]; query: string[]; body: string[] }; + serverUrl?: string; + /** The input type of the endpoint this call came from. */ +} + +/** What a merged tool needs to emit its two-call request. */ +export interface ComposedPlan { + first: ComposedCall; + second: ComposedCall; + /** Second-call fields filled from the first response, keyed by field name. */ + threaded: Record; +} + +export interface GroupResult { + tools: CandidateTool[]; + notes: string[]; +} + +/** The response property names a call's result exposes for threading. */ +function responseFields(tool: CandidateTool): Set { + return new Set(Object.keys(tool.outputSchema?.properties ?? {})); +} + +/** True when the right endpoint's path params can all be filled from left response fields. */ +function threadingFor( + left: CandidateTool, + right: CandidateTool, +): Record | undefined { + const available = responseFields(left); + const threaded: Record = {}; + for (const param of right.paramLocations?.path ?? []) { + if (!available.has(param)) return undefined; + threaded[param] = param; + } + return threaded; +} + +/** The verb tokens of a name, in order. */ +function tokensOf(name: string): string[] { + return name.split("-").filter(Boolean); +} + +function firstPathSegment(pathTemplate: string | undefined): string | undefined { + // Version prefixes ("v1", "v2") carry no resource identity of their own. + return pathTemplate + ?.split("/") + .filter(Boolean) + .find((segment) => !/^v\d+$/.test(segment)); +} + +/** Try the candidate's preferred names in order; return the first free one. */ +function pickMergedName(noun: string, resource: string, taken: Set): string | undefined { + const resourceSingular = pluralize.singular(resource); + const candidates = + noun === resourceSingular || noun === resource + ? [noun, `${noun}-flow`] + : [`${noun}-${resourceSingular}`, `${noun}-${resource}`, `${noun}-flow`]; + return candidates.find((candidate) => !taken.has(candidate)); +} + +/** Merge two schemas' properties; left wins on name conflict. */ +function mergedInputSchema( + left: CandidateTool, + right: CandidateTool, + threaded: Record, +): JsonSchema { + const properties: Record = { + ...(right.inputSchema.properties ?? {}), + ...(left.inputSchema.properties ?? {}), + }; + for (const field of Object.keys(threaded)) delete properties[field]; + const required = new Set([ + ...(left.inputSchema.required ?? []), + ...(right.inputSchema.required ?? []), + ]); + for (const field of Object.keys(threaded)) required.delete(field); + return { + type: "object", + properties, + ...(required.size > 0 ? { required: [...required].sort() } : {}), + }; +} + +/** + * Detect handshake pairs among named candidates and append merged proposals. + * Members stay in the list exactly as they were; the merged tool arrives + * marked so the safety layer gives it the write treatment its members have. + */ +export function groupHandshakes(candidates: CandidateTool[]): GroupResult { + const notes: string[] = []; + const added: CandidateTool[] = []; + const taken = new Set(candidates.map((tool) => tool.name)); + + const posts = candidates.filter( + (tool) => tool.httpMethod === "POST" && tool.pathTemplate && tool.paramLocations, + ); + + const consumed = new Set(); + for (const left of posts) { + if (consumed.has(left.id)) continue; + const leftTokens = tokensOf(left.name); + if (!leftTokens.some((token) => BEGIN_VERBS.has(token))) continue; + const noun = leftTokens[leftTokens.length - 1]; + if (!noun || noun.length < 3) continue; + const resource = firstPathSegment(left.pathTemplate); + if (!resource) continue; + + const right = posts.find( + (candidate) => + candidate !== left && + !consumed.has(candidate.id) && + firstPathSegment(candidate.pathTemplate) === resource && + tokensOf(candidate.name).some((token) => END_VERBS.has(token)) && + tokensOf(candidate.name).includes(noun), + ); + if (!right) continue; + + const threaded = threadingFor(left, right); + if (threaded === undefined) { + notes.push( + `${left.name} + ${right.name} look like one flow, but "${right.name}" takes path params the first response doesn't provide — left as separate tools.`, + ); + continue; + } + + const name = pickMergedName(noun, resource, taken); + if (!name) { + notes.push( + `${left.name} + ${right.name} look like one flow, but every merged name collided — left as separate tools.`, + ); + continue; + } + + taken.add(name); + consumed.add(left.id); + consumed.add(right.id); + + const resourceSingular = pluralize.singular(resource); + added.push({ + id: `${left.id} + ${right.id}`, + name, + source: { kind: "openapi", ref: `${left.source.ref} + ${right.source.ref}` }, + inputSchema: mergedInputSchema(left, right, threaded), + outputSchema: right.outputSchema, + inputTypeName: `${pascalCase(name)}Input`, + sideEffect: "write", + requiresAuth: left.requiresAuth || right.requiresAuth, + description: `Complete the ${resourceSingular} ${noun}. One action the API split into two calls; they run in order. Returns the completed ${noun}.`, + descriptionSource: "generated-template", + compose: { + first: { + httpMethod: left.httpMethod as NonNullable, + pathTemplate: left.pathTemplate as string, + paramLocations: left.paramLocations as ComposedCall["paramLocations"], + serverUrl: left.serverUrl, + }, + second: { + httpMethod: right.httpMethod as NonNullable, + pathTemplate: right.pathTemplate as string, + paramLocations: right.paramLocations as ComposedCall["paramLocations"], + serverUrl: right.serverUrl, + }, + threaded, + }, + }); + notes.push( + `Grouped ${left.name} + ${right.name} into ${name} — one action the API split in two calls. It starts withheld like its members; enable ${name} instead of the pair when you're satisfied.`, + ); + } + + return { tools: [...candidates, ...added], notes }; +} diff --git a/packages/codegen/src/outputs/tools-templates.ts b/packages/codegen/src/outputs/tools-templates.ts index f3a8f12..0596b71 100644 --- a/packages/codegen/src/outputs/tools-templates.ts +++ b/packages/codegen/src/outputs/tools-templates.ts @@ -59,7 +59,8 @@ export function generatedRegion( // fence, so the only runtime helper its live code uses is toolDisabled in // the execute scaffold (a defensive refusal if someone registers it by hand). const withheld = tool.withheld && !enabled; - const hasRoute = Boolean(tool.httpMethod && tool.pathTemplate && tool.paramLocations); + const hasRoute = + Boolean(tool.compose) || Boolean(tool.httpMethod && tool.pathTemplate && tool.paramLocations); const runtimeImports = withheld ? hasRoute ? "callApi, toolDisabled" @@ -170,8 +171,9 @@ export function generatedRegion( `/** The bare request, without the agent-facing result wrapping. Journeys`, ` * and your own code compose this; execute${pascal} is the agent-facing one. */`, `export async function fetch${pascal}(input: ${tool.inputTypeName}, signal?: AbortSignal) {`, - ` ${requestCall(tool)}`, - ` return data;`, + ...(tool.compose + ? composedFetchBody(tool.compose) + : [` ${requestCall(tool)}`, ` return data;`]), `}`, ] : []), @@ -226,7 +228,8 @@ export function generatedRegion( */ export function ownedRegionScaffold(tool: ReviewedTool): string { const pascal = pascalCase(tool.name); - const hasRoute = Boolean(tool.httpMethod && tool.pathTemplate && tool.paramLocations); + const hasRoute = + Boolean(tool.compose) || Boolean(tool.httpMethod && tool.pathTemplate && tool.paramLocations); // Endpoint-backed tools compose the raw caller from the generated region // (fetchX), so the default execute stays one line and journeys reuse the // exact same request. Schema-only tools keep the honest TODO. @@ -366,46 +369,103 @@ export function resolveApiBase(serverUrl: string | undefined): string | undefine } } -function requestCall(tool: ReviewedTool): string { - if (!tool.httpMethod || !tool.pathTemplate || !tool.paramLocations) { - return `const data = null; // TODO: call your app's existing code here.`; - } - - const { path: pathParams, query: queryParams, body: bodyParams } = tool.paramLocations; +/** + * The arguments to one callApi(...): the path expression (template params + * interpolated, non-local server URLs kept) plus method/query/body/signal. + * `pathRef` says where each path param's value comes from — ordinary tools + * read `input`, the second call of a composed tool reads the first result. + */ +function buildCallExpr(options: { + httpMethod: string; + pathTemplate: string; + paramLocations: { path: string[]; query: string[]; body: string[] }; + serverUrl?: string; + pathRef?: (param: string) => string; + skipFields?: Set; +}): string { + const { + httpMethod, + pathTemplate, + paramLocations, + serverUrl, + pathRef = inputRef, + skipFields = new Set(), + } = options; + const { path: pathParams, query: queryParamsAll, body: bodyParamsAll } = paramLocations; + const queryParams = queryParamsAll.filter((name) => !skipFields.has(name)); + const bodyParams = bodyParamsAll.filter((name) => !skipFields.has(name)); // "/pets/{id}" → `/pets/${input.id}`. Params the schema knows by name. - let pathExpr = `\`${tool.pathTemplate.replace(/\{([^}]+)\}/g, (_m, param: string) => `\${${inputRef(param)}}`)}\``; - if (pathParams.length === 0) pathExpr = JSON.stringify(tool.pathTemplate); + let pathExpr = `\`${pathTemplate.replace(/\{([^}]+)\}/g, (_m, param: string) => `\${${pathRef(param)}}`)}\``; + if (pathParams.length === 0) pathExpr = JSON.stringify(pathTemplate); // Base URL: default to the page's own origin so a deployed app calls its // own API. Baking the spec's servers[0] (often http://localhost:3001) into // the generated fetch would make every deployed tool call the visitor's // own machine. A non-local public server URL is kept as the default base; // a local one is not. - const base = resolveApiBase(tool.serverUrl); + const base = resolveApiBase(serverUrl); if (base) { const b = base.endsWith("/") ? base.slice(0, -1) : base; pathExpr = `\`${b}\${${pathExpr}}\``; } - const options: string[] = [`method: ${JSON.stringify(tool.httpMethod)}`]; + const args: string[] = [`method: ${JSON.stringify(httpMethod)}`]; if (queryParams.length > 0) { const entries = queryParams.map((name) => `${safeKey(name)}: ${inputRef(name)}`).join(", "); - options.push(`query: { ${entries} }`); + args.push(`query: { ${entries} }`); } if (bodyParams.length > 0) { if (bodyParams.length === 1 && bodyParams[0] === "body") { // A non-object request body arrives as a single "body" field. - options.push(`body: input.body`); + args.push(`body: input.body`); } else { const entries = bodyParams.map((name) => `${safeKey(name)}: ${inputRef(name)}`).join(", "); - options.push(`body: { ${entries} }`); + args.push(`body: { ${entries} }`); } } // The execute context's signal reaches fetch, so a cancelled call stops. - options.push("signal"); + args.push("signal"); + + return `${pathExpr}, { ${args.join(", ")} }`; +} - return `const data = await callApi(${pathExpr}, { ${options.join(", ")} });`; +function requestCall(tool: ReviewedTool): string { + if (!tool.httpMethod || !tool.pathTemplate || !tool.paramLocations) { + return `const data = null; // TODO: call your app's existing code here.`; + } + return `const data = await callApi(${buildCallExpr({ + httpMethod: tool.httpMethod, + pathTemplate: tool.pathTemplate, + paramLocations: tool.paramLocations, + serverUrl: tool.serverUrl, + })});`; +} + +/** "uploadId" on the first response: dot access when the name allows it. */ +function firstResultRef(param: string): string { + return /^[a-zA-Z_$][a-zA-Z0-9_$]*$/.test(param) + ? `firstResult.${param}` + : `firstResult[${JSON.stringify(param)}]`; +} + +/** + * The two calls of a grouped handshake tool: the first request runs, its + * response fields fill the second request's path params by exact name, and + * the second response is the tool's result. Threaded fields never reach the + * agent-facing input — that's the point of the composition. + */ +function composedFetchBody(plan: NonNullable): string[] { + const threaded = new Set(Object.keys(plan.threaded)); + return [ + ` const firstResult = (await callApi(${buildCallExpr(plan.first)})) as Record;`, + ` const data = await callApi(${buildCallExpr({ + ...plan.second, + pathRef: (param) => (threaded.has(param) ? firstResultRef(param) : inputRef(param)), + skipFields: threaded, + })});`, + ` return data;`, + ]; } /** diff --git a/packages/codegen/src/pipeline.ts b/packages/codegen/src/pipeline.ts index d75b888..73347e0 100644 --- a/packages/codegen/src/pipeline.ts +++ b/packages/codegen/src/pipeline.ts @@ -10,6 +10,7 @@ import { mkdir, readFile, writeFile } from "node:fs/promises"; import { dirname, resolve } from "node:path"; import { describeCandidateInputs, describeCandidateTool } from "./describe.js"; +import { groupHandshakes } from "./group.js"; import { pascalCase } from "./json-schema.js"; import { runLlmLayer } from "./llm.js"; import { mergeSchemaWithOperations } from "./merge.js"; @@ -131,6 +132,14 @@ export async function runGenerate( progress(`Renamed ${renames.length} tool${renames.length === 1 ? "" : "s"} for uniqueness`); } + // 4.5 Grouping: handshake endpoints (request-upload + complete-upload) are + // one action the API split into two calls. The merged tool is a + // withheld draft exactly like its members; the report names the + // proposal, and adopting it is enabling it. Members stay untouched. + const grouped = groupHandshakes(named); + notes.push(...grouped.notes); + const groupedTools = grouped.tools; + // Cross-run renames. The route ref is the durable identity; the name is // derived. When they drift apart (a better algorithm, a spec edit), the // tool's dashboard overrides are keyed by the old name and would silently @@ -139,7 +148,7 @@ export async function runGenerate( const crossRenames: { from: string; to: string }[] = []; if (options.previousNames) { const nameByRef = new Map(Object.entries(options.previousNames).map(([n, r]) => [r, n])); - for (const tool of named) { + for (const tool of groupedTools) { const before = nameByRef.get(refOf(tool)); if (before && before !== tool.name) crossRenames.push({ from: before, to: tool.name }); } @@ -159,12 +168,12 @@ export async function runGenerate( `${crossRenames.length} tool${crossRenames.length === 1 ? "" : "s"} renamed since the last run; their dashboard edits moved with them`, ); } - const namesLedger = Object.fromEntries(named.map((tool) => [tool.name, refOf(tool)])); + const namesLedger = Object.fromEntries(groupedTools.map((tool) => [tool.name, refOf(tool)])); // 5. Safety review: classify side effects, compute hints, scan for PII, // apply endpoint roles and config exclusions. Webhooks never come back. progress("Reviewing safety (classification, PII, auth)"); - const { tools, skipped } = reviewTools(named, config.safety); + const { tools, skipped } = reviewTools(groupedTools, config.safety); const authCount = tools.filter((t) => t.endpointRole === "auth").length; const adminCount = tools.filter((t) => t.endpointRole === "admin").length; if (authCount > 0) progress(`Disabled ${authCount} auth endpoint${authCount === 1 ? "" : "s"}`); diff --git a/packages/codegen/src/types.ts b/packages/codegen/src/types.ts index 4ebdd2e..f5c274e 100644 --- a/packages/codegen/src/types.ts +++ b/packages/codegen/src/types.ts @@ -93,6 +93,25 @@ export interface CandidateTool { * the route carries the auth/admin/destructive signal. */ endpointRef?: string; + /** + * For a grouped handshake tool: the two calls and how the first response + * feeds the second's inputs. Absent for ordinary single-endpoint tools. + */ + compose?: { + first: { + httpMethod: NonNullable; + pathTemplate: string; + paramLocations: { path: string[]; query: string[]; body: string[] }; + serverUrl?: string; + }; + second: { + httpMethod: NonNullable; + pathTemplate: string; + paramLocations: { path: string[]; query: string[]; body: string[] }; + serverUrl?: string; + }; + threaded: Record; + }; /** * Present when the tool annotates a literal
component instead of * generating a .webmcp.ts file. Set by the schema source from the entry's From f024cc0a42622484b14a3a91a920f2a519c650b6 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 03:26:34 +0530 Subject: [PATCH 13/41] Remove the LLM layer (--llm, --suggest) The user's own agent with the skill file covers what the layer proposed; a second model path in the CLI was weight that fought the lean direction. Deletes llm.ts, both flags and their provider-resolution flow, the config llm options, the suggestions field and rendering, and the suggest-only helpers (findSchemaModules, schemaExportsToJson, importSchemaModule). Plain generate is the whole story: deterministic always. --- packages/codegen/README.md | 1 - packages/codegen/src/cli-output.ts | 21 -- packages/codegen/src/cli.ts | 202 +------------ packages/codegen/src/describe.ts | 4 +- packages/codegen/src/detect-app.ts | 38 --- packages/codegen/src/llm.test.ts | 252 ---------------- packages/codegen/src/llm.ts | 398 ------------------------- packages/codegen/src/pipeline.ts | 15 - packages/codegen/src/sources/schema.ts | 34 --- packages/codegen/src/types.ts | 52 ---- site/content/docs/cli.mdx | 36 --- 11 files changed, 6 insertions(+), 1047 deletions(-) delete mode 100644 packages/codegen/src/llm.test.ts delete mode 100644 packages/codegen/src/llm.ts diff --git a/packages/codegen/README.md b/packages/codegen/README.md index 4cd1bcd..d6eaf56 100644 --- a/packages/codegen/README.md +++ b/packages/codegen/README.md @@ -119,7 +119,6 @@ export async function executeDeletePet(input: DeletePetInput) { | `generate --dry-run` | Preview everything, write nothing | | `generate --watch` | Re-generate when source files change | | `generate --force` | Write files even when the audit reports errors | -| `generate --suggest PATH` | Ask the LLM layer which schemas in PATH are worth declaring (proposals only) | | `generate --spec PATH` / `--out DIR` | Overrides without a config file | | `webmcp-codegen dev` | Open the tools dashboard (`--port N` to change the port) | | `webmcp-codegen init` | Write `codegen.config.mjs` for full control (needs the package installed) | diff --git a/packages/codegen/src/cli-output.ts b/packages/codegen/src/cli-output.ts index b37a358..93eae6e 100644 --- a/packages/codegen/src/cli-output.ts +++ b/packages/codegen/src/cli-output.ts @@ -239,18 +239,6 @@ export function renderSummary( } console.log(""); - // LLM proposals: visually distinct from findings (◦, cyan), because a - // suggestion is not a fact. Nothing here was applied to anything. - if (result.suggestions.length > 0) { - console.log( - ` ${c.cyan("◦")} ${bold(`${result.suggestions.length} LLM suggestion${result.suggestions.length === 1 ? "" : "s"}`)} ${dim("(proposals only; nothing applied)")}`, - ); - for (const suggestion of result.suggestions) { - console.log(dim(` ◦ ${suggestion.message}`)); - } - console.log(""); - } - // Next step console.log(` ${bold("Next:")} ${c.cyan("npx @webmcp-stack/codegen dev")}`); console.log(dim(" Review your tools, edit descriptions, test them live")); @@ -349,15 +337,6 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string } } - // LLM proposals, visually distinct from findings. - if (result.suggestions.length > 0) { - console.log(bold("LLM suggestions (proposals only; nothing applied):")); - for (const suggestion of result.suggestions) { - console.log(` ${c.cyan("◦")} ${dim(suggestion.message)}`); - } - console.log(""); - } - const notes = result.files.flatMap((file) => file.notes ?? []); if (notes.length > 0) { console.log(bold("File notes:")); diff --git a/packages/codegen/src/cli.ts b/packages/codegen/src/cli.ts index d7e6aa7..3ef978a 100644 --- a/packages/codegen/src/cli.ts +++ b/packages/codegen/src/cli.ts @@ -21,17 +21,16 @@ import { existsSync } from "node:fs"; import { readdir, readFile, writeFile } from "node:fs/promises"; import { join, resolve } from "node:path"; -import { pathToFileURL } from "node:url"; import { parseArgs } from "node:util"; import * as clack from "@clack/prompts"; import { dim, printBanner, renderSummary } from "./cli-output.js"; -import { CONFIG_FILE_NAMES, loadConfig } from "./config.js"; +import { CONFIG_FILE_NAMES } from "./config.js"; import { loadDataFile, saveDataFile } from "./data-file.js"; import { findSpecs } from "./detect.js"; -import { findSchemaLibraries, findSchemaModules, findWebApps } from "./detect-app.js"; +import { findSchemaLibraries, findWebApps } from "./detect-app.js"; // Node prints an ExperimentalWarning the first time it type-strips a .ts -// module (which --suggest does to load user schemas). That warning is Node +// module (loading a user's codegen.config.ts or schema modules). That warning is Node // talking about its own internals, not something the developer can act on, // so it never belongs in our output. const realEmitWarning = process.emitWarning.bind(process); @@ -43,12 +42,9 @@ process.emitWarning = ((warning: unknown, ...rest: unknown[]) => { }) as typeof process.emitWarning; import { startDevServer } from "./dev/server.js"; -import { hostedLlmProvider, resolveLlmProvider, runLlmLayer } from "./llm.js"; import { debug, enableVerbose, error, info, success, warn } from "./logger.js"; import { runGenerate } from "./pipeline.js"; import { resolveSetup } from "./setup.js"; -import { schemaExportsToJson } from "./sources/schema.js"; -import type { CodegenConfig } from "./types.js"; import { type JourneyFileInput, verifyJourneyFiles, verifyTools, verifyUrl } from "./verify.js"; import { applyWiring, planWiring, type WirePlan } from "./wire.js"; @@ -71,8 +67,6 @@ Flags --verbose Show every tool, not just the summary --force Write files even when the audit reports errors --skip-audit Skip the safety report - --suggest Ask the LLM layer which of your schemas are worth declaring - --llm Improve the names and descriptions of the tools being generated (LLM) --url URL With verify: also check the deployed page is live for visitors --config PATH Use a config file at PATH --port N Dashboard port (default: 4700) @@ -98,10 +92,6 @@ export interface CliFlags { spec?: string; out?: string; port?: number; - /** `generate --llm`: polish the generated tools' descriptions and names. */ - llm?: boolean; - /** `generate --suggest`: LLM proposals for undeclared schemas. */ - suggest?: boolean; /** `verify --url `: also check the page is live for visitors. */ url?: string; help: boolean; @@ -121,8 +111,6 @@ async function main(): Promise { spec: { type: "string" }, out: { type: "string" }, port: { type: "string" }, - suggest: { type: "boolean" }, - llm: { type: "boolean" }, url: { type: "string" }, help: { type: "boolean", default: false }, }, @@ -138,8 +126,6 @@ async function main(): Promise { spec: values.spec, out: values.out, port: values.port ? Number.parseInt(values.port, 10) : undefined, - suggest: values.suggest, - llm: values.llm, url: values.url, help: values.help, }; @@ -161,7 +147,7 @@ async function main(): Promise { case "verify": return verify(flags); case "generate": - return flags.suggest ? suggest() : generate(flags); + return generate(flags); default: error(`Unknown command: ${command}`); info(HELP); @@ -318,161 +304,6 @@ async function dev(port: number): Promise { return 0; } -/** - * Resolve the LLM provider for an explicit opt-in (--llm / --suggest). With a - * configured key this returns it silently; without one, an interactive - * terminal offers the hosted tier / own key / skip. Non-interactive runs - * (CI) get undefined, because a prompt can never block a pipeline. Both LLM - * flags share this so the chooser is identical everywhere. - */ -async function resolveProviderForOptIn( - llm: CodegenConfig["llm"], -): Promise> { - const configured = resolveLlmProvider(llm ?? {}); - if (configured) return configured; - if (!process.stdout.isTTY) { - info( - "\n◦ The LLM layer is off: no provider configured and no WEBMCP_LLM_API_KEY / " + - "OPENAI_API_KEY in the environment. Nothing proposed.\n", - ); - return undefined; - } - const choice = await clack.select({ - message: "The LLM layer needs an API key. How do you want to proceed?", - options: [ - { - value: "hosted", - label: "Use the free hosted tier", - hint: "webmcp-stack's shared key, rate-limited", - }, - { - value: "own", - label: "Enter my own API key", - hint: "OpenRouter, OpenAI, or any OpenAI-compatible provider", - }, - { value: "skip", label: "Skip", hint: "run without LLM features" }, - ], - }); - if (clack.isCancel(choice) || choice === "skip") { - info("\n◦ Skipped. Nothing proposed.\n"); - return undefined; - } - if (choice === "hosted") return hostedLlmProvider(); - const key = await clack.password({ - message: "Paste your API key (input is hidden):", - validate: (value) => - !value || value.trim().length === 0 ? "The key cannot be empty." : undefined, - }); - if (clack.isCancel(key)) { - info("\n◦ Skipped. Nothing proposed.\n"); - return undefined; - } - const provider = resolveLlmProvider({ ...llm, apiKey: key.trim() }); - if (!provider) { - warn("\nThat key did not resolve to a provider. Nothing proposed.\n"); - return undefined; - } - info(dim(" Key used for this run only. To save it: export WEBMCP_LLM_API_KEY=...")); - return provider; -} - -/** - * `generate --suggest`: the LLM layer's tool-worthiness proposals. The tool - * finds the schema modules itself — nobody should have to pass a file path - * to get proposals. A proposal surface only: it reads the discovered schemas, - * asks, and prints. Nothing is declared, generated, or written; declaring is - * the developer's edit. - */ -async function suggest(): Promise { - printBanner(); - const cwd = process.cwd(); - - // The llm settings live in the config when there is one. A missing config is - // fine here: --suggest is itself the explicit opt-in, so env keys are enough. - let llm: CodegenConfig["llm"]; - try { - llm = (await loadConfig(cwd)).config.llm; - } catch { - llm = undefined; - } - - const modules = await findSchemaModules(cwd); - if (modules.length === 0) { - info( - "\n◦ No schema modules found. The tool looks for files named like " + - '"schemas.ts" or "models.ts" under packages/, src/, apps/, or lib/. ' + - "If yours live elsewhere, declare them directly in codegen.config.mjs.\n", - ); - return 0; - } - const provider = await resolveProviderForOptIn(llm); - if (!provider) return 0; - - const allSchemas: { name: string; schemaText: string }[] = []; - for (const modulePath of modules) { - let moduleExports: Record; - try { - moduleExports = await importSchemaModule(cwd, modulePath); - } catch (error) { - debug(`could not load ${modulePath}: ${error instanceof Error ? error.message : error}`); - continue; - } - const { schemas, skipped } = schemaExportsToJson(moduleExports, cwd); - for (const entry of skipped) { - debug(`skipped ${entry.name}: ${entry.reason}`); - } - allSchemas.push(...schemas); - } - if (allSchemas.length === 0) { - info("\n◦ Found schema modules but no loadable schemas in them. Nothing to propose on.\n"); - return 0; - } - - const spinner = clack.spinner(); - spinner.start("Asking the LLM which schemas are worth declaring"); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: llm ?? {} }, - { tools: [], findings: [], suggestExports: allSchemas }, - ); - spinner.stop("Done"); - - info(""); - if (suggestions.length === 0) { - info(" ◦ The provider had no proposals. Declare schemas by hand, as usual."); - } - for (const suggestion of suggestions) { - info(` ◦ ${suggestion.message}`); - } - info( - dim("\n Proposals only; nothing was written. Declare what you want in codegen.config.mjs.\n"), - ); - return 0; -} - -/** - * Load a user module for --suggest. TypeScript loads only via Node's native - * type stripping (22.18+ / 23.6+): the leaning choice from the spec, kept as - * the single code path so the CLI stays zero-dependency. The tradeoff lives - * here on purpose: older runtimes and extensionless barrel imports get a - * clear, actionable error instead of a second loader. - */ -async function importSchemaModule( - cwd: string, - modulePath: string, -): Promise> { - const absolute = resolve(cwd, modulePath); - try { - return (await import(pathToFileURL(absolute).href)) as Record; - } catch (error) { - throw new Error( - `Could not load "${modulePath}". If it is TypeScript, run on Node 22.18+ (or 23.6+) ` + - "and import the schema file directly (explicit .ts extension; extensionless barrel " + - "re-exports need a bundler), or point --suggest at a plain-JS module.\n" + - `Underlying error: ${error instanceof Error ? error.message : String(error)}`, - ); - } -} - /** * The tool standard, measured locally: runs the pipeline exactly as generate * would (nothing written), then reports how the registered surface holds up @@ -655,31 +486,6 @@ async function generate(flags: CliFlags): Promise { }); } - // The LLM layer, on explicit opt-in only: polish the tools this run - // generated. Proposals print as `◦` lines; nothing is auto-applied, exit - // codes never change, and a failing provider is a note, not a failure. - if (flags.llm) { - const provider = await resolveProviderForOptIn(setup.config.llm); - if (provider) { - const spinner = clack.spinner(); - spinner.start("Improving names and descriptions"); - const suggestions = await runLlmLayer( - setup.config, - { tools: result.tools, findings: result.findings }, - undefined, - provider, - ); - spinner.stop("Done"); - if (suggestions.length === 0) { - info(" ◦ The provider had no proposals."); - } - for (const suggestion of suggestions) { - info(` ◦ ${suggestion.message}`); - } - info(""); - } - } - if (!flags.verbose) { renderSummary(result, setup, cwd, wiring); } diff --git a/packages/codegen/src/describe.ts b/packages/codegen/src/describe.ts index 0faddf2..f2fda75 100644 --- a/packages/codegen/src/describe.ts +++ b/packages/codegen/src/describe.ts @@ -21,8 +21,8 @@ * audit is only meaningful if a CI run is reproducible. * * This module covers layers 1-3 of the assembly order (source text, merge, - * synthesis). Layer 4 (LLM drafts) is advisory and lives in llm.ts; layer 5 - * (overrides) lives in the pipeline's override step, applied last so it wins. + * synthesis). Overrides live in the pipeline's override step, applied last + * so they always win. */ import pluralize from "pluralize"; diff --git a/packages/codegen/src/detect-app.ts b/packages/codegen/src/detect-app.ts index d113d03..77a9170 100644 --- a/packages/codegen/src/detect-app.ts +++ b/packages/codegen/src/detect-app.ts @@ -23,44 +23,6 @@ export interface WebApp { framework: "next" | "vite-react" | "nuxt" | "sveltekit" | "unknown"; } -/** - * Schema-module discovery for --suggest, mirroring findSpecs. Nobody should - * have to pass a file path to get proposals: we scan for modules that look - * like schema definitions, in the places they conventionally live. Returns - * module paths relative to cwd, likeliest first. - */ -const SCHEMA_MODULE_NAMES = /schemas?|models?|types|validation/i; -const SKIP_DIRS = new Set(["node_modules", "dist", ".git", ".next", "build", "out", "coverage"]); - -export async function findSchemaModules(cwd: string): Promise { - const found: string[] = []; - // Conventional homes first, so a monorepo's shared package wins over a - // deeply nested app file. - for (const base of ["packages", "src", "apps", "lib", "."]) { - await scanDir(join(cwd, base), base === "." ? "" : base, 0); - } - return [...new Set(found)]; - - async function scanDir(abs: string, rel: string, depth: number): Promise { - if (depth > 4) return; - let entries: import("node:fs").Dirent[]; - try { - entries = await readdir(abs, { withFileTypes: true }); - } catch { - return; - } - for (const entry of entries) { - if (entry.name.startsWith(".") || SKIP_DIRS.has(entry.name)) continue; - const entryRel = rel ? `${rel}/${entry.name}` : entry.name; - if (entry.isDirectory()) { - await scanDir(join(abs, entry.name), entryRel, depth + 1); - } else if (/\.(ts|mts|js|mjs)$/.test(entry.name) && SCHEMA_MODULE_NAMES.test(entry.name)) { - found.push(entryRel); - } - } - } -} - /** The validation libraries the schema source can read. */ const SCHEMA_LIBS = ["zod", "valibot", "arktype", "typebox"] as const; diff --git a/packages/codegen/src/llm.test.ts b/packages/codegen/src/llm.test.ts deleted file mode 100644 index 4b271ac..0000000 --- a/packages/codegen/src/llm.test.ts +++ /dev/null @@ -1,252 +0,0 @@ -import { describe, expect, it } from "vitest"; -import { hostedLlmProvider, resolveLlmProvider, runLlmLayer } from "./llm.js"; -import type { LlmProvider, ReviewedTool } from "./types.js"; - -/** A provider that records what it was asked and answers per task. */ -function mockProvider(answers: Record = {}) { - const calls: { task: string; prompt: string }[] = []; - const provider: LlmProvider = { - name: "mock", - async complete(task, prompt) { - calls.push({ task, prompt }); - return answers[task] ?? "{}"; - }, - }; - return { provider, calls }; -} - -function tool(overrides: Partial = {}): ReviewedTool { - return { - id: "GET /x", - name: "get-x", - source: { kind: "openapi", ref: "GET /x" }, - inputSchema: { type: "object", properties: {}, required: [] }, - inputTypeName: "GetXInput", - httpMethod: "GET", - sideEffect: "read", - endpointRole: "endpoint", - enabledByDefault: true, - withheld: false, - requiresAuth: false, - riskTier: "safe-read", - hints: { - readOnlyHint: true, - destructiveHint: false, - idempotentHint: true, - untrustedContentHint: false, - }, - piiInOutput: [], - description: "A fine description", - descriptionSource: "openapi-summary", - ...overrides, - }; -} - -describe("resolveLlmProvider", () => { - it("is off without the config key, even with an env key set", () => { - // An OPENAI_API_KEY in the environment must not turn a generate run into - // a network call nobody asked for; the config key is the opt-in. - expect(resolveLlmProvider(undefined, { OPENAI_API_KEY: "sk-test" })).toBeUndefined(); - }); - - it("uses the custom provider when one is given", () => { - const { provider } = mockProvider(); - expect(resolveLlmProvider({ provider })?.name).toBe("mock"); - }); - - it("builds the built-in provider from a config key, then env keys", () => { - expect(resolveLlmProvider({ apiKey: "sk-config" }, {})?.name).toContain("openai-compatible"); - expect(resolveLlmProvider({}, { WEBMCP_LLM_API_KEY: "sk-env" })?.name).toContain( - "openai-compatible", - ); - expect(resolveLlmProvider({}, { OPENAI_API_KEY: "sk-env" })?.name).toContain( - "openai-compatible", - ); - expect(resolveLlmProvider({}, {})).toBeUndefined(); - }); - - it("the hosted tier points at our proxy, key never in the package", async () => { - const provider = hostedLlmProvider(); - expect(provider.name).toContain("openai-compatible"); - // The wire call goes to our site; the bearer is a placeholder the proxy ignores. - let seenUrl = ""; - let seenAuth = ""; - const fetchSpy = async (input: RequestInfo | URL, init?: RequestInit) => { - seenUrl = String(input); - seenAuth = String((init?.headers as Record)?.authorization ?? ""); - return new Response(JSON.stringify({ choices: [{ message: { content: "{}" } }] }), { - status: 200, - }); - }; - const realFetch = globalThis.fetch; - globalThis.fetch = fetchSpy as typeof fetch; - try { - await provider.complete("describe", "prompt", "system"); - } finally { - globalThis.fetch = realFetch; - } - expect(seenUrl).toBe("https://webmcp-stack.vercel.app/api/llm/chat/completions"); - expect(seenAuth).toBe("Bearer hosted"); - }); -}); - -describe("runLlmLayer", () => { - it("proposes drafts only for machine-written text, never author text", async () => { - const { provider, calls } = mockProvider({ - describe: JSON.stringify({ - description: "Create a trip and open it in the editor.", - fields: { minutes: "Minutes worked on this task. A number from 30 to 600." }, - }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - tools: [ - tool({ name: "well-described" }), // author text: never sent - tool({ - name: "create-trip", - description: "create-trip", - descriptionSource: "generated-template", - synthesizedFields: ["minutes"], - }), - ], - findings: [], - }, - ); - - // Only the needy tool was asked about. - expect(calls.filter((call) => call.task === "describe")).toHaveLength(1); - expect(calls[0]?.prompt).toContain("create-trip"); - expect(suggestions.some((s) => s.message.includes("Create a trip and open it"))).toBe(true); - expect(suggestions.some((s) => s.field === "minutes" && s.message.includes("30 to 600"))).toBe( - true, - ); - }); - - it("suggests a producer for fields the audit flagged as unproducible", async () => { - const { provider } = mockProvider({ - relationship: JSON.stringify({ producer: "search-places" }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - tools: [tool({ name: "create-trip" }), tool({ name: "search-places" })], - findings: [ - { - level: "warning", - tool: "create-trip", - message: - 'Field "locationObject" says it comes from the "search-places" tool, but no such tool exists in this run.', - }, - ], - }, - ); - expect(suggestions.some((s) => s.message.includes('"search-places"'))).toBe(true); - }); - - it("never names a producer that does not exist in the run", async () => { - const { provider } = mockProvider({ - relationship: JSON.stringify({ producer: "hallucinated-tool" }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - tools: [tool({ name: "create-trip" })], - findings: [ - { - level: "warning", - tool: "create-trip", - message: 'Field "locationObject" ... but no such tool exists in this run.', - }, - ], - }, - ); - expect(suggestions.filter((s) => s.task === "relationship")).toHaveLength(0); - }); - - it("surfaces semantic contradictions as suggestions, not findings", async () => { - const { provider } = mockProvider({ - "semantic-review": JSON.stringify({ - contradictions: [{ tool: "get-x", issue: 'Says "fetch" but the schema writes.' }], - }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { tools: [tool({ name: "get-x" })], findings: [] }, - ); - expect( - suggestions.some((s) => s.task === "semantic-review" && s.message.includes("fetch")), - ).toBe(true); - }); - - it("proposes tools for --suggest exports", async () => { - const { provider } = mockProvider({ - suggest: JSON.stringify({ - tools: [{ name: "create-trip", description: "Create a trip.", write: true }], - }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - tools: [], - findings: [], - suggestExports: [{ name: "CreateTripInput", schemaText: '{"type":"object"}' }], - }, - ); - expect(suggestions.some((s) => s.task === "suggest" && s.message.includes("create-trip"))).toBe( - true, - ); - }); - - it("asks each distinct question once per run, however many tools share it", async () => { - const { provider, calls } = mockProvider({ - describe: JSON.stringify({ description: "Draft." }), - }); - const twin = { - description: "same", - descriptionSource: "generated-template" as const, - synthesizedFields: [] as string[], - inputSchema: { type: "object", properties: {}, required: [] }, - }; - await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - // Two tools with byte-identical questions hash to one provider call. - tools: [tool({ name: "same", ...twin }), tool({ name: "same", ...twin })], - findings: [], - }, - ); - expect(calls.filter((call) => call.task === "describe")).toHaveLength(1); - }); - - it("turns a provider failure into a note, never a run failure", async () => { - const broken: LlmProvider = { - name: "broken", - async complete() { - throw new Error("HTTP 500"); - }, - }; - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider: broken } }, - { - tools: [tool({ name: "t", description: "t", descriptionSource: "generated-template" })], - findings: [], - }, - ); - expect( - suggestions.some((s) => s.message.includes("failed") && s.message.includes("unaffected")), - ).toBe(true); - }); - - it("parses completions that wrap JSON in prose or fences", async () => { - const { provider } = mockProvider({ - "semantic-review": 'Here you go:\n```json\n{"contradictions": []}\n```\nHope that helps!', - }); - // No throw, no suggestions, no drama. - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { tools: [tool()], findings: [] }, - ); - expect(suggestions.filter((s) => s.task === "semantic-review")).toHaveLength(0); - }); -}); diff --git a/packages/codegen/src/llm.ts b/packages/codegen/src/llm.ts deleted file mode 100644 index 4f4df7a..0000000 --- a/packages/codegen/src/llm.ts +++ /dev/null @@ -1,398 +0,0 @@ -/** - * The LLM layer: advisory, always optional, never load-bearing. - * - * Why this file is shaped the way it is: the codegen's trust model rests on - * the audit being reproducible. Same input, same findings, same exit codes, - * on a laptop or in CI. A model breaks that, so the boundary here is a hard - * rule, not a guideline: - * - * THE MODEL PROPOSES. THE DEVELOPER DISPOSES. - * - * Concretely, in this version: - * - proposals surface only as `◦` report lines. Nothing is written to a - * file, because the acceptance surface (dashboard accept/reject) is a - * deliberate follow-up; without a place to dispose, nothing is applied. - * - the layer never classifies side effects or risk tiers, never changes - * exit codes, and never blocks a run. A missing key means off; a failing - * provider means one warning line and the deterministic run continues. - * - it runs only on what is still machine-written after the deterministic - * layers (template tool descriptions, synthesized field text). Author - * text is never sent back for "improvement". - * - * The prompts below are the shipped defaults, benchmarked against the WebMCP - * docs' examples. `llm.prompts` in the config overrides them per task: the - * developer owns the words going into their agent's prompt. - */ - -import { createHash } from "node:crypto"; -import type { - AuditFinding, - CodegenConfig, - LlmOptions, - LlmProvider, - LlmSuggestion, - LlmTask, - ReviewedTool, -} from "./types.js"; - -/** What the layer needs from the rest of the pipeline. */ -export interface LlmContext { - tools: ReviewedTool[]; - findings: AuditFinding[]; - /** Module exports for the `suggest` task: name + JSON Schema text. */ - suggestExports?: { name: string; schemaText: string }[]; -} - -const DEFAULT_PROMPTS: Record = { - describe: [ - "You write WebMCP tool descriptions. Agents pick tools by these words and fill", - "inputs from them, so be concrete, imperative, and one sentence per item. State", - "what the tool does and what each field means; mention constraints you are given.", - 'Answer with JSON only: { "description"?: string, "fields": { "": string } }.', - ].join(" "), - relationship: [ - "You are given a tool field whose description says its value comes from another", - "tool, plus the list of tools that exist. Name the likeliest producer tool, or", - 'null if none fits. Answer with JSON only: { "producer": string | null }.', - ].join(" "), - "semantic-review": [ - "You review WebMCP tool descriptions against their input schemas. Flag only real", - 'contradictions (a "fetch" description on a writing schema, a field described as', - "the opposite of its type). Silence is a good outcome. Answer with JSON only:", - '{ "contradictions": [{ "tool": string, "issue": string }] }.', - ].join(" "), - suggest: [ - "You are given exported validation schemas from a web app. Propose which should", - "become agent-facing WebMCP tools: user-meaningful actions, not plumbing. For each,", - "name (kebab-case, verb-first), one-sentence description, and whether it writes.", - 'Answer with JSON only: { "tools": [{ "name": string, "description": string, "write": boolean }] }.', - ].join(" "), -}; - -/** - * Resolve the configured provider, or undefined (layer off). Explicit opt-in - * is the config key: an OPENAI_API_KEY in the environment must not turn a - * `generate` run into a network call the developer never asked for. - * (`generate --suggest` is the explicit command, and honors env keys.) - */ -export function resolveLlmProvider( - options: LlmOptions | undefined, - env: NodeJS.ProcessEnv = process.env, -): LlmProvider | undefined { - if (!options) return undefined; - if (options.provider) return options.provider; - const apiKey = options.apiKey ?? env.WEBMCP_LLM_API_KEY ?? env.OPENAI_API_KEY; - if (!apiKey) return undefined; - return openAiCompatibleProvider( - apiKey, - options.baseUrl ?? "https://api.openai.com/v1", - options.model ?? "gpt-4o-mini", - ); -} - -/** - * The free hosted tier: our own proxy holds the key server-side, so a - * developer with no provider account can still try the LLM layer. Rate - * limits live on the proxy; the CLI just points at it. The proxy ignores - * the bearer — the real key never leaves the server. - */ -export const HOSTED_LLM_URL = "https://webmcp-stack.vercel.app/api/llm"; - -export function hostedLlmProvider(): LlmProvider { - return openAiCompatibleProvider("hosted", HOSTED_LLM_URL, "openai/gpt-4o-mini"); -} - -/** - * The built-in provider: any OpenAI-compatible chat-completions endpoint, - * via plain fetch. Zero dependencies is a feature of this package, so the - * wire format is spelled out rather than imported. - */ -function openAiCompatibleProvider(apiKey: string, baseUrl: string, model: string): LlmProvider { - return { - name: `openai-compatible (${model})`, - async complete(task, prompt, system) { - const response = await fetch(`${baseUrl}/chat/completions`, { - method: "POST", - headers: { - "content-type": "application/json", - authorization: `Bearer ${apiKey}`, - }, - body: JSON.stringify({ - model, - messages: [ - // The developer's per-task override wins over the shipped default; - // config.llm.prompts exists so teams can tune these. - { role: "system", content: system ?? DEFAULT_PROMPTS[task] }, - { role: "user", content: prompt }, - ], - temperature: 0, - }), - // A hung endpoint must never hang a generate run. - signal: AbortSignal.timeout(30_000), - }); - if (!response.ok) { - throw new Error(`LLM endpoint answered HTTP ${response.status}`); - } - const body = (await response.json()) as { - choices?: { message?: { content?: string } }[]; - }; - const content = body.choices?.[0]?.message?.content; - if (!content) throw new Error("LLM endpoint returned no content"); - return content; - }, - }; -} - -/** - * In-run cache, keyed by content hash. Many fields share a shape (every - * "name" string in a spec), so identical questions are asked once per run. - * Deliberately not a disk cache: a cache file in the user's repo is noise, - * and stale-across-runs is a bug class this sidesteps entirely. - */ -function cached(provider: LlmProvider): LlmProvider { - const seen = new Map>(); - return { - name: provider.name, - complete(task, prompt, system) { - // The system prompt is part of the question's identity: the same - // question under two different overrides must not share an answer. - const key = createHash("sha256") - .update(task) - .update("\0") - .update(system ?? "") - .update("\0") - .update(prompt) - .digest("hex"); - let pending = seen.get(key); - if (!pending) { - pending = provider.complete(task, prompt, system); - seen.set(key, pending); - } - return pending; - }, - }; -} - -/** Extract the JSON object from a completion that may wrap it in prose. */ -function parseCompletionJson(text: string): Record | undefined { - const direct = tryParse(text); - if (direct) return direct; - const fenced = /```(?:json)?\s*(\{[\s\S]*?\})\s*```/.exec(text); - if (fenced) return tryParse(fenced[1] as string); - const braces = /\{[\s\S]*\}/.exec(text); - return braces ? tryParse(braces[0]) : undefined; -} - -function tryParse(text: string): Record | undefined { - try { - const value: unknown = JSON.parse(text); - return value && typeof value === "object" ? (value as Record) : undefined; - } catch { - return undefined; - } -} - -/** - * Run every enabled touchpoint and collect proposals. Never throws: a broken - * provider yields one suggestion line explaining the failure, and the rest of - * the run is exactly the deterministic one. - */ -export async function runLlmLayer( - config: CodegenConfig, - context: LlmContext, - env?: NodeJS.ProcessEnv, - /** When the caller already resolved a provider (the interactive key - * chooser), pass it so the layer does not re-resolve from config/env. */ - explicitProvider?: LlmProvider, -): Promise { - const configured = explicitProvider ?? resolveLlmProvider(config.llm, env); - if (!configured) return []; - const provider = cached(configured); - const prompts = { ...DEFAULT_PROMPTS, ...config.llm?.prompts }; - const suggestions: LlmSuggestion[] = []; - - const jobs: Promise[] = [ - proposeDescriptions(provider, prompts.describe, context.tools, suggestions), - proposeProducers(provider, prompts.relationship, context, suggestions), - reviewSemantics(provider, prompts["semantic-review"], context.tools, suggestions), - ]; - if (context.suggestExports && context.suggestExports.length > 0) { - jobs.push(proposeTools(provider, prompts.suggest, context.suggestExports, suggestions)); - } - - // Each job catches its own failures; this allSettled is the belt to their - // suspenders, because an LLM failure must never surface as a run failure. - await Promise.allSettled(jobs); - return suggestions; -} - -/** - * Draft prose for what the deterministic layers left machine-written: - * template tool descriptions and synthesized field text. Author text is - * never in scope. - */ -async function proposeDescriptions( - provider: LlmProvider, - prompt: string, - tools: ReviewedTool[], - suggestions: LlmSuggestion[], -): Promise { - const needy = tools.filter( - (tool) => - tool.descriptionSource === "generated-template" || (tool.synthesizedFields ?? []).length > 0, - ); - for (const tool of needy) { - const fields = (tool.synthesizedFields ?? []) - .map((name) => { - const schema = tool.inputSchema.properties?.[name]; - return `"${name}": ${JSON.stringify(schema)}`; - }) - .join(", "); - const question = - `Tool "${tool.name}" (currently described as "${tool.description}"). ` + - (fields ? `Fields needing text: { ${fields} }.` : "No fields need text."); - try { - const answer = parseCompletionJson(await provider.complete("describe", question, prompt)); - if (typeof answer?.description === "string" && answer.description.trim()) { - suggestions.push({ - task: "describe", - tool: tool.name, - message: `${tool.name}: draft description: "${answer.description.trim()}"`, - }); - } - const fieldDrafts = answer?.fields; - if (fieldDrafts && typeof fieldDrafts === "object") { - for (const [field, text] of Object.entries(fieldDrafts)) { - if (typeof text === "string" && text.trim()) { - suggestions.push({ - task: "describe", - tool: tool.name, - field, - message: `${tool.name}.${field}: draft: "${text.trim()}"`, - }); - } - } - } - } catch (error) { - suggestions.push(failureNote("describe", tool.name, error)); - return; // One failure is enough; hammering a broken endpoint helps nobody. - } - } -} - -/** - * Where the deterministic rule found a field referencing a producer tool that - * does not exist, ask the model whether an existing tool could produce it. - * A suggestion here is inference, so it is always a proposal, never a finding. - */ -async function proposeProducers( - provider: LlmProvider, - prompt: string, - context: LlmContext, - suggestions: LlmSuggestion[], -): Promise { - const gaps = context.findings.filter( - (finding) => finding.tool && finding.message.includes("no such tool"), - ); - const toolNames = context.tools.map((tool) => tool.name); - for (const gap of gaps) { - const question = - `Field in tool "${gap.tool}": ${gap.message}\n` + - `Existing tools: ${toolNames.join(", ") || "(none)"}.`; - try { - const answer = parseCompletionJson(await provider.complete("relationship", question, prompt)); - const producer = answer?.producer; - if (typeof producer === "string" && toolNames.includes(producer)) { - suggestions.push({ - task: "relationship", - tool: gap.tool, - message: `${gap.tool}: "${producer}" looks like it could produce that value. Link them?`, - }); - } - } catch (error) { - suggestions.push(failureNote("relationship", gap.tool, error)); - return; - } - } -} - -/** - * A second pair of eyes on author-written descriptions: contradictions with - * the schema only. Advisory even when enabled; nothing here blocks anything. - */ -async function reviewSemantics( - provider: LlmProvider, - prompt: string, - tools: ReviewedTool[], - suggestions: LlmSuggestion[], -): Promise { - const described = tools - .filter((tool) => tool.descriptionSource !== "generated-template") - .slice(0, 25); // Cost guard: one bounded call, not a novel per spec. - if (described.length === 0) return; - const question = described - .map((tool) => { - const fields = Object.keys(tool.inputSchema.properties ?? {}).join(", "); - return `"${tool.name}": "${tool.description}" (fields: ${fields})`; - }) - .join("\n"); - try { - const answer = parseCompletionJson( - await provider.complete("semantic-review", question, prompt), - ); - const contradictions = answer?.contradictions; - if (Array.isArray(contradictions)) { - for (const item of contradictions) { - const entry = item as { tool?: unknown; issue?: unknown }; - if (typeof entry.tool === "string" && typeof entry.issue === "string") { - suggestions.push({ - task: "semantic-review", - tool: entry.tool, - message: `${entry.tool}: ${entry.issue}`, - }); - } - } - } - } catch (error) { - suggestions.push(failureNote("semantic-review", undefined, error)); - } -} - -/** `generate --suggest `: which exported schemas are worth a tool? */ -async function proposeTools( - provider: LlmProvider, - prompt: string, - exports_: { name: string; schemaText: string }[], - suggestions: LlmSuggestion[], -): Promise { - const question = exports_.map((entry) => `${entry.name}: ${entry.schemaText}`).join("\n\n"); - try { - const answer = parseCompletionJson(await provider.complete("suggest", question, prompt)); - const proposed = answer?.tools; - if (Array.isArray(proposed)) { - for (const item of proposed) { - const entry = item as { name?: unknown; description?: unknown; write?: unknown }; - if (typeof entry.name === "string" && typeof entry.description === "string") { - suggestions.push({ - task: "suggest", - tool: entry.name, - message: - `${entry.name}: ${entry.description} ` + - `(${entry.write === true ? "write" : "read"}; declare it in codegen.config.mjs to generate)`, - }); - } - } - } - } catch (error) { - suggestions.push(failureNote("suggest", undefined, error)); - } -} - -/** A provider failure is reported as a suggestion-shaped note, never an error. */ -function failureNote(task: LlmTask, tool: string | undefined, error: unknown): LlmSuggestion { - return { - task, - tool, - message: `LLM ${task} failed (${error instanceof Error ? error.message : String(error)}). The deterministic run is unaffected.`, - }; -} diff --git a/packages/codegen/src/pipeline.ts b/packages/codegen/src/pipeline.ts index 73347e0..cb3dd0f 100644 --- a/packages/codegen/src/pipeline.ts +++ b/packages/codegen/src/pipeline.ts @@ -12,7 +12,6 @@ import { dirname, resolve } from "node:path"; import { describeCandidateInputs, describeCandidateTool } from "./describe.js"; import { groupHandshakes } from "./group.js"; import { pascalCase } from "./json-schema.js"; -import { runLlmLayer } from "./llm.js"; import { mergeSchemaWithOperations } from "./merge.js"; import { resolveNames } from "./naming.js"; import { auditTools, reviewTools } from "./safety.js"; @@ -20,7 +19,6 @@ import type { AuditFinding, CodegenConfig, GeneratedFile, - LlmSuggestion, ReviewedTool, SkippedEndpoint, ToolOverrides, @@ -60,11 +58,6 @@ export interface GenerateResult { files: GeneratedFile[]; /** Human-facing pipeline notes, e.g. "stripped the shared v1 prefix". */ notes: string[]; - /** - * Advisory proposals from the LLM layer (`◦` lines in the report). Empty - * unless the layer is explicitly configured; never applied to files. - */ - suggestions: LlmSuggestion[]; /** Names that changed since the last run (old → new), overrides re-keyed. */ crossRenames: { from: string; to: string }[]; /** The names this run produced (name → route ref), for the caller to save. */ @@ -264,7 +257,6 @@ export async function runGenerate( findings, files: [], notes, - suggestions: [], crossRenames, namesLedger, migratedOverrides, @@ -273,12 +265,6 @@ export async function runGenerate( }; } - // 7b. The advisory LLM layer runs after the audit so its relationship - // proposals can react to findings, and before outputs so a slow endpoint - // never sits between the developer and their files. It only proposes: - // report lines, never writes, never exit codes. - const suggestions = await runLlmLayer(config, { tools, findings }); - // 8. Run the outputs, then write the files (unless this is a dry run). // Tools with a form pointer belong to the form output; without one // configured they generate as ordinary tool files, loudly. @@ -325,7 +311,6 @@ export async function runGenerate( findings, files, notes, - suggestions, crossRenames, namesLedger, migratedOverrides, diff --git a/packages/codegen/src/sources/schema.ts b/packages/codegen/src/sources/schema.ts index 9b29a9c..471e96e 100644 --- a/packages/codegen/src/sources/schema.ts +++ b/packages/codegen/src/sources/schema.ts @@ -230,37 +230,3 @@ function toJsonSchema( ); } } - -/** - * `generate --suggest` support: filter a module's exports to Standard Schemas - * and convert each to JSON Schema text for the proposal prompt. Lives next to - * the source so the conversion rules exist exactly once. Exports that are not - * schemas are skipped silently (a module may export anything); exports that - * look like schemas but fail conversion are reported, never silent. - */ -export function schemaExportsToJson( - moduleExports: Record, - anchorDir: string, -): { - schemas: { name: string; schemaText: string }[]; - skipped: { name: string; reason: string }[]; -} { - const schemas: { name: string; schemaText: string }[] = []; - const skipped: { name: string; reason: string }[] = []; - for (const [name, value] of Object.entries(moduleExports)) { - // Standard Schema (zod/valibot/arktype) carries a `~standard` marker; - // TypeBox is a bare JSON-Schema object. Both are valid — this loader was - // the one place that only accepted the marker, which is why TypeBox - // modules silently yielded nothing. - if (!isStandardSchema(value) && !isTypeBoxSchema(value)) continue; - try { - schemas.push({ - name, - schemaText: JSON.stringify(toJsonSchema(value, name, "schema", anchorDir)), - }); - } catch (error) { - skipped.push({ name, reason: error instanceof Error ? error.message : String(error) }); - } - } - return { schemas, skipped }; -} diff --git a/packages/codegen/src/types.ts b/packages/codegen/src/types.ts index f5c274e..b0d686e 100644 --- a/packages/codegen/src/types.ts +++ b/packages/codegen/src/types.ts @@ -275,56 +275,4 @@ export interface CodegenConfig { sources: Source[]; outputs: Output[]; safety?: SafetyOptions; - /** - * The opt-in LLM layer. Absent means off: the run is then exactly the - * deterministic one. The layer only ever proposes (report lines); it never - * writes files, never classifies risk, and never changes exit codes. - */ - llm?: LlmOptions; -} - -/** The four things the LLM layer may propose on. */ -export type LlmTask = "describe" | "relationship" | "semantic-review" | "suggest"; - -/** - * A model backend. Bring your own to use any vendor, or configure a key and - * use the built-in OpenAI-compatible one. One method, because the layer asks - * one kind of question. - */ -export interface LlmProvider { - name: string; - /** - * Ask one question. `system` is the per-task prompt (the built-in default, - * or the developer's override from config); providers that ignore it fall - * back to whatever the task implies. - */ - complete(task: LlmTask, prompt: string, system?: string): Promise; -} - -export interface LlmOptions { - /** A custom provider. Wins over apiKey when both are set. */ - provider?: LlmProvider; - /** API key for the built-in provider. Falls back to env WEBMCP_LLM_API_KEY, then OPENAI_API_KEY. */ - apiKey?: string; - /** OpenAI-compatible base URL. Default: https://api.openai.com/v1 */ - baseUrl?: string; - /** Model name for the built-in provider. Default: gpt-4o-mini */ - model?: string; - /** Override the shipped prompt per task, e.g. to match your domain's voice. */ - prompts?: Partial>; -} - -/** - * One proposal from the LLM layer. Rendered as `◦` lines, visually apart from - * audit findings, because a suggestion is not a fact: the developer disposes. - * Nothing here is ever applied to a file in this version; the acceptance - * surface (dashboard accept/reject) is a deliberate follow-up. - */ -export interface LlmSuggestion { - task: LlmTask; - /** The tool (and field, when relevant) this proposal is about. */ - tool?: string; - field?: string; - /** The one-line proposal text for the report. */ - message: string; } diff --git a/site/content/docs/cli.mdx b/site/content/docs/cli.mdx index 9cb1cf4..86443ca 100644 --- a/site/content/docs/cli.mdx +++ b/site/content/docs/cli.mdx @@ -20,8 +20,6 @@ The main command. Resolves where tools come from and where they go, in this orde --force Write files even when the audit reports errors --config PATH Use a config file at PATH --watch Re-generate when files change - --llm Improve the names and descriptions of the tools being generated (LLM) - --suggest Find schemas worth declaring as tools (LLM) ``` A successful run also wires registration into your app (two additive lines; see @@ -42,40 +40,6 @@ Re-runs on every relevant file change. Regeneration is cheap and merge-safe, so keeps tools in sync while you edit the spec. The watcher ignores its own outputs (`src/webmcp`, `.webmcp-codegen.json`), so it never loops. -## The two LLM flags - -Both are **opt-in** — plain `generate` never touches a model, never sends anything anywhere, -never needs a key. Turn one on only when you want the LLM's help. They answer two different -questions: - -**`generate --llm`** — *"make the tools I already generate better."* -You have a spec (or a config) and tools generate. The LLM reads each tool's name, description, -and fields and proposes clearer descriptions and names, printed as `◦` draft lines for you to -review. Nothing is auto-applied; your hand-written text always wins; safety classification is -never the model's call. - -**`generate --suggest`** — *"find tools I haven't declared yet."* -You have validation schemas but no generated tools. The CLI finds your schema modules itself -(no file path needed) and the LLM proposes which schemas are worth declaring as tools. Nothing -is written; declaring is your edit to `codegen.config.mjs`. - -The rule of thumb: **tools already exist → `--llm`; starting from schemas → `--suggest`.** - -### The API key - -Neither flag requires your own key. The first time you run one without a key configured, an -interactive terminal offers three choices: - -- **Use the free hosted tier** — webmcp-stack's shared key (rate-limited), zero setup. The key - lives on our server, never in the package. -- **Enter my own API key** — OpenRouter, OpenAI, or any OpenAI-compatible provider. Used for - this command only, never stored, input is masked. -- **Skip** — continue without LLM features, exactly as if you'd passed neither flag. - -In CI (non-interactive), both flags quietly produce the deterministic output — a prompt never -blocks a pipeline, and a failing provider becomes one note line, never a failure. To use your -own key always, set `WEBMCP_LLM_API_KEY` or `OPENAI_API_KEY` in your environment. - ## `npx @webmcp-stack/codegen verify` Measures your generated tools against the quality standard they are supposed to meet, before From 0beff216bd74192c14a9acec9085410e9b59a43f Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 03:27:12 +0530 Subject: [PATCH 14/41] Mark the direction doc's order of work shipped All six steps landed on this branch: spec sync, budgets, skill + evals, journeys wiring, handshake grouping, LLM layer removal. The doc records what each step became, so the plan now reads as history with the parked items (audit package, dashboard report, URL audit) still named. --- .../2026-09-07-what-to-double-down-on.md | 41 +++++++++++-------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/docs/research/2026-09-07-what-to-double-down-on.md b/docs/research/2026-09-07-what-to-double-down-on.md index 4de9fd4..79c9802 100644 --- a/docs/research/2026-09-07-what-to-double-down-on.md +++ b/docs/research/2026-09-07-what-to-double-down-on.md @@ -322,23 +322,30 @@ confirmation gates, the naming rules, descriptions that say what a tool returns, nested untrusted-content marking, and `verify` with its scorecard and `--url` check. -What's left, in order: - -1. Spec sync quick wins (details: docs/research/2026-09-10-spec-sync.md): - emit `title`, auto-set `consequentialHint` on destructive tools, add the - `exposedTo` config pass-through, fix the Chrome 149 version claims in - three docs files. Template edits only. -2. Close the budget gaps: the 500/150 description limits in generation and - `verify`, and the 1.5K output-truncation helper in the generated region. -3. The skill file (`assets/skill/SKILL.md` — written, needs the scaffold - wiring) plus its eval harness. -4. Journeys — ship the `createJourney` helper (`assets/journey.webmcp.ts` — - written, needs the scaffold wiring) and the verify checks for journey - files. -5. The grouping step — intent-level tools by default, proposal in the - report. -6. Delete the LLM layer (`--llm`, `--suggest`, shipped in 0.8) once the - skill file has shipped and nobody has complained. +Shipped 2026-09-10 on the journeys feature branch (feat/journeys): + +1. ~~Spec sync quick wins~~ — `title` emitted everywhere (tools and journey + gates), `consequentialHint` on destructive tools, `exposedTo` config + pass-through on the tools output, Chrome 149/150 version fixes. +2. ~~Budgets~~ — generation composes within 500/150 (fitBudget, sentence + cuts), verify's new error-level Budgets check measures the final text, + the runtime caps every `toolResult` at ~1.5K with a truncation notice. +3. ~~Skill file~~ — scaffolded at `.agents/skills/webmcp-tools/SKILL.md` + (cross-client location, Claude Code included), regenerated like the + runtime; eval harness at `packages/codegen/evals/skill/` with the + beenthere-lite fixture, nine cases, deterministic graders, and a passing + self-test (`node run.mjs --selftest`). +4. ~~Journeys~~ — `journey.webmcp.ts` scaffolded next to the runtime, the + barrel registers every `/journeys/*.webmcp.ts` it finds, + endpoint-backed tools emit the `fetchX` raw caller that steps compose, + and verify lints journey files (gate, budgets, step count, no raw fetch). +5. ~~Grouping~~ — `groupHandshakes` merges POST handshake pairs + (request-upload + complete-upload → upload-media) with exact-name + threading only; the merged tool is a withheld draft, members untouched, + the CLI prints the proposal line. +6. ~~LLM layer~~ — deleted: `--llm`, `--suggest`, the config options, the + provider flow, ~1,050 lines gone. The skill file is how rules reach + models now. Parked (out of scope for now): the audit-package extraction, the dashboard-as-report rework, and the paste-a-URL audit. From 4564f4fdef6d6b5ee8c4d8a3c83e60aa8ff437fd Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 17:04:09 +0530 Subject: [PATCH 15/41] Create 2026-09-11-pr-4-intent-surface.md --- .../reviews/2026-09-11-pr-4-intent-surface.md | 221 ++++++++++++++++++ 1 file changed, 221 insertions(+) create mode 100644 docs/reviews/2026-09-11-pr-4-intent-surface.md diff --git a/docs/reviews/2026-09-11-pr-4-intent-surface.md b/docs/reviews/2026-09-11-pr-4-intent-surface.md new file mode 100644 index 0000000..790095d --- /dev/null +++ b/docs/reviews/2026-09-11-pr-4-intent-surface.md @@ -0,0 +1,221 @@ +# PR #4 review: The intent surface (journeys, grouping, budgets, agent skill) + +**PR:** https://github.com/SouravInsights/webmcp-stack/pull/4 +**Branch:** `feat/intent-tools` (HEAD `0beff21`) against `main` (`ede5fd9`) +**Scope:** 44 files, +4,103 / -1,126 +**Reviewed:** 2026-09-11 + +## Verdict + +The core of this PR is sound and the restraint in the risky parts is the best +thing about it. Grouping merges only on exact, deterministic rules and skips +everything fuzzy. The submit gate lives in a generator-owned file, so a journey +cannot edit the gate itself. The spec sync is real (verified against the spec at +`97da8f5`). Tests, typecheck, the eval self-test, and a fixture regeneration all +pass. + +It is not merge-ready as-is. There is one generation bug that can make `generate` +produce a surface that its own `verify` fails, and several user-facing claims in +the new docs describe enforcement that the code does not implement. Those claims +sit on the safety story this product sells, so they need to be either built or +rewritten before they ship. + +## What I ran + +| Check | Result | +|---|---| +| `pnpm --filter @webmcp-stack/codegen test` | 220 passed, 15 files | +| `pnpm --filter @webmcp-stack/codegen typecheck` | clean | +| `node packages/codegen/evals/skill/run.mjs --selftest` | passed | +| `pnpm exec biome check packages/codegen` | exits 0, 1 warning, 1 info | +| Fixture regen (`generate --spec fixture/spec.json`) | byte-identical to the committed fixture | +| Spec cross-check at `/Users/souravinsights/Documents/Personal/webmcp` (`97da8f5`) | see below | + +Spec claims verified directly in `index.bs`: + +- `title` exists on `ModelContextTool` (`USVString`), and `RegisteredTool` carries it back (lines 1070-1072, 1220). +- `consequentialHint` is a real third `ToolAnnotations` boolean (line 1082). +- `exposedTo` is a real `ModelContextRegisterToolOptions.sequence` (line 1162). +- `executeTool(RegisteredTool, optional any, optional options)` returns `Promise` (line 611). +- Name rule is 1-128 chars over `[A-Za-z0-9._-]`, so the 30-char cap is Chrome guidance, not spec (line 158). +- `requestUserInteraction()` is absent from the spec, so the "watch item" framing is correct. + +## Findings + +### 1. `fitBudget` can return 151 characters for a 150-char budget (generation bug) + +`packages/codegen/src/describe.ts:53` + +```ts +return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}…`; +``` + +When the overflow has no space in the first `budget` characters, `wordEnd` is +`-1`, so the whole `budget`-length slice is kept and an ellipsis is appended after +it. Reproduced: + +``` +fitBudget("x".repeat(300), 150).length === 151 +``` + +These strings are author param descriptions (URLs, tokens, unbroken identifiers), +which is exactly where this branch fires. The 150-char cap is an error-level +`verify` check (`verify.ts`, "Budgets" check), so generation can emit output that +fails the repository's own CI gate. Fix: reserve the ellipsis inside the budget, +for example `slice.slice(0, Math.max(1, budget - 1)).trimEnd() + "…"`, and add a +test for the no-space case. + +### 2. The docs claim a surface check that does not count journey tools and does not fail CI + +`site/content/docs/journeys-faq.mdx` ("Won't journeys make my tool list even +bigger?") and `docs/research/2026-09-07-what-to-double-down-on.md` both say some +version of: `verify`'s surface-size check counts registered tools including +journey tools, and CI fails when the list grows. + +Neither is true as implemented: + +- `verifyTools` is called with `result.tools` (`cli.ts:337`), which contains only + generated endpoint tools. Journey step tools and submit tools are created at + runtime by `createJourney` and never appear there. `verifyJourneyFiles` counts + steps per journey only, and always as a warning. +- The surface check is `level: "warning"` (`verify.ts:280`), and `verify` exits 1 + only on `error` checks (`cli.ts`, `return errors > 0 ? 1 : 0`). So a 40-tool + surface passes with a warning. + +Either sum registered endpoint tools plus journey-provided tools into the surface +budget and make it an error, or rewrite the FAQ to say what is actually measured. + +### 3. "A tool that only makes sense inside a flow becomes its journey step's only face" is not enforced + +`site/content/docs/journeys-faq.mdx`, `site/content/docs/journeys.mdx`, +`docs/research/2026-09-07-what-to-double-down-on.md`. + +There is no link between a journey file and the generated tool it composes. When +`document-trip` composes `getAutocompleteTool`, the generated +`get-autocomplete.webmcp.ts` still registers standalone, because reads register by +default. Writes happen to be withheld by default, but a user who enables +`create-trip` standalone still gets two doors. Nothing in `verify` catches it. + +This is a convention that the developer has to follow by hand (or by `safety.exclude`), +not a property of the system. The docs present it as enforced ("that's enforced, +not aspirational"). Rewrite to describe it as a convention, or add the machinery. + +### 4. Journey steps are always advertised as read-only, but a step's `call` is arbitrary code + +`packages/codegen/assets/journey.webmcp.ts:154` registers every step with +`annotations: { readOnlyHint: true }`. A `ToolStep.call` is user code and can call +any exported function, including a mutating raw caller such as `fetchCreateTrip`. +`FreeStep.run` can do anything too. + +Two consequences: + +- The `readOnlyHint: true` is a hardcoded assumption that the audit cannot verify. + A step that performs a write is mislabeled and agents will treat it as safe. + `docs/specs/journeys.md` even lists a `navigate` side effect as a possible step. +- The `verify` "no direct fetch" check matches `/\b(?:fetch|callApi)\s*\(/` only. + `fetchCreateTrip(` and `executeCreateTrip(` do not match, so a journey that + performs the real write inside a step bypasses the submit gate and still passes + `verify`. The stated guarantee ("it can't skip the human's yes") only holds for + journeys that route all writes through `submit`. + +If read-only steps are a real invariant, enforce it (for example, require +`ToolStep.tool` to be a read tool, or check `readOnlyHint` at runtime and refuse +to register a step whose composed tool is not a read). If not, drop the claim. + +### 5. `createJourney` captures the model context too early + +`packages/codegen/assets/journey.webmcp.ts:108` does `const modelContext = getModelContext();` +once, when the journey definition is evaluated at module import. Generated tools +resolve `getModelContext()` inside `registerX()` at registration time. If the +runtime installs `document.modelContext` after module evaluation but before +`registerAllTools()` runs, journeys silently never register while ordinary tools +do. Resolve the context inside `registerSteps`/`registerSubmit` so the two paths +behave the same. + +### 6. Journey step descriptions are not budget-checked, but the docs say budgets are enforced + +The factory composes each step's agent-facing description as +`${base} Part of "${def.name}": ${def.goal}` (`journey.webmcp.ts:125`). `base` +already sits at the 500-char tool budget, so the composed string can exceed it. +`verifyJourneyFiles` only measures `description: "..."` literals inside the file +(`verify.ts`), not the runtime-composed text. The "character budgets, enforced" +claim is true for generated endpoint tools only. Worth either measuring composed +descriptions or documenting the gap. + +### 7. Brittle journey linting + +`verifyJourneyFiles` decides "has a submit gate" with +`/submit\s*:/ && /run\s*:/` over the whole file. A `submit:` string in a comment +plus any `run:` passes; a valid shape without those exact keys false-fails. The +step counter's brace matching does not skip string literals, so a `}` inside a +description truncates the count. These are lint false positives/negatives, not +safety holes, but they undermine "CI gates on it". A real parse or a tighter +scanner would be sturdier. + +### 8. Withheld tools still cannot be enabled in one edit + +Pre-existing on `main`, but this PR changes the same import logic and the promise +is central to the skill file. A withheld route-backed tool emits +`import { callApi, toolDisabled }` while its commented registration body uses +`getModelContext`, `requestUserConfirmation`, and `asToolError`. The enabling +comment only says "add toolResult", so uncommenting does not compile. At minimum +name every missing import in the comment, or just include the helpers. + +### 9. Smaller notes + +- `packages/codegen/evals/skill/run.mjs` imports `execFile`/`promisify`, then + `const execFileAsync = promisify(execFile); void execFileAsync;`. Dead code. +- `packages/codegen/src/group.test.ts:129` trips Biome's + `noTemplateCurlyInString` warning. `biome check` exits 0, so "biome clean" is + technically true, but the PR added the warning. +- `packages/codegen/evals/skill/README.md` says "Adding an LLM-as-judge pass is + possible later", which is fine, but note the PR deletes the LLM layer while the + README still frames an LLM grader as a future option. Harmless, just be deliberate. + +## Documentation drift + +These are outside the diff's file list but the PR makes them wrong: + +1. **Root `README.md:108`** still advertises `--suggest` and `--llm`. The site docs + (`cli.mdx`) and `packages/codegen/README.md` were updated; the root README was + not. This is the most visible stale reference. +2. **`docs/specs/journeys.md`** describes a `journeys: [...]` config block that was + never built. The shipped design is TypeScript files under `journeys/`. The spec + is marked "spec, not yet implemented", but the feature now exists in a different + shape. Update it or mark it superseded so the next agent does not implement the + old design. +3. **`docs/specs/generation-pipeline.md`, `codegen-design.md`, `tool-standard.md`** + still describe the LLM layer as shipped or planned. Add a "removed in 0.9" note. +4. **No changeset.** `AGENTS.md` requires a changeset for any user-facing change. + This removes two CLI flags, adds generated files, adds a skill file, and changes + the published package's `files` list. `.changeset/` has no pending entry. + +## Strengths worth keeping + +- **Grouping restraint.** `groupHandshakes` merges only under four deterministic + conditions, threads by exact name, and skips with a note when it cannot. The test + suite pins the beenthere pair, the unthreadable pair, the cross-resource pair, + the GETs-only negative, and the name-collision fallback. This is the right amount + of cleverness. +- **The gate is in owned code.** Putting `build(draft)` and `run` behind a + generator-rewritten file is the correct place for the safety. For journeys that + use the factory as intended, there is no path around confirmation. +- **Budgets have one source of truth.** `TOOL_DESCRIPTION_MAX`/`FIELD_DESCRIPTION_MAX` + are shared by generation and `verify`, so the measure matches the composer. +- **The eval harness grades files, not vibes.** Deterministic graders, a self-test + for the graders, and the "skill removed" control case to detect absorption are + all above the bar for a docs-driven feature. +- **The fixture is current.** Regenerating from `fixture/spec.json` produces + byte-identical files, so the eval baseline is not drifting from the templates. +- **Spec sync is honest.** Everything claimed in `docs/research/2026-09-10-spec-sync.md` + matches the spec at `97da8f5`, including the "not in the spec yet" watch items. + +## Suggested before merge + +1. Fix the `fitBudget` off-by-one and add a no-space test. +2. Fix findings 2, 3, and 4: either implement the surface/absorption/step-annotation + guarantees or rewrite the docs to match reality. Do not ship a safety claim that + the audit cannot back. +3. Resolve the model context inside `register` (finding 5). +4. Update root `README.md` and decide what to do with the stale specs. +5. Add a changeset. From 77616e59425d6b91b9b9c12b90fdf7437a431b0c Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 17:36:49 +0530 Subject: [PATCH 16/41] Update 2026-09-11-pr-4-intent-surface.md --- .../reviews/2026-09-11-pr-4-intent-surface.md | 360 +++++++----------- 1 file changed, 144 insertions(+), 216 deletions(-) diff --git a/docs/reviews/2026-09-11-pr-4-intent-surface.md b/docs/reviews/2026-09-11-pr-4-intent-surface.md index 790095d..11e67e2 100644 --- a/docs/reviews/2026-09-11-pr-4-intent-surface.md +++ b/docs/reviews/2026-09-11-pr-4-intent-surface.md @@ -1,221 +1,149 @@ -# PR #4 review: The intent surface (journeys, grouping, budgets, agent skill) +# PR #4 review: The intent surface **PR:** https://github.com/SouravInsights/webmcp-stack/pull/4 -**Branch:** `feat/intent-tools` (HEAD `0beff21`) against `main` (`ede5fd9`) -**Scope:** 44 files, +4,103 / -1,126 +**Branch:** `feat/intent-tools` against `main` **Reviewed:** 2026-09-11 -## Verdict +## The short version -The core of this PR is sound and the restraint in the risky parts is the best -thing about it. Grouping merges only on exact, deterministic rules and skips -everything fuzzy. The submit gate lives in a generator-owned file, so a journey -cannot edit the gate itself. The spec sync is real (verified against the spec at -`97da8f5`). Tests, typecheck, the eval self-test, and a fixture regeneration all -pass. - -It is not merge-ready as-is. There is one generation bug that can make `generate` -produce a surface that its own `verify` fails, and several user-facing claims in -the new docs describe enforcement that the code does not implement. Those claims -sit on the safety story this product sells, so they need to be either built or -rewritten before they ship. - -## What I ran - -| Check | Result | -|---|---| -| `pnpm --filter @webmcp-stack/codegen test` | 220 passed, 15 files | -| `pnpm --filter @webmcp-stack/codegen typecheck` | clean | -| `node packages/codegen/evals/skill/run.mjs --selftest` | passed | -| `pnpm exec biome check packages/codegen` | exits 0, 1 warning, 1 info | -| Fixture regen (`generate --spec fixture/spec.json`) | byte-identical to the committed fixture | -| Spec cross-check at `/Users/souravinsights/Documents/Personal/webmcp` (`97da8f5`) | see below | - -Spec claims verified directly in `index.bs`: - -- `title` exists on `ModelContextTool` (`USVString`), and `RegisteredTool` carries it back (lines 1070-1072, 1220). -- `consequentialHint` is a real third `ToolAnnotations` boolean (line 1082). -- `exposedTo` is a real `ModelContextRegisterToolOptions.sequence` (line 1162). -- `executeTool(RegisteredTool, optional any, optional options)` returns `Promise` (line 611). -- Name rule is 1-128 chars over `[A-Za-z0-9._-]`, so the 30-char cap is Chrome guidance, not spec (line 158). -- `requestUserInteraction()` is absent from the spec, so the "watch item" framing is correct. - -## Findings - -### 1. `fitBudget` can return 151 characters for a 150-char budget (generation bug) - -`packages/codegen/src/describe.ts:53` - -```ts -return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}…`; -``` - -When the overflow has no space in the first `budget` characters, `wordEnd` is -`-1`, so the whole `budget`-length slice is kept and an ellipsis is appended after -it. Reproduced: - -``` -fitBudget("x".repeat(300), 150).length === 151 -``` - -These strings are author param descriptions (URLs, tokens, unbroken identifiers), -which is exactly where this branch fires. The 150-char cap is an error-level -`verify` check (`verify.ts`, "Budgets" check), so generation can emit output that -fails the repository's own CI gate. Fix: reserve the ellipsis inside the budget, -for example `slice.slice(0, Math.max(1, budget - 1)).trimEnd() + "…"`, and add a -test for the no-space case. - -### 2. The docs claim a surface check that does not count journey tools and does not fail CI - -`site/content/docs/journeys-faq.mdx` ("Won't journeys make my tool list even -bigger?") and `docs/research/2026-09-07-what-to-double-down-on.md` both say some -version of: `verify`'s surface-size check counts registered tools including -journey tools, and CI fails when the list grows. - -Neither is true as implemented: - -- `verifyTools` is called with `result.tools` (`cli.ts:337`), which contains only - generated endpoint tools. Journey step tools and submit tools are created at - runtime by `createJourney` and never appear there. `verifyJourneyFiles` counts - steps per journey only, and always as a warning. -- The surface check is `level: "warning"` (`verify.ts:280`), and `verify` exits 1 - only on `error` checks (`cli.ts`, `return errors > 0 ? 1 : 0`). So a 40-tool - surface passes with a warning. - -Either sum registered endpoint tools plus journey-provided tools into the surface -budget and make it an error, or rewrite the FAQ to say what is actually measured. - -### 3. "A tool that only makes sense inside a flow becomes its journey step's only face" is not enforced - -`site/content/docs/journeys-faq.mdx`, `site/content/docs/journeys.mdx`, -`docs/research/2026-09-07-what-to-double-down-on.md`. - -There is no link between a journey file and the generated tool it composes. When -`document-trip` composes `getAutocompleteTool`, the generated -`get-autocomplete.webmcp.ts` still registers standalone, because reads register by -default. Writes happen to be withheld by default, but a user who enables -`create-trip` standalone still gets two doors. Nothing in `verify` catches it. - -This is a convention that the developer has to follow by hand (or by `safety.exclude`), -not a property of the system. The docs present it as enforced ("that's enforced, -not aspirational"). Rewrite to describe it as a convention, or add the machinery. - -### 4. Journey steps are always advertised as read-only, but a step's `call` is arbitrary code - -`packages/codegen/assets/journey.webmcp.ts:154` registers every step with -`annotations: { readOnlyHint: true }`. A `ToolStep.call` is user code and can call -any exported function, including a mutating raw caller such as `fetchCreateTrip`. -`FreeStep.run` can do anything too. - -Two consequences: - -- The `readOnlyHint: true` is a hardcoded assumption that the audit cannot verify. - A step that performs a write is mislabeled and agents will treat it as safe. - `docs/specs/journeys.md` even lists a `navigate` side effect as a possible step. -- The `verify` "no direct fetch" check matches `/\b(?:fetch|callApi)\s*\(/` only. - `fetchCreateTrip(` and `executeCreateTrip(` do not match, so a journey that - performs the real write inside a step bypasses the submit gate and still passes - `verify`. The stated guarantee ("it can't skip the human's yes") only holds for - journeys that route all writes through `submit`. - -If read-only steps are a real invariant, enforce it (for example, require -`ToolStep.tool` to be a read tool, or check `readOnlyHint` at runtime and refuse -to register a step whose composed tool is not a read). If not, drop the claim. - -### 5. `createJourney` captures the model context too early - -`packages/codegen/assets/journey.webmcp.ts:108` does `const modelContext = getModelContext();` -once, when the journey definition is evaluated at module import. Generated tools -resolve `getModelContext()` inside `registerX()` at registration time. If the -runtime installs `document.modelContext` after module evaluation but before -`registerAllTools()` runs, journeys silently never register while ordinary tools -do. Resolve the context inside `registerSteps`/`registerSubmit` so the two paths -behave the same. - -### 6. Journey step descriptions are not budget-checked, but the docs say budgets are enforced - -The factory composes each step's agent-facing description as -`${base} Part of "${def.name}": ${def.goal}` (`journey.webmcp.ts:125`). `base` -already sits at the 500-char tool budget, so the composed string can exceed it. -`verifyJourneyFiles` only measures `description: "..."` literals inside the file -(`verify.ts`), not the runtime-composed text. The "character budgets, enforced" -claim is true for generated endpoint tools only. Worth either measuring composed -descriptions or documenting the gap. - -### 7. Brittle journey linting - -`verifyJourneyFiles` decides "has a submit gate" with -`/submit\s*:/ && /run\s*:/` over the whole file. A `submit:` string in a comment -plus any `run:` passes; a valid shape without those exact keys false-fails. The -step counter's brace matching does not skip string literals, so a `}` inside a -description truncates the count. These are lint false positives/negatives, not -safety holes, but they undermine "CI gates on it". A real parse or a tighter -scanner would be sturdier. - -### 8. Withheld tools still cannot be enabled in one edit - -Pre-existing on `main`, but this PR changes the same import logic and the promise -is central to the skill file. A withheld route-backed tool emits -`import { callApi, toolDisabled }` while its commented registration body uses -`getModelContext`, `requestUserConfirmation`, and `asToolError`. The enabling -comment only says "add toolResult", so uncommenting does not compile. At minimum -name every missing import in the comment, or just include the helpers. - -### 9. Smaller notes - -- `packages/codegen/evals/skill/run.mjs` imports `execFile`/`promisify`, then - `const execFileAsync = promisify(execFile); void execFileAsync;`. Dead code. -- `packages/codegen/src/group.test.ts:129` trips Biome's - `noTemplateCurlyInString` warning. `biome check` exits 0, so "biome clean" is - technically true, but the PR added the warning. -- `packages/codegen/evals/skill/README.md` says "Adding an LLM-as-judge pass is - possible later", which is fine, but note the PR deletes the LLM layer while the - README still frames an LLM grader as a future option. Harmless, just be deliberate. - -## Documentation drift - -These are outside the diff's file list but the PR makes them wrong: - -1. **Root `README.md:108`** still advertises `--suggest` and `--llm`. The site docs - (`cli.mdx`) and `packages/codegen/README.md` were updated; the root README was - not. This is the most visible stale reference. -2. **`docs/specs/journeys.md`** describes a `journeys: [...]` config block that was - never built. The shipped design is TypeScript files under `journeys/`. The spec - is marked "spec, not yet implemented", but the feature now exists in a different - shape. Update it or mark it superseded so the next agent does not implement the - old design. -3. **`docs/specs/generation-pipeline.md`, `codegen-design.md`, `tool-standard.md`** - still describe the LLM layer as shipped or planned. Add a "removed in 0.9" note. -4. **No changeset.** `AGENTS.md` requires a changeset for any user-facing change. - This removes two CLI flags, adds generated files, adds a skill file, and changes - the published package's `files` list. `.changeset/` has no pending entry. - -## Strengths worth keeping - -- **Grouping restraint.** `groupHandshakes` merges only under four deterministic - conditions, threads by exact name, and skips with a note when it cannot. The test - suite pins the beenthere pair, the unthreadable pair, the cross-resource pair, - the GETs-only negative, and the name-collision fallback. This is the right amount - of cleverness. -- **The gate is in owned code.** Putting `build(draft)` and `run` behind a - generator-rewritten file is the correct place for the safety. For journeys that - use the factory as intended, there is no path around confirmation. -- **Budgets have one source of truth.** `TOOL_DESCRIPTION_MAX`/`FIELD_DESCRIPTION_MAX` - are shared by generation and `verify`, so the measure matches the composer. -- **The eval harness grades files, not vibes.** Deterministic graders, a self-test - for the graders, and the "skill removed" control case to detect absorption are - all above the bar for a docs-driven feature. -- **The fixture is current.** Regenerating from `fixture/spec.json` produces - byte-identical files, so the eval baseline is not drifting from the templates. -- **Spec sync is honest.** Everything claimed in `docs/research/2026-09-10-spec-sync.md` - matches the spec at `97da8f5`, including the "not in the spec yet" watch items. - -## Suggested before merge - -1. Fix the `fitBudget` off-by-one and add a no-space test. -2. Fix findings 2, 3, and 4: either implement the surface/absorption/step-annotation - guarantees or rewrite the docs to match reality. Do not ship a safety claim that - the audit cannot back. -3. Resolve the model context inside `register` (finding 5). -4. Update root `README.md` and decide what to do with the stale specs. -5. Add a changeset. +The hard parts are built well. Grouping the upload handshake only when the rules +are exact is the right call, and putting the submit gate in a generator-owned +file means a journey can't edit the gate out. Tests pass, types pass, and the +spec sync is real. + +But before this merges I'd fix one generation bug and correct two safety +promises in the new docs. The bug can make the tool fail the project's own CI. +The promises are worse in a quiet way: they tell a reader the audit enforces +things it doesn't, and the whole pitch of this product is that the audit is +where the safety lives. + +## What I actually ran + +- Test suite: 220 pass. +- Typecheck: clean. +- Eval self-test (`run.mjs --selftest`): passes. +- Regenerated the eval fixture from its spec: byte-for-byte identical to what's + committed, so the eval baseline matches the current templates. +- Checked every spec claim against the spec fork at `97da8f5`. `title`, + `consequentialHint`, `exposedTo`, the `executeTool` signature, and the 1-128 + name rule are all exactly as described. `requestUserInteraction()` really is + absent, so the "watch item" framing is honest. + +## The problems, in plain terms + +### 1. The generator can produce a value that's one character too long + +A parameter description is supposed to be trimmed to 150 characters. If the +description is one long unbroken string (a URL, a token, an ID with no spaces), +the trimmer keeps all 150 characters and then adds an ellipsis, so the result is +151. I reproduced it. + +Why that matters: `verify` later flags anything over 150 as an **error**, and the +project's CI gate fails on errors. So a developer runs `generate`, gets a file the +tool wrote, and then their own pipeline rejects it. And they can't just edit the +generated region to fix it, because that's the region that gets overwritten. The +tool should never produce text that its own checker refuses. + +The cause is one line in `packages/codegen/src/describe.ts` that appends the +ellipsis after slicing to the budget instead of inside it. + +### 2. Two safety promises the docs make, that the code doesn't keep + +Both are in the new journey FAQ (and one is repeated in the direction doc): + +**"Journey tools count toward the surface limit, and CI fails when the list +gets too big."** Neither half is true. The surface check only looks at the +generated endpoint tools. The journey step tools are built at runtime and never +reach that code, so they aren't counted at all. And going over the limit is a +warning, not an error, so it doesn't fail CI either. + +**"A tool that only makes sense inside a journey won't register on its own, so +the journey shrinks your surface."** Nothing implements this. When +`document-trip` gets its place data from `get-autocomplete`, that tool still +registers on its own, because reads register by default. The convention is real, +but following it is on the developer. The docs present it as enforced. + +Why it matters: a reader will trust that the audit has their back here and stop +paying attention. Then they ship forty tools and nothing in `verify` objects. + +### 3. A journey step is labeled "read-only" even when it writes + +Every journey step is registered as read-only, no matter what it does. But a +step's `call` is just user code. It can call the real create-trip caller directly +instead of going through the submit gate. When it does: + +- The tool is advertised to the agent as safe to call, so the agent calls it. +- The write happens with no confirmation prompt. +- `verify` doesn't catch it, because its "no direct fetch" check looks for + `fetch(` and `callApi(`, not for `fetchCreateTrip(`. + +So the human-in-the-loop guarantee only holds for a journey that routes every +write through `submit`. A sloppy journey (or a coding agent having a bad day) can +route around it, and the audit will stay quiet. For a tool whose reason to exist +is "the audit blocks the dangerous stuff," this is the finding I'd care most +about. + +There's a related assumption baked in: `docs/specs/journeys.md` even lists a +`navigate` side effect as a possible step, which plainly isn't a read. + +### 4. Journeys can silently fail to register + +Generated tools ask the browser for the WebMCP API at the moment they register. +Journeys ask once, when the file is first loaded, and remember the answer +forever. If the API appears in between those two moments, the tools register and +the journeys don't, with no error. The fix is small: ask at registration time, +the same way the tools do. + +### 5. Journey step descriptions can go over the 500-character limit + +The factory takes the generated tool's description, which is already trimmed to +500, and appends `Part of "document-trip": ...`. So the final text can be over +budget, and `verify` only measures the description written in the journey file, +not the text the factory builds at runtime. Again: the "budgets enforced" claim +holds for endpoint tools, but not for journey tools. + +## Smaller things, worth a look but not blockers + +- **The root `README.md` still sells the two flags this PR deletes.** It says + `--suggest` and `--llm` exist. The package README and the docs site were + updated; the root one was missed. This is the most visible stale line. +- **`docs/specs/journeys.md` describes a design that was never built.** It + describes declaring journeys in a config block. What shipped is TypeScript + files. The spec is marked "not yet implemented", but the feature now exists in + a different shape, so the next person will read the wrong plan. +- **The old LLM layer is still all over `docs/specs/`.** Fine to defer, but a + one-line "removed in 0.9" note would stop someone rebuilding it. +- **No changeset.** The repo's own rule is to add one for user-facing changes, + and this removes two CLI flags, adds files, and changes what the package + publishes. +- **Small cleanups:** dead code in `evals/skill/run.mjs` (imports and then voids + `execFile`), and one Biome warning from a test string in `group.test.ts`. + +## What's genuinely good here + +Worth saying plainly, because most of this doc is complaints: + +- **Grouping is disciplined.** It merges only when four exact rules line up, and + skips anything fuzzy with a note. The tests pin the real beenthere pair, the + unthreadable pair, the cross-resource pair, and a GET negative. That's the + right instinct: a decision it doesn't have to make is a decision it can't get + wrong. +- **The gate sits in owned code.** For journeys that use the factory as intended, + there is genuinely no way around the confirmation. That's the right design. +- **Budgets have one source of truth.** Generation and `verify` read the same + constants, so the measure matches the composer. +- **The evals grade files, not vibes.** A self-test for the graders and a + "skill removed" control case for detecting absorption is more rigor than most + features get. +- **The spec sync is honest.** The watch items really aren't in the spec. + +## If I were merging this + +1. Fix the one-character overflow and add a test for the no-space case. +2. Decide on the step read-only labels and the write-bypass. Either enforce that + steps are reads, or stop calling them read-only, and teach `verify` to spot a + step that calls a known write tool. +3. Correct the two doc promises (surface count, tool absorption), or build them. + Don't ship a safety claim the audit can't back. +4. Ask the browser for the WebMCP API at registration time. +5. Update the root README and add a changeset. From 114c84e2e568b38f746e4d3b9aaf3f33d481a36c Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:21:03 +0530 Subject: [PATCH 17/41] fix(generated output): keep the 1.5K truncation notice on one line MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The notice was written as a "\n…" string inside the outer template literal that builds runtime.webmcp.ts, so the emitted file contained a real newline inside a double-quoted string. Every generated runtime was a syntax error, which means no generated tool compiled. Emit the escape instead, and add generated-code.test.ts, which runs every template output through the TypeScript compiler so an escaping mistake cannot ship again. Regenerate the eval fixture's runtime to match. --- .../fixture/src/webmcp/runtime.webmcp.ts | 3 +- .../src/outputs/generated-code.test.ts | 145 ++++++++++++++++++ .../codegen/src/outputs/tools-templates.ts | 2 +- 3 files changed, 147 insertions(+), 3 deletions(-) create mode 100644 packages/codegen/src/outputs/generated-code.test.ts diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts index df07e76..9939c09 100644 --- a/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts +++ b/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts @@ -125,8 +125,7 @@ export async function callApi( const TOOL_OUTPUT_MAX = 1536; const TRUNCATED_NOTICE = - " -… [truncated to fit the 1.5K output budget — return a smaller slice or paginate]"; + "\n… [truncated to fit the 1.5K output budget — return a smaller slice or paginate]"; /** * Wrap a result in the MCP shape, so tool bodies stay one line. The result diff --git a/packages/codegen/src/outputs/generated-code.test.ts b/packages/codegen/src/outputs/generated-code.test.ts new file mode 100644 index 0000000..e53dd16 --- /dev/null +++ b/packages/codegen/src/outputs/generated-code.test.ts @@ -0,0 +1,145 @@ +import ts from "typescript"; +import { describe, expect, it } from "vitest"; +import type { ReviewedTool } from "../types.js"; +import { + barrelSource, + generatedRegion, + ownedRegionScaffold, + runtimeSource, +} from "./tools-templates.js"; + +/** + * These tests exist because the templates are TypeScript built from template + * literals, and an escape that is correct in the source file can be wrong in + * the emitted file. The 1.5K truncation notice shipped as a string literal + * broken across two lines exactly this way: the source had "\n", the outer + * template literal turned it into a real newline, and every generated + * runtime.webmcp.ts was a syntax error. Parsing the output is the only check + * that catches that class of mistake. + */ + +/** A reviewed tool with every field the templates read, overridable. */ +function reviewedTool(overrides: Partial = {}): ReviewedTool { + return { + id: "GET /orders/{id}", + name: "get-order-status", + source: { kind: "openapi", ref: "GET /orders/{id}" }, + inputSchema: { + type: "object", + properties: { orderId: { type: "string", description: "The order ID." } }, + required: ["orderId"], + }, + outputSchema: { type: "object", properties: { status: { type: "string" } } }, + inputTypeName: "GetOrderStatusInput", + httpMethod: "GET", + pathTemplate: "/orders/{orderId}", + paramLocations: { path: ["orderId"], query: [], body: [] }, + sideEffect: "read", + endpointRole: "endpoint", + requiresAuth: false, + enabledByDefault: true, + withheld: false, + description: "Returns the current status of an order.", + descriptionSource: "openapi-summary", + riskTier: "safe-read", + hints: { + readOnlyHint: true, + destructiveHint: false, + idempotentHint: true, + untrustedContentHint: false, + }, + piiInOutput: [], + ...overrides, + } as ReviewedTool; +} + +/** Syntactic diagnostics only: we do not resolve the runtime import. */ +function syntaxErrors(source: string): string[] { + const result = ts.transpileModule(source, { + reportDiagnostics: true, + compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext }, + }); + return (result.diagnostics ?? []).map((diagnostic) => + ts.flattenDiagnosticMessageText(diagnostic.messageText, " "), + ); +} + +describe("generated files parse", () => { + it("runtime.webmcp.ts is valid syntax and keeps the truncation escape", () => { + const runtime = runtimeSource(); + expect(syntaxErrors(runtime)).toEqual([]); + // A real newline inside this double-quoted string is the bug we shipped + // once; the emitted file must carry the two-character escape instead. + expect(runtime).toContain('"\\n'); + }); + + it("an enabled endpoint tool parses", () => { + const tool = reviewedTool(); + expect(syntaxErrors(`${generatedRegion(tool)}${ownedRegionScaffold(tool)}`)).toEqual([]); + }); + + it("a withheld write tool parses, comments and all", () => { + const tool = reviewedTool({ + name: "create-trip", + id: "POST /trips", + source: { kind: "openapi", ref: "POST /trips" }, + httpMethod: "POST", + pathTemplate: "/trips", + paramLocations: { path: [], query: [], body: ["title"] }, + inputSchema: { type: "object", properties: { title: { type: "string" } } }, + sideEffect: "write", + riskTier: "write-confirm", + enabledByDefault: false, + withheld: true, + hints: { + readOnlyHint: false, + destructiveHint: false, + idempotentHint: false, + untrustedContentHint: false, + }, + }); + expect(syntaxErrors(`${generatedRegion(tool)}${ownedRegionScaffold(tool)}`)).toEqual([]); + }); + + it("the barrel parses, with and without journeys", () => { + expect(syntaxErrors(barrelSource([reviewedTool()]))).toEqual([]); + expect(syntaxErrors(barrelSource([reviewedTool()], ["document-trip.webmcp.ts"]))).toEqual([]); + }); + + it("a composed handshake tool parses", () => { + const tool = reviewedTool({ + name: "upload-media", + id: "POST /media/request-upload + POST /media/uploads/{uploadId}/complete", + source: { + kind: "openapi", + ref: "POST /media/request-upload + POST /media/uploads/{uploadId}/complete", + }, + httpMethod: undefined, + pathTemplate: undefined, + paramLocations: undefined, + inputSchema: { + type: "object", + properties: { fileName: { type: "string" } }, + required: ["fileName"], + }, + sideEffect: "write", + riskTier: "write-confirm", + enabledByDefault: false, + withheld: true, + compose: { + first: { + httpMethod: "POST", + pathTemplate: "/media/request-upload", + paramLocations: { path: [], query: [], body: ["fileName"] }, + }, + second: { + httpMethod: "POST", + pathTemplate: "/media/uploads/{uploadId}/complete", + paramLocations: { path: ["uploadId"], query: [], body: [] }, + }, + threaded: { uploadId: "uploadId" }, + }, + }); + expect(syntaxErrors(`${generatedRegion(tool)}${ownedRegionScaffold(tool)}`)).toEqual([]); + }); +}); diff --git a/packages/codegen/src/outputs/tools-templates.ts b/packages/codegen/src/outputs/tools-templates.ts index 0596b71..3cf1ef8 100644 --- a/packages/codegen/src/outputs/tools-templates.ts +++ b/packages/codegen/src/outputs/tools-templates.ts @@ -616,7 +616,7 @@ export async function callApi( const TOOL_OUTPUT_MAX = 1536; const TRUNCATED_NOTICE = - "\n… [truncated to fit the 1.5K output budget — return a smaller slice or paginate]"; + "\\n… [truncated to fit the 1.5K output budget — return a smaller slice or paginate]"; /** * Wrap a result in the MCP shape, so tool bodies stay one line. The result From a6699e31bc2aa816ec210c0bfbf272a0920950f3 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:21:11 +0530 Subject: [PATCH 18/41] fix(audit): treat character budgets as guidance, count journey tools Chrome's 500/150 character budgets are authoring advice, not browser rules: the spec only rejects an empty description or a name outside 1-128 chars. Generation no longer silently shortens text a person or a spec wrote. Machine drafted text is composed to fit; author text is kept in full, and verify reports an overrun as a warning for both tools and journey files. fitBudget also reserves room for its ellipsis, so an unbroken token can no longer come back one character over budget. The surface check now counts journey tools too (one per step plus the submit gate), because they register at runtime and previously escaped the total. --- packages/codegen/src/cli.ts | 15 ++++++-- packages/codegen/src/describe.test.ts | 37 ++++++++++++++----- packages/codegen/src/describe.ts | 53 ++++++++++++++++++--------- packages/codegen/src/verify.test.ts | 33 +++++++++++++---- packages/codegen/src/verify.ts | 48 +++++++++++++++++++----- 5 files changed, 137 insertions(+), 49 deletions(-) diff --git a/packages/codegen/src/cli.ts b/packages/codegen/src/cli.ts index 3ef978a..a758617 100644 --- a/packages/codegen/src/cli.ts +++ b/packages/codegen/src/cli.ts @@ -45,7 +45,13 @@ import { startDevServer } from "./dev/server.js"; import { debug, enableVerbose, error, info, success, warn } from "./logger.js"; import { runGenerate } from "./pipeline.js"; import { resolveSetup } from "./setup.js"; -import { type JourneyFileInput, verifyJourneyFiles, verifyTools, verifyUrl } from "./verify.js"; +import { + countJourneyTools, + type JourneyFileInput, + verifyJourneyFiles, + verifyTools, + verifyUrl, +} from "./verify.js"; import { applyWiring, planWiring, type WirePlan } from "./wire.js"; const HELP = ` @@ -333,11 +339,11 @@ async function verify(flags: CliFlags): Promise { }); const registered = result.tools.filter((tool) => !tool.withheld); - const checks = verifyTools(result.tools); // Journey files are the user's code, so verify can't get them from the // pipeline's tool list — it reads the journeys/ folder of each tools - // output itself and lints what it finds there. + // output itself and lints what it finds there. Read them first: the surface + // count needs to include the tools they register at runtime. const journeyInputs: JourneyFileInput[] = []; for (const output of setup.config.outputs) { if (output.kind !== "tools") continue; @@ -355,6 +361,9 @@ async function verify(flags: CliFlags): Promise { }); } } + + const journeyToolCount = countJourneyTools(journeyInputs); + const checks = verifyTools(result.tools, { journeyToolCount }); checks.push(...verifyJourneyFiles(journeyInputs)); info(""); diff --git a/packages/codegen/src/describe.test.ts b/packages/codegen/src/describe.test.ts index 0688e44..8a0b411 100644 --- a/packages/codegen/src/describe.test.ts +++ b/packages/codegen/src/describe.test.ts @@ -24,12 +24,20 @@ describe("fitBudget", () => { }); it("hard-cuts at a word boundary with an ellipsis when no good sentence break exists", () => { - const text = "a b c d e f g h i j k l m n o p q r s t u v w x y z ".repeat(6) + "end"; + const text = `${"a b c d e f g h i j k l m n o p q r s t u v w x y z ".repeat(6)}end`; const fitted = fitBudget(text, 150); expect(fitted.length).toBeLessThanOrEqual(150); expect(fitted.endsWith("…")).toBe(true); expect(fitted).not.toContain("end"); }); + + it("never exceeds the budget for a single unbroken token", () => { + // The regression that started this: a URL or token with no spaces used to + // come back at budget + 1 because the ellipsis was appended after the cut. + const fitted = fitBudget("x".repeat(300), 150); + expect(fitted.length).toBeLessThanOrEqual(150); + expect(fitted.endsWith("…")).toBe(true); + }); }); describe("describeConstraints", () => { @@ -432,17 +440,25 @@ describe("describeCandidateInputs through nullable wrappers", () => { }); describe("description budgets", () => { - it("caps author field text at the 150-character parameter budget", () => { - const result = describeField("notes", { - type: "string", - description: `The notes field of the record. ${"More detail about things. ".repeat(9)}`, - }); - expect(result.description.length).toBeLessThanOrEqual(FIELD_DESCRIPTION_MAX); - expect(result.description.startsWith("The notes field of the record.")).toBe(true); + it("keeps long author field text verbatim instead of silently cutting it", () => { + const authored = `The notes field of the record. ${"More detail about things. ".repeat(9)}`; + const result = describeField("notes", { type: "string", description: authored.trim() }); expect(result.synthesized).toBe(false); + // Chrome's budget is guidance, not a browser rule: the author's sentence + // survives and verify warns, rather than this layer dropping half of it. + expect(result.description.length).toBeGreaterThan(FIELD_DESCRIPTION_MAX); + expect(result.description).toContain("More detail about things."); + }); + + it("fits machine-drafted field text to the 150-character budget", () => { + // No author text, so the draft is ours to trim; the long pattern makes it + // overflow the budget. + const result = describeField("value", { type: "string", pattern: "a".repeat(200) }); + expect(result.synthesized).toBe(true); + expect(result.description.length).toBeLessThanOrEqual(FIELD_DESCRIPTION_MAX); }); - it("caps the assembled tool description at the 500-character budget", () => { + it("keeps long author tool text verbatim instead of cutting at 500", () => { const candidate = { name: "list-trips", description: `List the trips. ${"A long explanation of everything this endpoint could ever do. ".repeat(12)}`, @@ -450,6 +466,7 @@ describe("description budgets", () => { outputSchema: { type: "array" }, } as unknown as CandidateTool; describeCandidateTool(candidate); - expect(candidate.description.length).toBeLessThanOrEqual(TOOL_DESCRIPTION_MAX); + expect(candidate.description.length).toBeGreaterThan(TOOL_DESCRIPTION_MAX); + expect(candidate.description.startsWith("List the trips.")).toBe(true); }); }); diff --git a/packages/codegen/src/describe.ts b/packages/codegen/src/describe.ts index f2fda75..db3e241 100644 --- a/packages/codegen/src/describe.ts +++ b/packages/codegen/src/describe.ts @@ -13,6 +13,13 @@ * - Append, never replace. Author text (a spec description, a `.describe()`) * stays verbatim; synthesized constraints follow it. The author's words are * always the better text. + * - Bound the machine's words, never the author's. Chrome's 500/150 budgets + * are authoring guidance, not spec rules (the browser only rejects an + * empty description or a name outside 1-128 chars). So machine-drafted + * text is composed to fit the budget by construction, and author text is + * never silently shortened. If author text runs long, verify warns and + * the developer decides; losing half a sentence to a character counter is + * worse than a description that is a few characters over. * - Only fill silence. A field with no text at all gets a draft built from * its name, type, and constraints, and that field is marked as * machine-written so the audit can see it. Machine text is a floor, not a @@ -33,16 +40,21 @@ export const TOOL_DESCRIPTION_MAX = 500; export const FIELD_DESCRIPTION_MAX = 150; /** - * Fit text to a character budget. A text that fits passes through untouched. - * One that overflows is cut at the last sentence boundary that keeps at - * least half the budget (a cut near the end keeps the author's thought, at - * the price of trailing sentences); otherwise it hard-cuts at a word - * boundary and ends with an ellipsis. Composed text is always budget-safe - * before it leaves this module, and verify measures the final result. + * Fit machine-drafted text to a character budget. A text that fits passes + * through untouched. One that overflows is cut at the last sentence boundary + * that keeps at least half the budget (a cut near the end keeps the thought, + * at the price of trailing sentences); otherwise it hard-cuts at a word + * boundary and ends with an ellipsis. The ellipsis is reserved inside the + * budget, so the result never exceeds it, even for a single unbroken token. + * + * Only call this on text this module generated. Author text is left alone + * on purpose (see the module comment). */ export function fitBudget(text: string, budget: number): string { if (text.length <= budget) return text; - const slice = text.slice(0, budget); + // Leave one character for the ellipsis, so the result can never be + // budget + 1 when there is no space to cut at. + const slice = text.slice(0, budget - 1); const sentenceEnd = Math.max( slice.lastIndexOf(". "), slice.lastIndexOf("! "), @@ -50,7 +62,8 @@ export function fitBudget(text: string, budget: number): string { ); if (sentenceEnd >= Math.floor(budget / 2)) return slice.slice(0, sentenceEnd + 1); const wordEnd = slice.lastIndexOf(" "); - return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}…`; + const body = wordEnd > 0 ? slice.slice(0, wordEnd) : slice; + return `${body.trimEnd()}…`; } /** @@ -318,9 +331,12 @@ export function describeField( } } - // The 150-character parameter budget applies to the final text, author or - // machine: overflow is cut at a sentence boundary (see fitBudget). - result.description = fitBudget(result.description, FIELD_DESCRIPTION_MAX); + // Only machine-drafted text is composed to fit the budget. Author text is + // preserved in full; verify warns on it instead of this layer silently + // dropping the author's words. + if (result.synthesized) { + result.description = fitBudget(result.description, FIELD_DESCRIPTION_MAX); + } return result; } @@ -453,7 +469,11 @@ export function describeCandidateTool(candidate: CandidateTool): void { const returns = candidate.outputSchema ? returnShapeSentence(candidate.name, candidate.outputSchema) : ""; - candidate.description = [sentence, returns].filter(Boolean).join(" "); + // No author text existed, so this is ours to compose: fit it. + candidate.description = fitBudget( + [sentence, returns].filter(Boolean).join(" "), + TOOL_DESCRIPTION_MAX, + ); candidate.descriptionSource = "generated-template"; return; } @@ -465,10 +485,7 @@ export function describeCandidateTool(candidate: CandidateTool): void { : ""; // The join is between sentences: the base earns its period first. const base = returns && !/[.!?]$/.test(normalized) ? `${normalized}.` : normalized; - // The 500-character tool budget applies to the final text, author or - // machine: overflow is cut at a sentence boundary (see fitBudget). - candidate.description = fitBudget( - [base, returns].filter(Boolean).join(" "), - TOOL_DESCRIPTION_MAX, - ); + // Author text plus a machine return sentence. The author's half is not + // shortened; verify warns if the total runs past the budget. + candidate.description = [base, returns].filter(Boolean).join(" "); } diff --git a/packages/codegen/src/verify.test.ts b/packages/codegen/src/verify.test.ts index f541a62..60137af 100644 --- a/packages/codegen/src/verify.test.ts +++ b/packages/codegen/src/verify.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "vitest"; import type { ReviewedTool } from "./types.js"; -import { verifyJourneyFiles, verifyTools } from "./verify.js"; +import { countJourneyTools, verifyJourneyFiles, verifyTools } from "./verify.js"; function reviewedTool(overrides: Partial = {}): ReviewedTool { return { @@ -24,15 +24,15 @@ function reviewedTool(overrides: Partial = {}): ReviewedTool { } describe("verify description budgets", () => { - it("flags a tool description over the 500-character budget as an error", () => { + it("warns (not errors) when a tool description is over the 500-character budget", () => { const checks = verifyTools([reviewedTool({ description: `List. ${"word ".repeat(120)}` })]); const budgets = checks.find((check) => check.area === "Budgets"); - expect(budgets?.level).toBe("error"); + expect(budgets?.level).toBe("warning"); expect(budgets?.findings[0]).toContain("get-order-status"); expect(budgets?.findings[0]).toContain("500"); }); - it("flags a parameter description over the 150-character budget, with its field", () => { + it("warns on a parameter description over the 150-character budget, with its field", () => { const checks = verifyTools([ reviewedTool({ inputSchema: { @@ -42,7 +42,7 @@ describe("verify description budgets", () => { }), ]); const budgets = checks.find((check) => check.area === "Budgets"); - expect(budgets?.level).toBe("error"); + expect(budgets?.level).toBe("warning"); expect(budgets?.findings[0]).toContain("get-order-status → notes"); expect(budgets?.findings[0]).toContain("150"); }); @@ -62,7 +62,7 @@ describe("verify description budgets", () => { }), ]); const budgets = checks.find((check) => check.area === "Budgets"); - expect(budgets?.level).toBe("error"); + expect(budgets?.level).toBe("warning"); expect(budgets?.findings[0]).toContain("remark"); }); @@ -133,7 +133,7 @@ export const documentTrip = createJourney({ expect(checks.some((check) => check.findings[0]?.includes("no submit gate"))).toBe(true); }); - it("errors on over-budget descriptions", () => { + it("warns on over-budget descriptions", () => { const checks = verifyJourneyFiles([ { path: "journeys/wordy.webmcp.ts", @@ -144,11 +144,28 @@ export const documentTrip = createJourney({ }, ]); const budget = checks.find( - (check) => check.level === "error" && check.summary.includes("budget"), + (check) => check.level === "warning" && check.summary.includes("budget"), ); expect(budget?.findings[0]).toContain("500"); }); + it("counts the tools a journey registers: one per step plus the submit gate", () => { + // goodJourney has one step and one submit, so it registers two tools. + expect(countJourneyTools([{ path: "journeys/a.webmcp.ts", contents: goodJourney }])).toBe(2); + }); + + it("adds journey tools to the surface total and warns past the budget", () => { + const endpointTools = Array.from({ length: 24 }, (_, i) => + reviewedTool({ name: `list-thing-${i}`, withheld: false }), + ); + // 24 endpoints alone are under 25; a journey pushes the real surface past it. + const checks = verifyTools(endpointTools, { journeyToolCount: 4 }); + const surface = checks.find((check) => check.area === "Surface"); + expect(surface?.level).toBe("warning"); + expect(surface?.summary).toContain("28"); + expect(surface?.summary).toContain("4 journey"); + }); + it("warns on more than five steps", () => { const steps = Array.from( { length: 6 }, diff --git a/packages/codegen/src/verify.ts b/packages/codegen/src/verify.ts index bf47550..d357b07 100644 --- a/packages/codegen/src/verify.ts +++ b/packages/codegen/src/verify.ts @@ -183,7 +183,10 @@ function hasBareField( * each check serves is named in its area, so the scorecard maps to the * design doc's table without translation. */ -export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { +export function verifyTools( + tools: ReviewedTool[], + options?: { journeyToolCount?: number }, +): VerifyCheck[] { const registered = tools.filter((tool) => !tool.withheld); const longNames = registered @@ -253,16 +256,17 @@ export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { check("Annotations", readWithoutHint, "reads declare readOnlyHint; content declares its trust"), ]; - // The character budgets are errors: they exist to keep tool text inside - // agent guardrails, and CI gates on them (generation already composes - // within budget, so offenders here are hand-written or overrides). + // Chrome's character budgets are authoring guidance, not spec rules: the + // browser only rejects an empty description or a bad name. So an overrun is + // a warning that points at a shorter rewrite, never an error that blocks CI. + // (A long description is a quality smell, not a broken tool.) const budgetOffenders = [...longDescriptions, ...longParamDescriptions]; if (budgetOffenders.length > 0) { checks.push({ area: "Budgets", summary: `${budgetOffenders.length} over budget`, findings: budgetOffenders, - level: "error", + level: "warning", }); } @@ -277,13 +281,23 @@ export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { }); } - if (registered.length > 25) { + // Journey tools register at runtime, so they are counted by the caller and + // passed in: the surface an agent actually sees is endpoint tools plus + // journey steps plus their submit gates. Missing this would let a journey + // quietly grow the surface past the budget the docs promise is watched. + const journeyToolCount = options?.journeyToolCount ?? 0; + const surfaceTotal = registered.length + journeyToolCount; + if (surfaceTotal > 25) { + const breakdown = + journeyToolCount > 0 + ? `${registered.length} endpoint and ${journeyToolCount} journey` + : `${registered.length}`; checks.push({ area: "Surface", - summary: `${registered.length} registered`, + summary: `${surfaceTotal} registered (${breakdown})`, findings: [ - `${registered.length} tools register on this surface — agents choose measurably worse past a handful. ` + - "Withhold unreviewed tools, or narrow with safety.exclude.", + `${surfaceTotal} tools register on this surface (${breakdown}) — agents choose measurably worse past a handful. ` + + "Withhold unreviewed tools, split journeys, or narrow with safety.exclude.", ], level: "warning", }); @@ -416,7 +430,7 @@ export function verifyJourneyFiles(files: JourneyFileInput[]): VerifyCheck[] { area: "Journeys", summary: `${budget.length} over budget`, findings: budget, - level: "error", + level: "warning", }); } if (warnings.length > 0) { @@ -438,6 +452,20 @@ export function verifyJourneyFiles(files: JourneyFileInput[]): VerifyCheck[] { return checks; } +/** + * How many tools a set of journey files registers on the page: one per step + * plus the submit gate. Used by the surface check, because these tools are + * created at runtime and never pass through the pipeline's tool list. + */ +export function countJourneyTools(files: JourneyFileInput[]): number { + let count = 0; + for (const { contents } of files) { + if (!/createJourney\s*\(/.test(contents)) continue; + count += journeyStepNames(contents).length + 1; + } + return count; +} + /** The --url probe: is the page actually live for a visitor's browser? */ export async function verifyUrl(url: string): Promise { const findings: AuditFinding[] = []; From 049a29ca98e640686f3ae8981339865ed4e2a34d Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:21:23 +0530 Subject: [PATCH 19/41] fix(journeys): honest step hints, late-bound context, bounded descriptions A step's call and run are user code, so the factory cannot know whether a step writes. Steps now inherit the composed tool's readOnlyHint (a free step with no run reads as a read) instead of always claiming read-only, with an opt-in readOnly override for the cases the author knows better. Resolve the WebMCP context at registration time, matching generated tools, so a browser or polyfill that installs WebMCP after the module loads still gets the journey registered. Compose step descriptions within the 500-char budget: keep the journey name, drop the repeated goal, and fit the base when the goal would overflow. --- packages/codegen/assets/journey.webmcp.ts | 59 +++++++++++++++++-- .../fixture/src/webmcp/journey.webmcp.ts | 59 +++++++++++++++++-- 2 files changed, 108 insertions(+), 10 deletions(-) diff --git a/packages/codegen/assets/journey.webmcp.ts b/packages/codegen/assets/journey.webmcp.ts index 93a777d..718e25f 100644 --- a/packages/codegen/assets/journey.webmcp.ts +++ b/packages/codegen/assets/journey.webmcp.ts @@ -44,7 +44,12 @@ type Json = Record; */ export interface ToolStep { /** The generated tool object, e.g. getAutocompleteTool. */ - tool: { description?: string; inputSchema?: Json }; + tool: { + description?: string; + inputSchema?: Json; + /** The generated tool's annotations. The step inherits readOnlyHint from it. */ + annotations?: { readOnlyHint?: boolean }; + }; /** * The raw caller the generated file exports. Receives the step's input * plus the draft so far, so a later step can feed on an earlier one's @@ -59,6 +64,12 @@ export interface ToolStep { description?: string; /** Override the agent-facing input schema. Default: the tool's own. */ input?: Json; + /** + * Override the read-only hint. Default: the composed tool's own hint, so a + * step that composes a read stays read-only and a step that composes a + * write does not. Set this only when you know the step's behavior differs. + */ + readOnly?: boolean; } /** @@ -71,6 +82,11 @@ export interface FreeStep { input: Json; provides: string[]; run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** + * Override the read-only hint. Default: true when the step has no `run` + * (it only stores its input in the draft), false when `run` can do work. + */ + readOnly?: boolean; } export type JourneyStep = ToolStep | FreeStep; @@ -95,6 +111,30 @@ function isToolStep(step: JourneyStep): step is ToolStep { return "tool" in step; } +/** Chrome's published budget for one tool description. */ +const TOOL_DESCRIPTION_MAX = 500; + +/** Fit machine-composed text to a budget, reserving room for the ellipsis. */ +function fitText(text: string, budget: number): string { + if (text.length <= budget) return text; + const slice = text.slice(0, Math.max(1, budget - 1)); + const wordEnd = slice.lastIndexOf(" "); + return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}…`; +} + +/** + * Whether calling this step can change anything outside the draft. A step + * that composes a generated read tool inherits the tool's readOnlyHint; a + * free step that only stores input is a read; everything else defaults to + * "not read-only", because a step's `call`/`run` is user code we cannot + * inspect. We never advertise a write as safe just because it is a step. + */ +function stepReadOnly(step: JourneyStep): boolean { + if (step.readOnly !== undefined) return step.readOnly; + if (isToolStep(step)) return step.tool.annotations?.readOnlyHint === true; + return step.run === undefined; +} + /** "document-trip-search-places" → "Document Trip Search Places" (native UIs). */ function toTitle(kebab: string): string { return kebab @@ -105,8 +145,6 @@ function toTitle(kebab: string): string { } export function createJourney(def: JourneyDef) { - const modelContext = getModelContext(); - /** The shared draft. Page-scoped on purpose: reloads start clean. */ let draft: Json = {}; @@ -122,7 +160,13 @@ export function createJourney(def: JourneyDef) { function stepDescription(step: JourneyStep): string { const base = step.description ?? (isToolStep(step) ? step.tool.description : undefined) ?? "Journey step."; - return `${base} Part of "${def.name}": ${def.goal}`; + const suffix = ` Part of "${def.name}": ${def.goal}`; + if (base.length + suffix.length <= TOOL_DESCRIPTION_MAX) return base + suffix; + // The step's own sentence matters more than repeating the whole goal. + // Keep the journey name (the grouping signal), drop the goal, and fit the + // base so the composed text never exceeds the budget. + const tag = ` Part of "${def.name}".`; + return `${fitText(base, TOOL_DESCRIPTION_MAX - tag.length)}${tag}`; } function stepInput(step: JourneyStep): Json { @@ -143,6 +187,10 @@ export function createJourney(def: JourneyDef) { } async function registerSteps(signal?: AbortSignal): Promise { + // Resolve the context here, not at createJourney() time: a browser or + // polyfill that installs WebMCP after this module loads must still get the + // journey registered. Generated tools look it up the same way. + const modelContext = getModelContext(); if (!modelContext) return; for (const [key, step] of Object.entries(def.steps)) { await modelContext.registerTool( @@ -151,7 +199,7 @@ export function createJourney(def: JourneyDef) { title: toTitle(`${def.name}-${key}`), description: stepDescription(step), inputSchema: stepInput(step), - annotations: { readOnlyHint: true }, + annotations: { readOnlyHint: stepReadOnly(step) }, execute: async (input, context) => { try { await runStep(step, input as Json, context?.signal); @@ -172,6 +220,7 @@ export function createJourney(def: JourneyDef) { } async function registerSubmit(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); if (!modelContext) return; await modelContext.registerTool( { diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts index 93a777d..718e25f 100644 --- a/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts +++ b/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts @@ -44,7 +44,12 @@ type Json = Record; */ export interface ToolStep { /** The generated tool object, e.g. getAutocompleteTool. */ - tool: { description?: string; inputSchema?: Json }; + tool: { + description?: string; + inputSchema?: Json; + /** The generated tool's annotations. The step inherits readOnlyHint from it. */ + annotations?: { readOnlyHint?: boolean }; + }; /** * The raw caller the generated file exports. Receives the step's input * plus the draft so far, so a later step can feed on an earlier one's @@ -59,6 +64,12 @@ export interface ToolStep { description?: string; /** Override the agent-facing input schema. Default: the tool's own. */ input?: Json; + /** + * Override the read-only hint. Default: the composed tool's own hint, so a + * step that composes a read stays read-only and a step that composes a + * write does not. Set this only when you know the step's behavior differs. + */ + readOnly?: boolean; } /** @@ -71,6 +82,11 @@ export interface FreeStep { input: Json; provides: string[]; run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** + * Override the read-only hint. Default: true when the step has no `run` + * (it only stores its input in the draft), false when `run` can do work. + */ + readOnly?: boolean; } export type JourneyStep = ToolStep | FreeStep; @@ -95,6 +111,30 @@ function isToolStep(step: JourneyStep): step is ToolStep { return "tool" in step; } +/** Chrome's published budget for one tool description. */ +const TOOL_DESCRIPTION_MAX = 500; + +/** Fit machine-composed text to a budget, reserving room for the ellipsis. */ +function fitText(text: string, budget: number): string { + if (text.length <= budget) return text; + const slice = text.slice(0, Math.max(1, budget - 1)); + const wordEnd = slice.lastIndexOf(" "); + return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}…`; +} + +/** + * Whether calling this step can change anything outside the draft. A step + * that composes a generated read tool inherits the tool's readOnlyHint; a + * free step that only stores input is a read; everything else defaults to + * "not read-only", because a step's `call`/`run` is user code we cannot + * inspect. We never advertise a write as safe just because it is a step. + */ +function stepReadOnly(step: JourneyStep): boolean { + if (step.readOnly !== undefined) return step.readOnly; + if (isToolStep(step)) return step.tool.annotations?.readOnlyHint === true; + return step.run === undefined; +} + /** "document-trip-search-places" → "Document Trip Search Places" (native UIs). */ function toTitle(kebab: string): string { return kebab @@ -105,8 +145,6 @@ function toTitle(kebab: string): string { } export function createJourney(def: JourneyDef) { - const modelContext = getModelContext(); - /** The shared draft. Page-scoped on purpose: reloads start clean. */ let draft: Json = {}; @@ -122,7 +160,13 @@ export function createJourney(def: JourneyDef) { function stepDescription(step: JourneyStep): string { const base = step.description ?? (isToolStep(step) ? step.tool.description : undefined) ?? "Journey step."; - return `${base} Part of "${def.name}": ${def.goal}`; + const suffix = ` Part of "${def.name}": ${def.goal}`; + if (base.length + suffix.length <= TOOL_DESCRIPTION_MAX) return base + suffix; + // The step's own sentence matters more than repeating the whole goal. + // Keep the journey name (the grouping signal), drop the goal, and fit the + // base so the composed text never exceeds the budget. + const tag = ` Part of "${def.name}".`; + return `${fitText(base, TOOL_DESCRIPTION_MAX - tag.length)}${tag}`; } function stepInput(step: JourneyStep): Json { @@ -143,6 +187,10 @@ export function createJourney(def: JourneyDef) { } async function registerSteps(signal?: AbortSignal): Promise { + // Resolve the context here, not at createJourney() time: a browser or + // polyfill that installs WebMCP after this module loads must still get the + // journey registered. Generated tools look it up the same way. + const modelContext = getModelContext(); if (!modelContext) return; for (const [key, step] of Object.entries(def.steps)) { await modelContext.registerTool( @@ -151,7 +199,7 @@ export function createJourney(def: JourneyDef) { title: toTitle(`${def.name}-${key}`), description: stepDescription(step), inputSchema: stepInput(step), - annotations: { readOnlyHint: true }, + annotations: { readOnlyHint: stepReadOnly(step) }, execute: async (input, context) => { try { await runStep(step, input as Json, context?.signal); @@ -172,6 +220,7 @@ export function createJourney(def: JourneyDef) { } async function registerSubmit(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); if (!modelContext) return; await modelContext.registerTool( { From 822809bdafcf82c380446191f9d06d7b6f3bf5b1 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:21:32 +0530 Subject: [PATCH 20/41] docs: bring the existing docs back in line with the shipped surface Root README: drop the deleted LLM flags, mark the character budgets, output cap, and exposedTo as shipped, describe grouping and journeys and the skill file, correct the generated-tool example, and fix the example package name. journeys.md spec: record that the shipped shape is TypeScript files that call createJourney, not the config block this early spec described. Journeys docs and FAQ: the surface check warns and now counts journey tools; absorption and step safety are described as conventions the developer follows, not guarantees the tool enforces. Skill file: stop claiming a name override in .webmcp-codegen.json (only description, enabled, and field text are supported), and teach the derived read-only step hint. --- README.md | 50 ++++++++++++------- docs/specs/journeys.md | 7 ++- packages/codegen/assets/skill/SKILL.md | 10 ++-- .../.agents/skills/webmcp-tools/SKILL.md | 10 ++-- site/content/docs/journeys-faq.mdx | 18 ++++--- site/content/docs/journeys.mdx | 11 ++-- 6 files changed, 67 insertions(+), 39 deletions(-) diff --git a/README.md b/README.md index 1298d7f..b852d62 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@

webmcp-stack

The open-source developer stack for WebMCP.

-

Today: codegen. The goal is the whole agent-surface lifecycle in one stack: Generate, Understand, Review, Test, Control, Observe, Secure.

+

Today: codegen. It generates a safe, reviewable agent surface from the contract you already have, and it is built to grow into the rest of the lifecycle.

npm version MIT license @@ -48,7 +48,9 @@ You can, and it works. What you get back is different every time, and nothing ch - **Decides what agents may do.** Every endpoint is classified read, write, or destructive from the HTTP verb, corrected when the name disagrees (`POST /orders/{id}/cancel` is destructive, `POST /search` is a read). Reads work immediately. Everything else is generated but not registered, so enabling a write is a deliberate edit. Webhooks are skipped, auth and admin endpoints are flagged, and anything it cannot classify starts disabled. - **Asks the user before any mutation.** Write and destructive tools confirm each call with the user (a plain dialog you can replace). The confirmation lives in the generated region of the file, so it cannot be edited away and survive regeneration. -- **Writes the text agents read.** Constraints become sentences ("A number from 30 to 600."), names come from intent (`generate-story`, not `post-trips-trip-id-story-generate`), and every description says what the tool returns. Your own text always wins; machine-written text is marked and flagged in the audit. +- **Writes the text agents read.** Constraints become sentences ("A number from 30 to 600."), names come from intent (`generate-story`, not `post-trips-trip-id-story-generate`), and every description says what the tool returns. Your own text always wins; machine-written text is marked and flagged in the audit. Chrome's character budgets are treated as guidance, not law: machine text is composed to fit, author text is never silently shortened, and `verify` warns on an overrun. +- **Groups actions the API split.** A begin/end pair like `request-upload` plus `uploads/{uploadId}/complete` is one action, not two. The generator detects the pair, threads the first response into the second by exact name, and adds one withheld coarse tool next to the members. A pair it cannot thread is skipped with a note. +- **Scaffolds journeys and an agent skill.** `journey.webmcp.ts` plus `journeys/*.webmcp.ts` express a multi-step flow: a page-scoped draft, one tool per step, and a submit gate that refuses until every step is done and confirms with the human before the real write. The barrel registers them and `verify` lints the files. `.agents/skills/webmcp-tools/SKILL.md` teaches your own coding agent the rules. - **Says what can be trusted.** Free-text outputs get `untrustedContentHint`; PII-looking fields are named in the report and in a comment in the file; mutating tools on authenticated endpoints carry a session warning; `execute` calls your real endpoint, so your own validation still runs. - **Annotates real forms.** A tool that maps to a visible `` can annotate it in place, so the agent fills the same controls the user sees, and the user reviews and submits the write. @@ -57,7 +59,7 @@ You can, and it works. What you get back is different every time, and nothing ch A real tool from a real app: `create-trip`, one of 70+ tools generated for [beenthere.page](https://beenthere.page) from its OpenAPI spec, shortened for the README: ```ts -// --- webmcp-codegen: generated. Do not edit this region. --- +// ─── webmcp-codegen: generated. Do not edit this region. ─── /** * Create a new trip. Returns the trip. * Source: POST /v1/trips/ (openapi). Risk: write-confirm. @@ -65,23 +67,34 @@ A real tool from a real app: `create-trip`, one of 70+ tools generated for [been */ export const createTripTool = { name: "create-trip", + title: "Create Trip", description: "Create a new trip. Returns the trip.", inputSchema: createTripInputSchema, // title, dates, location, theme, field notes - annotations: { readOnlyHint: false, untrustedContentHint: true }, + annotations: { + readOnlyHint: false, + untrustedContentHint: true, + consequentialHint: false, + }, }; +// Journeys and your own code compose this raw caller; executeCreateTrip wraps it +// in the agent-facing result shape. +export async function fetchCreateTrip(input: CreateTripInput, signal?: AbortSignal) { + const data = await callApi("/v1/trips/", { method: "POST", body: { ... }, signal }); + return data; +} + // A write tool, so it is withheld: the registration is generated but commented // out, including the built-in user confirmation, until you enable it: // const confirmed = await requestUserConfirmation( // "Allow the agent to: Create a new trip. Returns the trip.", // ); -// --- webmcp-codegen: end generated. Your code below survives regeneration. --- +// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── export async function executeCreateTrip(input: CreateTripInput, signal?: AbortSignal) { return toolDisabled("create-trip.webmcp.ts"); - // Uncomment to go live: - // const data = await callApi("https://api.beenthere.page/v1/trips/", { method: "POST", ... }); - // return toolResult(data); + // Uncomment to go live, and uncomment the registration above: + // return toolResult(await fetchCreateTrip(input, signal)); } ``` @@ -90,9 +103,9 @@ The bar the ecosystem is converging on (Chrome's WebMCP best-practices and tool- | The bar | Today | |---|---| | Verb-first, intent-shaped names, 30 characters or fewer | Enforced on every run | -| Descriptions say what the tool does and when, positively, within 500 characters; every parameter described within 150 | Assembled on every run; length budgets measured by `verify` next | -| Outputs within 1.5K characters; errors that help recovery; user-written content marked `untrustedContentHint` | Errors and annotations enforced; output budget next | -| Exposure as a decision: `readOnlyHint`, registration only where usable, `exposedTo` origin scoping | Annotations and withheld-by-default enforced; `exposedTo` lands as the runtime stabilizes | +| Descriptions say what the tool does and when, positively, within 500 characters; every parameter described within 150 | Assembled on every run; author text is never shortened, and `verify` warns on overruns | +| Outputs within 1.5K characters; errors that help recovery; user-written content marked `untrustedContentHint` | Errors, annotations, and the output cap enforced; the cap lives in the generated runtime | +| Exposure as a decision: `readOnlyHint`, registration only where usable, `exposedTo` origin scoping | Annotations and withheld-by-default enforced; `exposedTo` is a config pass-through to registration | | The human in the loop: visible page effects, confirmation on consequential actions | Confirmation enforced, in the generated region where it can't be edited away; visible-effect hook scaffolded | | No steering: descriptions never instruct the agent or encode flow control | Flagged in the audit | | The schema is not the security boundary; the app still validates at run time | Stated in the output; `execute` calls your real endpoint | @@ -101,24 +114,23 @@ The bar the ecosystem is converging on (Chrome's WebMCP best-practices and tool- ## Changing things later -- **`.webmcp-codegen.json`**: per-tool overrides for descriptions, names, enabled state, fields. Applied last, so your text always wins. +- **`.webmcp-codegen.json`**: per-tool overrides for descriptions, enabled state, and field text. Applied last, so your text always wins. - **Dashboard** (`dev`): edit descriptions, toggle tools, run tools against real endpoints. Writes back to the overrides file. - **Audit**: problems reported in plain language every run: missing descriptions, mislabeled verbs, PII in outputs, agent-instruction smells, oversized surfaces. Errors block file writing; warnings do not. - **`verify`**: the standard, checked locally. Built for CI. -- **LLM help if you want it**: `--suggest` proposes tools worth declaring, `--llm` drafts descriptions. Advisory only: it never classifies risk, never changes exit codes, and a plain `generate` never makes a network call. +- **Skill file**: `.agents/skills/webmcp-tools/SKILL.md` teaches your own coding agent the rules (naming, description budgets, journeys). Regenerated on every `generate`; add your own skill directory to stack project-specific rules on top. ## Where this is going Codegen first: - **More sources.** OpenAPI and validation schemas today, tRPC on the list. The rule holds: contracts, not codebases, and the CLI never scans app code. -- **Journeys.** One tool per CRUD operation is rarely the right shape; agents do better with a few coarse, intent-level tools. Declare a flow once (draft store, step tools, a submit gate), run it in the dashboard, review the generated Mermaid diagram. -- **The dashboard becomes the review surface.** LLM suggestions arrive as an accept/reject queue that writes to the overrides file. In progress now. -- **Evals for the LLM layer.** A prompt change becomes a visible regression, not a vibe. +- **The dashboard becomes the review surface.** A browse-and-score report over the generated surface, with the editing UI kept for the overrides it writes. Parked until the audit package exists. +- **Skill-file evals.** Shipped at `packages/codegen/evals/skill/`: a prompt set, a generated fixture, deterministic graders, and a control case that runs the sharpest prompt with the skill removed, so you can tell when a model has absorbed the rules. Then the stack around it: **audit** (point it at a URL, get a report on the surface a visiting agent would find) and **telemetry** (how agents actually use your tools). The goal is one stack where each tool covers one stage of the lifecycle and they compound. The bet underneath: websites are growing an agent-facing surface the way they grew APIs, and that surface needs the same kind of tooling, with higher stakes, because the caller is a model acting as your user, inside your page. -The guarantees hold through all of it: the repo pins the WebMCP draft it targets and watches for spec drift; your overrides, execute bodies, and review decisions survive every regeneration; breaking changes print the exact fix. Deterministic where possible, LLM where useful, honest always. +The guarantees hold through all of it: the repo pins the WebMCP draft it targets and watches for spec drift; your overrides, execute bodies, and review decisions survive every regeneration; breaking changes print the exact fix. Deterministic where it can be, honest always. ## Principles @@ -134,7 +146,7 @@ The guarantees hold through all of it: the repo pins the WebMCP draft it targets | `packages/codegen` | `@webmcp-stack/codegen` | The CLI and the generation pipeline: sources (OpenAPI, validation schemas), outputs, the safety audit, the dev dashboard. | | `examples/openapi-petstore` | private | Example app with tools generated from the Petstore OpenAPI spec. | | `site/` | private | Landing page and documentation (Next.js + Fumadocs). | -| `docs/` | - | Design specs (`specs/`), decision notes (`notes/`), and [what this project is](./docs/about.md). | +| `docs/` | - | [About](./docs/about.md), design specs (`specs/`), decision notes (`notes/`), research (`research/`), and reviews (`reviews/`). | | `scripts/` | - | Committed git hooks (lint on commit, lint + typecheck + test before push). | | `brand/` | - | Logo and brand assets. | @@ -152,7 +164,7 @@ pnpm lint:fix ``` ```bash -pnpm --filter openapi-petstore dev # example app +pnpm --filter example-openapi-petstore dev # example app pnpm --filter site dev # landing page & docs on :3001 ``` diff --git a/docs/specs/journeys.md b/docs/specs/journeys.md index cb5736f..0cc98be 100644 --- a/docs/specs/journeys.md +++ b/docs/specs/journeys.md @@ -8,7 +8,12 @@ > tools that pipeline produces). Evidence: > `docs/research/2026-09-02-chrome-webmcp-docs-analysis.md`. > -> Status: spec, not yet implemented. Ships after the generation pipeline (0.5). +> Status: shipped in 0.9, but in a different shape than this early spec. The +> config-declaration design below (a `journeys:` block) was never built. +> What ships: TypeScript files under the tools output's `journeys/` folder that +> import `createJourney` from the generator-owned `journey.webmcp.ts`, plus a +> `verify` lint over those files. Kept for decision history; treat the config +> block below as superseded. ## The problem diff --git a/packages/codegen/assets/skill/SKILL.md b/packages/codegen/assets/skill/SKILL.md index b2b496d..3edf9c1 100644 --- a/packages/codegen/assets/skill/SKILL.md +++ b/packages/codegen/assets/skill/SKILL.md @@ -72,8 +72,8 @@ when the user's request is casual. - Each `*.webmcp.ts` has a generated region between the `webmcp-codegen` markers — never edit inside it; regeneration rewrites it. Your work goes below the marker (the `execute` body) or in - `.webmcp-codegen.json` (description/name/enabled overrides, which survive - regeneration and always win over generated text). + `.webmcp-codegen.json` (description, enabled, and field-text overrides, which + survive regeneration and always win over generated text). - After editing tools, run `npx @webmcp-stack/codegen verify` and fix what it reports. @@ -124,5 +124,7 @@ export const documentTrip = createJourney({ (`fetchX`); the submit's `run` is the real write tool's `execute`, so its confirmation and validation still apply. Never write a direct `fetch` in a journey file — `verify` flags it. -- The submit gate is the only write in a journey; step tools are reads or - draft-writes and stay read-only. +- Keep real writes in the submit gate. A tool-backed step inherits the composed + tool's read-only hint, and a free step that only stores input is read-only + too. Never route a mutating call through a step: it would be advertised as + safe and skip the confirmation the submit performs. diff --git a/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md b/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md index b2b496d..3edf9c1 100644 --- a/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md +++ b/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md @@ -72,8 +72,8 @@ when the user's request is casual. - Each `*.webmcp.ts` has a generated region between the `webmcp-codegen` markers — never edit inside it; regeneration rewrites it. Your work goes below the marker (the `execute` body) or in - `.webmcp-codegen.json` (description/name/enabled overrides, which survive - regeneration and always win over generated text). + `.webmcp-codegen.json` (description, enabled, and field-text overrides, which + survive regeneration and always win over generated text). - After editing tools, run `npx @webmcp-stack/codegen verify` and fix what it reports. @@ -124,5 +124,7 @@ export const documentTrip = createJourney({ (`fetchX`); the submit's `run` is the real write tool's `execute`, so its confirmation and validation still apply. Never write a direct `fetch` in a journey file — `verify` flags it. -- The submit gate is the only write in a journey; step tools are reads or - draft-writes and stay read-only. +- Keep real writes in the submit gate. A tool-backed step inherits the composed + tool's read-only hint, and a free step that only stores input is read-only + too. Never route a mutating call through a step: it would be advertised as + safe and skip the confirmation the submit performs. diff --git a/site/content/docs/journeys-faq.mdx b/site/content/docs/journeys-faq.mdx index 9057170..d152fb8 100644 --- a/site/content/docs/journeys-faq.mdx +++ b/site/content/docs/journeys-faq.mdx @@ -163,13 +163,17 @@ about the product; our CLI can't. ## Won't journeys make my tool list even bigger? -They shouldn't — followed properly, they shrink it. The rule: a tool that only -makes sense inside a flow doesn't get registered on its own; its journey step -is its only face. Writes reachable through a journey submit stay withheld as -standalone tools forever. So endpoints collapse into flows instead of -double-registering. And `verify`'s surface-size check counts *registered* -tools, journey tools included — if the list grows past what's good for agents, -CI fails. That's enforced, not aspirational. +They can, if you let them. `verify` now counts journey tools in the surface +total (one per step plus the submit gate) and warns when the whole surface grows +past what's good for agents. It's a warning, not a failed build: the browser has +no limit on how many tools you register, only agent quality does. + +Keeping the surface small is a convention you follow, not something the +generator does for you. A read tool that a journey uses still registers on its +own, so if it only makes sense inside the flow, withhold it in +`.webmcp-codegen.json` or exclude it with `safety.exclude`. Writes reachable +through a journey submit stay withheld as standalone tools unless you enable +them, so leave them withheld and the submit is their only door. ## What if two steps use the same draft field name? diff --git a/site/content/docs/journeys.mdx b/site/content/docs/journeys.mdx index 3cd940e..0ef4391 100644 --- a/site/content/docs/journeys.mdx +++ b/site/content/docs/journeys.mdx @@ -142,10 +142,13 @@ export const documentTrip = createJourney({ stays withheld as a standalone tool forever; the journey's submit gate becomes its only public door. Journeys should shrink the surface an agent sees, not grow it. -- **The safety doesn't depend on the journey file.** The gate, the confirmation, - and the draft clearing are enforced by the shared helper, not by anything you - write in the definition. A journey file can be written sloppily; it can't - skip the human's yes. +- **The submit gate can't be edited away, but a step is still your code.** The + gate, the confirmation, and the draft clearing live in the shared helper, so a + journey file can't remove them. A step's `call` and `run`, though, are code you + or your agent write. Keep steps to reads and draft writes; put anything that + changes real data in the submit, where the human is asked first. `verify` + only marks a step read-only when the tool it composes is a read, but it can't + inspect arbitrary code, so the discipline is still yours. - **The draft is per page load, per journey.** Within one continuous page view it's fully reliable. Across a reload, it's gone — deliberately. Two parallel attempts at the same journey on one page would share one draft; in practice From 95a918c7d355c4bc283ab65223de993eb5ef5912 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:21:41 +0530 Subject: [PATCH 21/41] docs: add the missing guides for why, what happens next, and prompting Three gaps were filled: - "Why give your site tools for agents" explains the problem, why a small intent-shaped surface beats a mirror of the API, what the tool does and does not do, and where WebMCP honestly is today. - "After you generate" is the loop that turns a generated surface into one you trust: review it, enable writes deliberately, fix descriptions, test in the browser, write journeys, wire CI, regenerate. - "Working with your coding agent" covers the shipped skill file, how to prompt an agent to improve descriptions and write journeys, example prompts, and what to check when it is done. The Introduction is reframed around the same goal, with an honest "what it does not do" section and links into the new pages; the Quickstart points onward. --- site/content/docs/after-you-generate.mdx | 146 ++++++++++++++++++ site/content/docs/index.mdx | 114 +++++++------- site/content/docs/meta.json | 11 +- site/content/docs/quickstart.mdx | 16 ++ site/content/docs/why-agent-tools.mdx | 107 +++++++++++++ site/content/docs/working-with-your-agent.mdx | 131 ++++++++++++++++ 6 files changed, 469 insertions(+), 56 deletions(-) create mode 100644 site/content/docs/after-you-generate.mdx create mode 100644 site/content/docs/why-agent-tools.mdx create mode 100644 site/content/docs/working-with-your-agent.mdx diff --git a/site/content/docs/after-you-generate.mdx b/site/content/docs/after-you-generate.mdx new file mode 100644 index 0000000..842f2ec --- /dev/null +++ b/site/content/docs/after-you-generate.mdx @@ -0,0 +1,146 @@ +--- +title: After you generate +description: The tools exist. Now decide what agents may actually do, check it, ship journeys for the real goals, and keep it honest as your API changes. +--- + +Generating is the easy part. This page is the rest: the loop you actually run, +in the order that works. + + + If you have not generated anything yet, start with the + [Quickstart](/docs/quickstart). This page assumes you have a `src/webmcp` + folder and a passing `generate`. + + +## 1. Look at what you got + +Before you change anything, read the report and the scorecard. + +```bash +npx @webmcp-stack/codegen dev # browse and inspect every tool +npx @webmcp-stack/codegen verify # the quality scorecard, exits 1 on errors +``` + +The dashboard is for understanding, not decorating: every tool with its route, +risk label, and findings. `verify` tells you where the surface is weak. Fix +errors first; they block generation. Warnings can wait until the tool matters. + +## 2. Decide what agents may actually do + +Reads are registered so you can see the surface. Writes and destructive tools +are generated but **not registered**. Enabling one is a deliberate edit, in the +dashboard or in `.webmcp-codegen.json`: + +```json title=".webmcp-codegen.json" +{ + "overrides": { + "delete-trip": { "enabled": true } + } +} +``` + +Turn on only what you would let a careful assistant do, and turn them on one at +a time. A surface with four reviewed writes is worth more than one with forty +unreviewed ones. If an endpoint exists but an agent should never touch it, leave +it withheld or exclude it with `safety.exclude`. + +## 3. Fix the descriptions that matter + +This is the highest-leverage edit in the whole loop. A tool's name and +description are literally part of the prompt the model reasons over, and the +default from your spec is often a terse summary written for humans. + +Open the tools that matter and rewrite the description as three things: what it +does, when to use it, and what it returns. + +```text +Before: Get trips +After: List the trips the signed-in user has saved, newest first. Returns an + array of trips with title, dates, and place. +``` + +Descriptions live in the generated region, so edit them in +`.webmcp-codegen.json` (the dashboard writes here too), never in the file above +the marker. Your overrides survive every regeneration. + +## 4. Test it the way a person would + +Open your app in Chrome, turn on `chrome://flags/#enable-webmcp-testing`, and +use the DevTools panel under **Application > WebMCP**. Call a read tool and +check the data. Call a write tool and go all the way through the confirmation +dialog, because that is the path a real user takes. + +The dashboard can run tools too, but it runs them server-side, without your +browser session. Use it to check that the request is built correctly; use +DevTools to check it works while signed in. See [Chrome DevTools WebMCP +panel](/docs/devtools). + +## 5. Write journeys for the goals that take several calls + +Some things a user asks for cannot be one tool call. "Log the trip I took to +Lisbon" is search, then details, then a confirmed create. That shape is a +[journey](/docs/journeys): a shared draft, one tool per step, and a submit gate +that refuses until every step is done and asks the human before the real write. + +Journeys are the part the generator cannot derive from your API, because the +flow lives in your product. Your coding agent writes them, guided by the skill +file. [Working with your coding agent](/docs/working-with-your-agent) shows how +to ask. + +## 6. Make the effects visible + +When a tool changes something, the page should move. An agent acting while a +person watches, with no visible change, is a bug in the experience even when the +request succeeded. Update the same UI your human flow updates: navigate, +invalidate a query, dispatch an event. + +The generated tools leave a marked spot for exactly this, below the marker where +you own the code. The pattern is in [Make the effect +visible](/docs/visible-effects). + +## 7. Put the checks in CI + +`verify` exits `1` on errors, so it is a one-line CI gate: + +```bash +npx @webmcp-stack/codegen verify +``` + +Run it on pull requests that touch your spec or your tools. A destructive +endpoint that lost its classification, a description that tries to instruct the +agent, a tool with nothing an agent can act on: all of it shows up before it +ships. Add `generate --dry-run` if you also want the audit for the spec itself. + +## 8. Regenerate as your API changes + +The point of generating is that the second run is safe. When your API changes, +re-run `generate`: + +```bash +npx @webmcp-stack/codegen generate +``` + +Only the generated region of each file updates. Your `execute()` bodies, your +overrides, and your dashboard edits stay put. If you hand-edited a generated +region, the tool leaves your file alone and writes its version to a `.new` +sibling instead of overwriting you. See [The regeneration +contract](/docs/regeneration). + +## 9. Keep the surface small + +The healthiest sign is not "we exposed everything". It is a surface an agent can +hold in its head. Three habits do most of the work: + +- Withhold anything an agent should not do on its own. +- Prefer a journey over exposing each step as a standalone tool. +- When the surface grows, ask whether two tools are really one intent. + +`verify` warns when the surface grows past what agents handle well, and counts +journey tools too, so this stays visible instead of drifting. + +## 10. When something breaks + +Most first-run problems are a URL or a session, not the tool. The +[Troubleshooting](/docs/troubleshooting) page covers the common ones: a 404 on a +relative URL, the dashboard working where the browser does not, and a +withheld tool that never appears. diff --git a/site/content/docs/index.mdx b/site/content/docs/index.mdx index b500390..49411b7 100644 --- a/site/content/docs/index.mdx +++ b/site/content/docs/index.mdx @@ -1,80 +1,90 @@ --- title: Introduction -description: Generate safe, typed, human-reviewed WebMCP tools from your OpenAPI spec or validation schemas. +description: Generate safe, reviewable WebMCP tools from the API contract you already have, so agents can act on your site without guessing. --- ## The problem -WebMCP lets your web app expose tools that AI agents can call, but somebody has to write -those tools. Today that means hand-writing a `registerTool()` call per action: a name, a -description, a JSON Schema for the inputs, an `execute` function. For one endpoint that's a -chore. For an API with seventy endpoints it's a project nobody starts. +People are already pointing agents at the sites they use. Today those agents +read your HTML and guess which control does what. [WebMCP](https://github.com/webmachinelearning/webmcp) +is the browser's answer: your page can register a small set of typed tools that +an agent calls directly, in the signed-in user's session. -And the details matter more than they look: a tool's **description is literally part of the -prompt** the agent reasons over, and its **schema is the contract** the agent fills in. -Hand-written copies of things your code already knows go stale, drift, and lie. +Somebody has to write those tools. Hand-writing a `registerTool()` call per +action, a name, a description, a JSON Schema, an `execute` function, is a chore +for one endpoint and a project nobody starts for seventy. And the details matter +more than they look: a tool's **description is part of the prompt** the agent +reasons over, and its **schema is the contract** the agent fills in. Hand-written +copies of what your code already knows drift, and drift means the agent calls +the wrong thing. -## What webmcp-codegen does +## What webmcp-stack does -You already have the source of truth: an OpenAPI spec, or validation schemas (zod, valibot, -arktype, TypeBox) your app uses. webmcp-codegen turns them into WebMCP tools, written as real -TypeScript files in your repo: +You already have the source of truth: an OpenAPI spec, or the validation +schemas (zod, valibot, arktype, TypeBox) your app uses. `@webmcp-stack/codegen` +turns them into WebMCP tools, written as real TypeScript files in your repo: ```bash npx @webmcp-stack/codegen generate ``` -``` -Detected apps/server/openapi/openapi.json (override with --spec) -Found your web app: apps/web (next) - -webmcp-codegen: 7 tool(s) - - note: Stripped the shared "v1" version prefix from tool names. +In that one run, three things happen. Tools are generated: reads work +immediately, mutations are generated but withheld until you enable them. Bad +agent endpoints are filtered or flagged: webhooks, auth, admin. And registration +is wired into your app, as two additive lines you can see and undo. - 1 endpoint(s) skipped: - POST /v1/payments/webhook: a webhook receives server callbacks +Beyond the endpoints: - list-trips [read] ← GET /v1/trips/ - create-trip [write] withheld - delete-trip [destructive] withheld - ... - - registration: - created apps/web/src/webmcp/register.tsx - added 2 lines to apps/web/src/app/layout.tsx -``` - -Three things happen in that one run. Tools are generated (read tools with working -implementations, mutations generated but disabled until you uncomment them). Endpoints that -should not be agent tools are filtered or flagged (webhooks, auth, admin). And registration -is wired into your app, two additive lines you can see and undo. +- **Handshake grouping.** Two calls that are really one action (request an + upload, then complete it) become one withheld coarse tool. +- **Journeys.** Multi-step goals get a shared draft, one tool per step, and a + submit gate that asks the human before the real write. See + [Journeys](/docs/journeys). +- **A skill file.** `generate` writes `.agents/skills/webmcp-tools/SKILL.md` so + your own coding agent knows the rules when it extends the surface. See + [Working with your coding agent](/docs/working-with-your-agent). ## Three promises -**Tools work out of the box.** The spec knows the method, path, and parameters, so read -tools are born calling your real API with the signed-in user's session. Mutations get the -same working code, commented out, one deliberate edit away. +**Tools work out of the box.** The spec knows the method, path, and parameters, +so read tools are born calling your real API with the signed-in user's session. +Mutations get the same working code, commented out, one deliberate edit away. -**You own the output.** Generated files are plain TypeScript in your repo. No runtime -dependency on webmcp-codegen. Uninstall it after generating and everything still works. +**You own the output.** Generated files are plain TypeScript in your repo. No +runtime dependency on webmcp-stack. Uninstall it after generating and everything +still works. -**Regeneration never clobbers your code.** The API contract lives above a marker line and -regenerates freely. Your code lives below it and is never touched. Hand-edit the generated -region and you get a `.new` file to merge, never a silent overwrite. +**Regeneration never clobbers your code.** The API contract lives above a marker +line and regenerates freely. Your code lives below it and is never touched. +Hand-edit the generated region and you get a `.new` file to merge, never a +silent overwrite. -## See it before you run it +## What it does not do -Every claim on this page is verifiable without installing anything: +Being clear about the edges is part of using it well: -```bash -npx @webmcp-stack/codegen generate --dry-run -``` +- It does not decide your product's intents. That is your knowledge, or your + agent's, guided by the skill file. +- It does not make an unsafe API safe. Every tool calls your real endpoint, so + your server still validates, authorizes, and rate-limits. +- It does not run a model. Generation is deterministic, no key, no network. + +## Where to start -A dry run writes nothing. It shows you the tools, the safety report, and the plan. + + + Why a small, well-described surface matters, and what a model can and cannot do with your site. + + + One command, working tools, wired into your app. + + + The loop that turns a generated surface into one you trust. + + ## Status -WebMCP is an early-stage spec. The tools work today in Chrome behind a flag, and elsewhere -with a small polyfill. OpenAPI is the supported source; tRPC, Zod, and Prisma are on the -roadmap. +WebMCP is an early-stage spec. The tools work today in Chrome behind a flag and +in the origin trial, and elsewhere with a small polyfill. OpenAPI is the +supported source; validation schemas are supported; tRPC is on the roadmap. diff --git a/site/content/docs/meta.json b/site/content/docs/meta.json index fca9225..707c7d1 100644 --- a/site/content/docs/meta.json +++ b/site/content/docs/meta.json @@ -2,15 +2,18 @@ "title": "webmcp-stack", "pages": [ "index", + "why-agent-tools", "quickstart", - "cli", - "configuration", + "after-you-generate", + "working-with-your-agent", + "journeys", + "journeys-faq", "safety", "regeneration", "visible-effects", - "journeys", - "journeys-faq", "guides", + "configuration", + "cli", "devtools", "troubleshooting" ] diff --git a/site/content/docs/quickstart.mdx b/site/content/docs/quickstart.mdx index 0d1452c..df623ac 100644 --- a/site/content/docs/quickstart.mdx +++ b/site/content/docs/quickstart.mdx @@ -76,3 +76,19 @@ the marker never changes; your code below it is never touched. Add summaries to your OpenAPI operations as you go. A tool's description goes straight into the agent's prompt, and the audit reminds you when one is missing. + +## 6. Keep going + +Generation is the start of the loop, not the end. Where to read next: + + + + Review the surface, enable writes deliberately, add the checks to CI, and regenerate as your API changes. + + + How to prompt your own agent to improve descriptions and write journeys. + + + The multi-step primitive for goals that take more than one call. + + diff --git a/site/content/docs/why-agent-tools.mdx b/site/content/docs/why-agent-tools.mdx new file mode 100644 index 0000000..9d10134 --- /dev/null +++ b/site/content/docs/why-agent-tools.mdx @@ -0,0 +1,107 @@ +--- +title: Why give your site tools for agents +description: Agents already read your pages and guess. A small set of well-described tools is how your site takes part on purpose, instead of by accident. +--- + +## Agents are already using your site + +An agent is a program that reads a page and then acts. It books the flight, files +the expense, cancels the subscription, checks the order. People are already +pointing agents at the sites they use, and those agents are reading your HTML +and guessing which button means "refund". + +That guess is the problem. HTML was built to be rendered for a person, not +called by a program. The agent sees a wall of markup, infers that the big red +button is the destructive one, and sometimes it is right. WebMCP is the +browser's answer: a page can register a small set of typed tools, with names, +descriptions, and input schemas, that an agent calls directly, in the signed-in +user's session. + +So the choice is not whether agents will act on your site. It is whether they +act through something you described, or through something they inferred. + +## A few good tools beat a mirror of your API + +The instinct is to expose everything. Resist it. + +A tool's description is part of the prompt the model reasons over. Fifty tools +make the model choose worse, not better. The surface agents use well is small +and intent-shaped: + +- `search-flights` instead of `get-v1-flights` +- `cancel-order` instead of `delete-order-item` +- `document-trip` instead of `post-trips` plus `post-trips-id-media` + +Think about the handful of things a helpful person would do on your site for +someone else. Those are the tools. The rest is API you keep for yourself. + +## It is a safety decision, not just a feature + +The caller is a model acting as your user, and it may be reading page content an +attacker influenced. Every tool you expose is both an API and a security +surface, and more tools means more chances to pick the wrong one. A few rules +follow from that, and this tool applies them by default: + +- Reads can work immediately. They change nothing. +- Writes stay off until a human turns them on, one at a time. +- Destructive actions ask the user to confirm before every call. +- The schema is not the security boundary. Your server still validates every + call, because the tool calls your real endpoint, not a generated shortcut. + +## What this tool does for you + +`@webmcp-stack/codegen` turns the contract you already have, an OpenAPI spec or +your validation schemas, into real TypeScript files in your repo: + +- It names and describes each endpoint the way an agent reads it, and writes + the input schema from the types you already defined. +- It classifies every endpoint read, write, or destructive, and withholds the + ones an agent should not see yet. +- It generates the user-confirmation step for mutations in a region you cannot + accidentally edit away. +- It scaffolds a skill file so your own coding agent writes the pieces that need + product knowledge, like journeys, in the shape this tool expects. +- It audits the result, in plain language, and `verify` fails CI when something + is wrong. + +## What it does not do + +Being clear about the edges is part of using it well: + +- **It does not decide your product's intents.** An OpenAPI spec says + `POST /v1/trips` exists. It does not say that creating a trip is really + "search, set details, confirm". That lives in your product, and a person or + their agent has to write it. See [Working with your coding + agent](/docs/working-with-your-agent). +- **It does not make an unsafe API safe.** It calls your real endpoint on + purpose, so your validation, permissions, and rate limits still do the real + work. +- **It does not guarantee an agent uses your tools well.** Good names and + descriptions make it far more likely. Nothing makes it certain. +- **It does not run a model.** Generation is deterministic. No key, no network, + no surprises between two runs. + +## Honesty about where WebMCP is + +WebMCP is early. The tools work today in Chrome behind a flag and in the origin +trial, and elsewhere with a small polyfill. The specification is still moving, +and it may change in ways that affect what you generated. This project pins the +draft it targets and reports spec drift, but you should treat the generated +surface as something you review, not something you set and forget. + +That is also why the output is plain files in your repo with no runtime +dependency. If this tool disappears tomorrow, your tools keep working. + +## Where to go next + + + + Generate a working surface in about five minutes. + + + The tools exist. Now decide what agents may actually do, and keep it honest. + + + How to prompt an agent to extend the surface and write journeys. + + diff --git a/site/content/docs/working-with-your-agent.mdx b/site/content/docs/working-with-your-agent.mdx new file mode 100644 index 0000000..7943b63 --- /dev/null +++ b/site/content/docs/working-with-your-agent.mdx @@ -0,0 +1,131 @@ +--- +title: Working with your coding agent +description: The generator writes the parts an API can describe. Your own agent writes the parts that need product knowledge, including journeys. Here is how to ask it. +--- + +## The generator and your agent do different jobs + +The generator is good at what the API contract already states: the method, the +path, the types, the risk, the confirmation. It cannot know that "create a trip" +on your site means *search for the place, set the dates, then confirm*, or that +your product is a journal for trips people took rather than a planner for trips +they might take. + +That knowledge lives with you, and your coding agent can help you write it down, +if you give it the context the tool cannot. + +## The skill file + +Every `generate` run writes a skill file to your repo: + +```text +.agents/skills/webmcp-tools/SKILL.md +``` + +It is the rules this tool expects, written for an agent to read: the naming +rules, the character budgets, the description format, the exposure decisions, +the execute contract, and the journey pattern. It is regenerated on every run, +so it stays in step with the version you have. + +Agent tools discover it at that path: AGENTS.md, Claude Code, and the generic +"skills" standard all look there. If your project has its own skill directory, +skills stack, so your project rules sit on top of these. + +You do not have to do anything to activate it. A capable coding agent opens the +repo, and the rules are in front of it. Your job is to ask for the right thing. + +## What to give the agent + +Three things turn a vague request into a good tool or journey: + +1. **The product context.** What the site is, who uses it, and when in their +life the action happens. This is the part the generator cannot see, and it is +where wrong-but-plausible names come from. +2. **A pointer to the skill file**, so the agent follows the naming, budget, and +safety rules instead of inventing its own. +3. **The existing generated tools**, so the agent composes them instead of +re-implementing the API. + +## Prompts that work + +**Improve a tool's description.** Descriptions are the highest-leverage edit, +and a small prompt goes a long way: + +> Read `.agents/skills/webmcp-tools/SKILL.md`. Then look at +> `src/webmcp/list-trips.webmcp.ts`. The description reads mechanical. Rewrite +> it so an agent knows what it does, when to use it, and what it returns, within +> the skill's budget. Put the new text in `.webmcp-codegen.json`, not in the +> generated region. + +**Write a journey.** This is the one worth getting right. A good prompt names +the goal, the data the agent cannot invent, and the moment that needs a human: + +> Read `.agents/skills/webmcp-tools/SKILL.md` and the generated tools in +> `src/webmcp`. +> +> Write a journey called `document-trip` in `src/webmcp/journeys`. Goal: let a +> user record a trip they have already taken and open the editor to write its +> story. +> +> - The trip needs a resolved `locationObject`, which only the autocomplete +> endpoint can provide. Compose the generated `get-autocomplete` tool for +> that step and store the result on the draft. +> - The user sets a title and dates in a second step. +> - Creating the trip is a write, so it goes through the submit gate, not a +> step. Use the real `create-trip` execute as the submit's `run`. +> +> Run `npx @webmcp-stack/codegen verify` when you are done and fix what it +> reports. + +**Review the whole surface.** Useful before a release: + +> Read `.agents/skills/webmcp-tools/SKILL.md`. Review every tool in +> `src/webmcp`. For each one, tell me: is the name intent-shaped, does the +> description say what it returns, and should an agent be able to call it at +> all? Do not enable any withheld tool. List what you would change and why. + +## What to check when the agent is done + +An agent can be confidently wrong, especially about intent. Before you accept +its work: + +- **Read the diff.** Did it edit inside a generated region? If so, ask it to + move the change to `.webmcp-codegen.json` instead, or it will be overwritten. +- **Run `verify`.** It catches the mechanical mistakes: a description over + budget, a journey with no submit gate, a direct `fetch` that bypasses the + generated callers. +- **Check the safety, not just the syntax.** A step tool should be a read. If a + step calls something that writes, the write is not behind the submit gate, and + the human confirmation is not in the path. Ask the agent to move it into the + submit. +- **Check the name against your product.** `plan-trip` on a memories product + passes every mechanical check and is still wrong. This is the one no linter + catches, and it is why the skill file keeps asking the agent to understand the + product first. + + + A journey is only as safe as the code in it. The submit gate, the + confirmation, and the draft clearing live in the generator-owned + `journey.webmcp.ts` and cannot be edited away. A step's `call` and `run`, on + the other hand, are ordinary code. Keep writes in the submit. + + +## Do not let the agent decide exposure + +The agent can propose that a write be enabled. It should not do it on its own. +Enabling a write is a human decision about what a model may do on your site +while acting as your user. Keep that click. + +## Next + + + + The multi-step primitive, end to end. + + + The full loop, from reviewing the surface to watching it in CI. + + + How classification and withholding actually work. + + From 8ea1fbfb98c02a897e3312a8cef31ff3b2ec6d29 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:21:49 +0530 Subject: [PATCH 22/41] docs(site): close the frontmatter on devtools and troubleshooting Both pages were missing the closing `---`, so the title and description were parsed as body text and came back undefined in page metadata and in the generated llms.txt. --- site/content/docs/devtools.mdx | 1 + site/content/docs/troubleshooting.mdx | 1 + 2 files changed, 2 insertions(+) diff --git a/site/content/docs/devtools.mdx b/site/content/docs/devtools.mdx index 0f68797..964c36c 100644 --- a/site/content/docs/devtools.mdx +++ b/site/content/docs/devtools.mdx @@ -1,6 +1,7 @@ --- title: Chrome DevTools WebMCP panel description: Inspect, test, and debug your tools with Chrome's built-in WebMCP panel. +--- Chrome DevTools has a dedicated WebMCP panel that shows every tool your page exposes to AI agents. It's the fastest way to verify your tools work before an agent ever sees them. diff --git a/site/content/docs/troubleshooting.mdx b/site/content/docs/troubleshooting.mdx index a5109d0..b8c7e58 100644 --- a/site/content/docs/troubleshooting.mdx +++ b/site/content/docs/troubleshooting.mdx @@ -1,6 +1,7 @@ --- title: Troubleshooting description: Common errors and how to fix them. +--- ## "My tools 404 in DevTools" From d404fc227ee0c115c6b8cd63828d9fa6d647dcff Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:21:50 +0530 Subject: [PATCH 23/41] docs(site): make the docs agent-ready Publish /llms.txt as a curated index of every docs page, /llms-full.txt as the whole docs set in one markdown file for a single read, a robots.txt that allows crawlers and points at the sitemap, and sitemap.xml from the docs source. --- site/app/llms-full.txt/route.ts | 34 ++++++++++++++++++++++++++++ site/app/llms.txt/route.ts | 39 +++++++++++++++++++++++++++++++++ site/app/robots.ts | 14 ++++++++++++ site/app/sitemap.ts | 14 ++++++++++++ 4 files changed, 101 insertions(+) create mode 100644 site/app/llms-full.txt/route.ts create mode 100644 site/app/llms.txt/route.ts create mode 100644 site/app/robots.ts create mode 100644 site/app/sitemap.ts diff --git a/site/app/llms-full.txt/route.ts b/site/app/llms-full.txt/route.ts new file mode 100644 index 0000000..a992e38 --- /dev/null +++ b/site/app/llms-full.txt/route.ts @@ -0,0 +1,34 @@ +import { readFile } from "node:fs/promises"; +import { join } from "node:path"; + +/** Generated at build time, so it is a static file on the deployed site. */ +export const dynamic = "force-static"; + +/** + * Every docs page as raw markdown, in the order the navigation uses, for an + * agent that would rather read the whole thing in one request than crawl. + * Frontmatter is stripped; the page bodies already start with their headings. + */ +export async function GET() { + const dir = join(process.cwd(), "content/docs"); + const meta = JSON.parse(await readFile(join(dir, "meta.json"), "utf8")) as { + pages: string[]; + }; + + const parts: string[] = [ + "# webmcp-stack docs (full text)", + "", + "> The complete documentation for @webmcp-stack/codegen, concatenated for a single read.", + "", + ]; + + for (const slug of meta.pages) { + const raw = await readFile(join(dir, `${slug}.mdx`), "utf8"); + const body = raw.replace(/^---[\s\S]*?---\s*/, "").trim(); + parts.push(body, "", "---", ""); + } + + return new Response(parts.join("\n"), { + headers: { "content-type": "text/plain; charset=utf-8" }, + }); +} diff --git a/site/app/llms.txt/route.ts b/site/app/llms.txt/route.ts new file mode 100644 index 0000000..8ebf21a --- /dev/null +++ b/site/app/llms.txt/route.ts @@ -0,0 +1,39 @@ +import { source } from "@/lib/source"; + +/** Generated at build time, so it is a static file on the deployed site. */ +export const dynamic = "force-static"; + +const BASE = "https://webmcp-stack.vercel.app"; + +/** + * The llms.txt index: a short, curated map of the docs for an agent to read + * before fetching anything. Format follows the llms.txt convention. + */ +export function GET() { + const pages = source.getPages().sort((a, b) => a.url.localeCompare(b.url)); + const lines = [ + "# webmcp-stack", + "", + "> Generate safe, reviewable WebMCP tools from the API contract you already have, so AI agents can act on a site in the signed-in user's session.", + "", + "## Docs", + "", + ...pages.map((page) => + `- [${page.data.title}](${BASE}${page.url}): ${page.data.description ?? ""}`.trim(), + ), + "", + "## Full text", + "", + `- [llms-full.txt](${BASE}/llms-full.txt): every docs page concatenated for a single read.`, + "", + "## Optional", + "", + "- [GitHub](https://github.com/SouravInsights/webmcp-stack)", + "- [npm](https://www.npmjs.com/package/@webmcp-stack/codegen)", + "", + ]; + + return new Response(lines.join("\n"), { + headers: { "content-type": "text/plain; charset=utf-8" }, + }); +} diff --git a/site/app/robots.ts b/site/app/robots.ts new file mode 100644 index 0000000..41e8706 --- /dev/null +++ b/site/app/robots.ts @@ -0,0 +1,14 @@ +import type { MetadataRoute } from "next"; + +const BASE = "https://webmcp-stack.vercel.app"; + +/** + * Everything here is meant to be read, including by AI agents. The docs also + * publish /llms.txt and /llms-full.txt for exactly that. + */ +export default function robots(): MetadataRoute.Robots { + return { + rules: [{ userAgent: "*", allow: "/" }], + sitemap: `${BASE}/sitemap.xml`, + }; +} diff --git a/site/app/sitemap.ts b/site/app/sitemap.ts new file mode 100644 index 0000000..c13d643 --- /dev/null +++ b/site/app/sitemap.ts @@ -0,0 +1,14 @@ +import type { MetadataRoute } from "next"; +import { source } from "@/lib/source"; + +const BASE = "https://webmcp-stack.vercel.app"; + +export default function sitemap(): MetadataRoute.Sitemap { + const pages = source.getPages().map((page) => ({ + url: `${BASE}${page.url}`, + changeFrequency: "monthly" as const, + priority: 0.6, + })); + + return [{ url: BASE, changeFrequency: "monthly", priority: 1 }, ...pages]; +} From 7b13c7cb552956059224c211d98b7c918bec6b0a Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:22:07 +0530 Subject: [PATCH 24/41] chore: remove dead code in the eval runner and silence a false positive run.mjs imported execFile/promisify only to void them. group.test.ts asserts an emitted template literal, which biome's noTemplateCurlyInString reads as a mistake; silencing it beats rewriting the assertion to satisfy the linter. --- packages/codegen/evals/skill/run.mjs | 5 +---- packages/codegen/src/group.test.ts | 1 + 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/packages/codegen/evals/skill/run.mjs b/packages/codegen/evals/skill/run.mjs index 2a257ab..faf4745 100644 --- a/packages/codegen/evals/skill/run.mjs +++ b/packages/codegen/evals/skill/run.mjs @@ -21,12 +21,11 @@ * AGENT_CMD='claude -p "{PROMPT}" --output-format json' */ -import { execFile, spawn } from "node:child_process"; +import { spawn } from "node:child_process"; import { cp, mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { dirname, join, relative } from "node:path"; import { fileURLToPath } from "node:url"; -import { promisify } from "node:util"; const here = dirname(fileURLToPath(import.meta.url)); const FIXTURE = join(here, "fixture"); @@ -390,8 +389,6 @@ async function main() { process.exit(allPassed ? 0 : 1); } -const execFileAsync = promisify(execFile); -void execFileAsync; main().catch((error) => { console.error(error); process.exit(1); diff --git a/packages/codegen/src/group.test.ts b/packages/codegen/src/group.test.ts index 0650c22..3cdca5d 100644 --- a/packages/codegen/src/group.test.ts +++ b/packages/codegen/src/group.test.ts @@ -126,6 +126,7 @@ describe("composed tool templates", () => { expect(region).toContain("export async function fetchUploadMedia("); expect(region).toContain('const firstResult = (await callApi("/v1/media/request-upload"'); + // biome-ignore lint/suspicious/noTemplateCurlyInString: asserting the emitted template literal, not interpolating in this test. expect(region).toContain("${firstResult.uploadId}/complete"); expect(region).toContain("body: { fileName: input.fileName }"); // The merged tool is a write: the confirmation gate applies. From 89d18b7cc70068894a00cee314593f783b33254a Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:22:07 +0530 Subject: [PATCH 25/41] chore: add a changeset for the intent surface Covers the shipped journeys, grouping, budgets, skill file, and the removed LLM layer, plus the fixes in this branch. Publishing stays with the maintainer. --- .changeset/intent-surface.md | 40 ++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 .changeset/intent-surface.md diff --git a/.changeset/intent-surface.md b/.changeset/intent-surface.md new file mode 100644 index 0000000..1ecabdd --- /dev/null +++ b/.changeset/intent-surface.md @@ -0,0 +1,40 @@ +--- +"@webmcp-stack/codegen": minor +--- + +**The intent surface: journeys, handshake grouping, budgets, and an agent skill.** + +`generate` now scaffolds a skill file at `.agents/skills/webmcp-tools/SKILL.md` +so your own coding agent learns the naming rules, description budgets, the +execute contract, and the journey pattern. It also emits `title` and +`consequentialHint` from the current WebMCP spec, and accepts an `exposedTo` +config pass-through. + +Handshake endpoints that are one action split across two calls (a +request-upload plus a complete-upload) are detected and merged into one +withheld tool, thread-wiring the first response into the second by exact name. +Fuzzy pairs are skipped with a note. + +Journeys are the new multi-step primitive. `journey.webmcp.ts` is scaffolded +next to the runtime and regenerated every run; `journeys/*.webmcp.ts` files +import `createJourney` from it, the barrel registers them, and `verify` lints +them (submit gate present, step count, no direct `fetch`). `verify` also counts +journey tools in the surface total now. + +The built-in LLM layer is removed: `--llm`, `--suggest`, the provider flow, and +the config options. Rules reach models through the skill file; the CLI never +calls a model. + +**Budgets follow the tool's judgment better.** Chrome's 500/150 character +budgets are authoring guidance, not browser rules (the spec only rejects an +empty description or a name outside 1-128 chars). So generation no longer +silently shortens text a person or a spec wrote: machine-drafted text is +composed to fit, author text is kept in full, and `verify` reports an overrun +as a warning instead of a blocking error. The string trimmer also no longer +returns one character over budget on an unbroken token. + +**Fixes:** the generated `runtime.webmcp.ts` shipped with a string literal +broken across two lines, so it did not parse; it is valid again. Journey steps +no longer claim to be read-only when the tool they compose is a write, and +journeys resolve the WebMCP API at registration time so a late-installed +polyfill still registers them. From cda59789d47961258f1c21f2cb3b83f6e3ea6468 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 19:22:07 +0530 Subject: [PATCH 26/41] docs(reviews): record the post-review findings and the fixes made The review gained the runtime escaping bug found while typechecking the journey factory, the budget-is-guidance reasoning, and a short section on what was implemented. --- .../reviews/2026-09-11-pr-4-intent-surface.md | 297 ++++++++++++------ 1 file changed, 208 insertions(+), 89 deletions(-) diff --git a/docs/reviews/2026-09-11-pr-4-intent-surface.md b/docs/reviews/2026-09-11-pr-4-intent-surface.md index 11e67e2..03cbbb8 100644 --- a/docs/reviews/2026-09-11-pr-4-intent-surface.md +++ b/docs/reviews/2026-09-11-pr-4-intent-surface.md @@ -7,143 +7,262 @@ ## The short version The hard parts are built well. Grouping the upload handshake only when the rules -are exact is the right call, and putting the submit gate in a generator-owned -file means a journey can't edit the gate out. Tests pass, types pass, and the -spec sync is real. - -But before this merges I'd fix one generation bug and correct two safety -promises in the new docs. The bug can make the tool fail the project's own CI. -The promises are worse in a quiet way: they tell a reader the audit enforces -things it doesn't, and the whole pitch of this product is that the audit is -where the safety lives. - -## What I actually ran - -- Test suite: 220 pass. -- Typecheck: clean. -- Eval self-test (`run.mjs --selftest`): passes. -- Regenerated the eval fixture from its spec: byte-for-byte identical to what's - committed, so the eval baseline matches the current templates. -- Checked every spec claim against the spec fork at `97da8f5`. `title`, - `consequentialHint`, `exposedTo`, the `executeTool` signature, and the 1-128 - name rule are all exactly as described. `requestUserInteraction()` really is - absent, so the "watch item" framing is honest. +are exact is the right call. Putting the submit gate in a generator-owned file +means a journey can't edit the gate out. Tests pass, types pass, and the spec +sync is real. + +The one thing to settle before touching code: codegen treats Chrome's character +advice as if it were a law the browser enforces. It isn't. Once that's clear, the +right fixes get easier, and one of them (the trimmer) stops being a straight bug +fix and becomes a design decision. + +## First: are those character limits actually rules? + +This matters, so I checked the spec itself rather than trusting the summary in +the code comments. + +**What the browser actually rejects.** In `index.bs`, registering a tool fails +only for these reasons (lines 668-678): + +- a tool with that name is already registered +- the name or the description is an empty string +- the name is longer than 128 characters, or contains anything other than + letters, digits, `_`, `-`, or `.` +- the `inputSchema` can't be serialized to JSON + +**There is no description length rule in the spec.** Not 150, not 500. Nothing. +The spec's own roadmap text says the opposite: the 128-char name is the only +"nominal size restriction," and "further work is needed to evaluate the right +size limits for titles, names, and other inputs" (line 1816, issue #73). + +**Where 150/500/1.5K come from.** The spec README's Best Practices section points +at two Chrome guides (`best-practices` and `secure-tools`) for the detailed +numbers. Those guides are the source. They are authoring advice, and exceeding +them does not error. Nothing rejects a 600-character description. + +So we have three different things that were being treated as one: + +1. **Hard rules the browser enforces.** Name length and charset, non-empty name + and description. Break these and registration fails. Codegen must never break + these, and today it doesn't. +2. **Chrome's authoring guidance.** 30-char names, 150-char parameter text, + 500-char tool text, roughly 1.5K of output. These help the model pick and + call tools, and keep context small. Overrunning them is a quality smell, not + a failure. +3. **This project's own policy.** Generation truncates to those numbers and + `verify` marks overruns as errors. That is a choice codegen made. It's a + defensible choice, but it's a policy, not a law. + +## What should codegen do about it? + +The goal is a tool that writes good descriptions, not a tool that wins a +character-count contest. A few conclusions: + +**Never silently shorten text a human or a spec wrote.** That's the real problem +with the current behavior. The developer (or their spec) had a precise sentence; +the generator quietly chops it. The generated region can't be hand-edited, so the +only way back is an override file most people won't know about. If something has +to give, keep the full text and tell them it's long. Silently changing meaning is +the worst of the options. + +**When codegen writes the description itself, aim at the budget by construction.** +The generator controls its own phrasing, so it can compose text that naturally +fits, instead of composing something too long and then cutting it. This is where +being budget-aware actually belongs. + +**Report overruns, don't block on them.** A 160-character description is not a +broken tool. Make it a warning in `verify` that says the real length, so a +developer or their coding agent can decide whether to tighten it. Keep `error` +for the things that genuinely break: a spec-invalid name, an empty name or +description, an unserializable schema. + +**Make the budgets configurable.** Teams that want Chrome's exact numbers as a +CI gate can turn on strict mode. The default shouldn't fail a build over a +sentence that reads fine. + +**Watch how truncation and `verify` fight each other.** Today `verify` warns when +a description doesn't say what the tool returns. But the most common reason a +description overflows is that it has a second sentence saying what it returns, +and the trimmer cuts exactly that sentence off. So the generator can create the +problem it then complains about. Whatever is decided for budgets has to be +decided for both generation and `verify` at the same time, or they'll contradict +each other. + +**Keep the output cap as the exception.** The ~1.5K limit on tool results is +different: it protects the model's context at runtime, the helper already adds a +visible "truncated" notice, and cutting a giant JSON blob doesn't destroy +meaning the way cutting a sentence does. Truncation is the right tool there. ## The problems, in plain terms -### 1. The generator can produce a value that's one character too long +### 1. The description trimmer has a real bug, but the deeper issue is the trimming -A parameter description is supposed to be trimmed to 150 characters. If the -description is one long unbroken string (a URL, a token, an ID with no spaces), -the trimmer keeps all 150 characters and then adds an ellipsis, so the result is -151. I reproduced it. +Two separate things here. -Why that matters: `verify` later flags anything over 150 as an **error**, and the -project's CI gate fails on errors. So a developer runs `generate`, gets a file the -tool wrote, and then their own pipeline rejects it. And they can't just edit the -generated region to fix it, because that's the region that gets overwritten. The -tool should never produce text that its own checker refuses. +The bug: a parameter description is supposed to be cut to 150 characters. If the +text is one long unbroken string (a URL, a token, an ID), the trimmer keeps all +150 and then adds an ellipsis, landing at 151. I reproduced it. Since `verify` +treats 150 as an error, the generator can produce text its own checker rejects, +and the developer can't fix it in the generated region. If we keep any trimming, +that line needs to reserve room for the ellipsis. -The cause is one line in `packages/codegen/src/describe.ts` that appends the -ellipsis after slicing to the budget instead of inside it. +The deeper issue: per the section above, 150 is Chrome's advice, and cutting a +description at a character boundary is a blunt way to honor it. Before fixing +the off-by-one, decide whether silent truncation of author text is the behavior +we want at all. My suggestion is no: compose short, warn on long. But if the +team decides to keep truncating, fix the one line and add a test for the +no-space case. ### 2. Two safety promises the docs make, that the code doesn't keep -Both are in the new journey FAQ (and one is repeated in the direction doc): +Both are in the new journey FAQ (and repeated in the direction doc). **"Journey tools count toward the surface limit, and CI fails when the list -gets too big."** Neither half is true. The surface check only looks at the -generated endpoint tools. The journey step tools are built at runtime and never -reach that code, so they aren't counted at all. And going over the limit is a -warning, not an error, so it doesn't fail CI either. +gets too big."** Neither half is true. The surface check only looks at generated +endpoint tools; journey step tools are built at runtime and never reach it. And +going over the limit is a warning, so it doesn't fail CI. **"A tool that only makes sense inside a journey won't register on its own, so the journey shrinks your surface."** Nothing implements this. When `document-trip` gets its place data from `get-autocomplete`, that tool still -registers on its own, because reads register by default. The convention is real, -but following it is on the developer. The docs present it as enforced. +registers on its own, because reads register by default. It's a convention the +developer has to follow, presented in the docs as if the system enforces it. -Why it matters: a reader will trust that the audit has their back here and stop -paying attention. Then they ship forty tools and nothing in `verify` objects. +Why it matters: a reader will trust that the audit is watching this and stop +watching it themselves, then ship forty tools while `verify` stays quiet. -### 3. A journey step is labeled "read-only" even when it writes +The other AI's suggested resolution is reasonable: count the journey tools (steps +plus one submit per file, which you can already compute from the linted files) +and make over-limit a real error, then reword the absorption sentence to say it's +a convention. Auto-hiding a standalone read that a journey fully absorbs can wait +for later. -Every journey step is registered as read-only, no matter what it does. But a +### 3. A journey step is labeled "read-only" even when it might write + +Every journey step is registered as read-only, whatever it actually does. A step's `call` is just user code. It can call the real create-trip caller directly -instead of going through the submit gate. When it does: +instead of going through the submit gate. When that happens, the tool is +advertised to the agent as safe, the write goes through with no confirmation, +and the audit says nothing. + +The other AI pushed back on my first framing, and it was right: a regex can't be +the answer here, because the deepest bypass is editing the owned half of +`create-trip.webmcp.ts`, and no lint will catch that. That's a separate integrity +feature. -- The tool is advertised to the agent as safe to call, so the agent calls it. -- The write happens with no confirmation prompt. -- `verify` doesn't catch it, because its "no direct fetch" check looks for - `fetch(` and `callApi(`, not for `fetchCreateTrip(`. +But there are two different cases and it's worth keeping them apart: -So the human-in-the-loop guarantee only holds for a journey that routes every -write through `submit`. A sloppy journey (or a coding agent having a bad day) can -route around it, and the audit will stay quiet. For a tool whose reason to exist -is "the audit blocks the dangerous stuff," this is the finding I'd care most -about. +- **A step that calls a generated write caller** (`fetchCreateTrip` or + `executeCreateTrip`). No editing required, just a coding agent taking a + shortcut. This is reachable today, and a lint *can* flag it, because the caller + is generated and its tool is already classified as a write. Worth doing as a + tripwire, not a guarantee. +- **A hand-edited owned region.** Not catchable by any lint. Out of scope here. -There's a related assumption baked in: `docs/specs/journeys.md` even lists a -`navigate` side effect as a possible step, which plainly isn't a read. +The honest fix for the actual defect is to stop claiming the steps are read-only +when we can't prove it, and to write down that a journey is only as safe as the +person who wrote it. The lint tripwire is a bonus. ### 4. Journeys can silently fail to register Generated tools ask the browser for the WebMCP API at the moment they register. -Journeys ask once, when the file is first loaded, and remember the answer -forever. If the API appears in between those two moments, the tools register and -the journeys don't, with no error. The fix is small: ask at registration time, -the same way the tools do. +Journeys ask once, when the file first loads, and remember the answer. If the API +appears in between, the tools register and the journeys don't, with no error. +Fix is small: look it up at registration time, the same way the tools do. -### 5. Journey step descriptions can go over the 500-character limit +### 5. Journey step descriptions can go over the limit -The factory takes the generated tool's description, which is already trimmed to -500, and appends `Part of "document-trip": ...`. So the final text can be over -budget, and `verify` only measures the description written in the journey file, -not the text the factory builds at runtime. Again: the "budgets enforced" claim -holds for endpoint tools, but not for journey tools. +The factory takes a description already trimmed to 500 and appends +`Part of "document-trip": ...`, so the final text can be over. `verify` only +measures the text in the journey file, not what the factory builds at runtime. +This is the same policy question as #1: if budgets are guidance, this is a +warning at most; if they're hard, trim the base before appending so the composed +text fits. ## Smaller things, worth a look but not blockers - **The root `README.md` still sells the two flags this PR deletes.** It says `--suggest` and `--llm` exist. The package README and the docs site were - updated; the root one was missed. This is the most visible stale line. + updated; the root one was missed. - **`docs/specs/journeys.md` describes a design that was never built.** It describes declaring journeys in a config block. What shipped is TypeScript files. The spec is marked "not yet implemented", but the feature now exists in - a different shape, so the next person will read the wrong plan. -- **The old LLM layer is still all over `docs/specs/`.** Fine to defer, but a - one-line "removed in 0.9" note would stop someone rebuilding it. + a different shape, so the next person reads the wrong plan. +- **The old LLM layer is still all over `docs/specs/`.** A one-line "removed in + 0.9" note would stop someone rebuilding it. - **No changeset.** The repo's own rule is to add one for user-facing changes, and this removes two CLI flags, adds files, and changes what the package publishes. -- **Small cleanups:** dead code in `evals/skill/run.mjs` (imports and then voids - `execFile`), and one Biome warning from a test string in `group.test.ts`. +- **Small cleanups:** dead code in `evals/skill/run.mjs`, and one Biome warning + from a test string in `group.test.ts`. ## What's genuinely good here -Worth saying plainly, because most of this doc is complaints: - - **Grouping is disciplined.** It merges only when four exact rules line up, and skips anything fuzzy with a note. The tests pin the real beenthere pair, the - unthreadable pair, the cross-resource pair, and a GET negative. That's the - right instinct: a decision it doesn't have to make is a decision it can't get - wrong. + unthreadable pair, the cross-resource pair, and a GET negative. - **The gate sits in owned code.** For journeys that use the factory as intended, - there is genuinely no way around the confirmation. That's the right design. + there's genuinely no way around the confirmation. - **Budgets have one source of truth.** Generation and `verify` read the same - constants, so the measure matches the composer. + constants. That's the right structure even if the policy around it needs a + decision. - **The evals grade files, not vibes.** A self-test for the graders and a - "skill removed" control case for detecting absorption is more rigor than most - features get. + "skill removed" control case is more rigor than most features get. - **The spec sync is honest.** The watch items really aren't in the spec. ## If I were merging this -1. Fix the one-character overflow and add a test for the no-space case. -2. Decide on the step read-only labels and the write-bypass. Either enforce that - steps are reads, or stop calling them read-only, and teach `verify` to spot a - step that calls a known write tool. -3. Correct the two doc promises (surface count, tool absorption), or build them. - Don't ship a safety claim the audit can't back. -4. Ask the browser for the WebMCP API at registration time. -5. Update the root README and add a changeset. +1. Decide the budget policy first: guidance (warn) or hard limit (error). Then + make generation and `verify` agree. My vote is guidance, composed short, with + warnings instead of silent cuts. +2. If truncation stays, fix the off-by-one so it can never return 151. +3. Correct the two doc promises about surface counting and tool absorption. +4. Stop labeling journey steps read-only, and add the best-effort write-caller + tripwire. +5. Look up the WebMCP API at registration time. +6. Update the root README and add a changeset. + +## Post-review discovery: the generated runtime did not parse + +While typechecking the copied journey factory, I found something worse than +everything above. The 1.5K output cap this PR added writes a truncation notice: + +```ts +const TRUNCATED_NOTICE = + "\n… [truncated to fit the 1.5K output budget]"; +``` + +That `\n` sits inside the outer template literal that builds the runtime file, +so the emitted file contains a real newline inside a double-quoted string. +Every generated `runtime.webmcp.ts` is a syntax error, which means no generated +tool compiles at all. The committed eval fixture carried the same broken file, +and because the graders only match text and never compile, it passed every +check. This was invisible until something tried to parse the output. + +## What was implemented + +All of the above is now in the branch, plus the runtime fix: + +1. **Budget policy is guidance, not law.** `describe.ts` no longer silently + shortens author or spec text. Machine-drafted text is composed to fit; + author text is kept in full. `verify` reports overruns as warnings, for both + tools and journey files. Tests updated to lock the new behavior. +2. **The trimmer off-by-one is fixed** and covered by a test with an unbroken + token. +3. **Journey steps no longer claim to be read-only by default.** A tool-backed + step inherits the composed tool's `readOnlyHint`; a free step with no `run` + is read-only; anything else defaults to not-read-only, with an opt-in + `readOnly` override. +4. **The WebMCP API is resolved at registration time** in the journey factory, + matching generated tools. +5. **Journey step descriptions are composed within budget**, keeping the + journey-name tag and fitting the base when the goal would overflow. +6. **The surface check counts journey tools** (one per step plus the submit + gate) via `countJourneyTools`, so the docs' promise is now true. +7. **The generated runtime parses again.** `generated-code.test.ts` runs every + output through the TypeScript compiler to catch escaping mistakes. +8. **Docs corrected:** the FAQ's surface and absorption claims, the safety + bullet on the journeys page, the skill file's step guidance, the root README + (removed the deleted LLM flags), and a status note on `docs/specs/journeys.md`. +9. **A changeset was added.** The fixture was regenerated and the skill copies + were synced. `biome check` is clean, typecheck passes, and 229 tests pass. From dc1faadf73dc05bd1c9cc1091f25453953b16864 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 23:07:32 +0530 Subject: [PATCH 27/41] docs(journeys): rewrite around a relatable example and a clear mental model The page told a story about a specific product and left the reader to extract how journeys actually work. It now: - opens with the problem (a goal that takes several calls and needs a value the agent cannot invent), so the reader knows why the feature exists; - frames a journey as a form filled in steps, with the submit as the one confirmed Send button; - walks one complete example, booking a ride by resolving a destination, then setting a pickup time, then requesting the ride behind the gate; - shows the agent's side of the protocol, including the "still needed" replies; - explains how state moves through call, store, provides, and build. The FAQ and the two guides added earlier use the same example so the docs read as one set. --- site/content/docs/after-you-generate.mdx | 4 +- site/content/docs/journeys-faq.mdx | 228 ++++++------ site/content/docs/journeys.mdx | 333 ++++++++++-------- site/content/docs/working-with-your-agent.mdx | 25 +- 4 files changed, 313 insertions(+), 277 deletions(-) diff --git a/site/content/docs/after-you-generate.mdx b/site/content/docs/after-you-generate.mdx index 842f2ec..2e7062c 100644 --- a/site/content/docs/after-you-generate.mdx +++ b/site/content/docs/after-you-generate.mdx @@ -77,8 +77,8 @@ panel](/docs/devtools). ## 5. Write journeys for the goals that take several calls -Some things a user asks for cannot be one tool call. "Log the trip I took to -Lisbon" is search, then details, then a confirmed create. That shape is a +Some things a user asks for cannot be one tool call. Booking a ride is resolve +the destination, set the pickup time, then confirm. That shape is a [journey](/docs/journeys): a shared draft, one tool per step, and a submit gate that refuses until every step is done and asks the human before the real write. diff --git a/site/content/docs/journeys-faq.mdx b/site/content/docs/journeys-faq.mdx index d152fb8..ebc6b9d 100644 --- a/site/content/docs/journeys-faq.mdx +++ b/site/content/docs/journeys-faq.mdx @@ -1,172 +1,164 @@ --- title: Journey questions, answered -description: Every "dumb question" about journeys, answered plainly — the draft, the lifecycle, what the agent does vs. what you do, and the edge cases. +description: The draft, the lifecycle, what the agent does and what you do, and the edge cases, answered in plain terms. --- # Journey questions, answered The [journeys page](/docs/journeys) explains the feature. This page answers the -questions people actually ask once they read the code — in plain terms, no -assumed knowledge. None of these are dumb questions; they're the ones everyone -has. +questions people ask once they read the code. None of them are dumb questions; +they are the ones almost everyone has. ## Is `journey.webmcp.ts` a journey? -No — and this is the most common confusion. That file never creates a journey. -It's the factory: a reusable function, `createJourney()`, that journey files -call. Think Lego baseplate and Lego models. `journey.webmcp.ts` is the -baseplate — shared, generic, owned by the generator. `document-trip.webmcp.ts` -is a model someone builds on it. The engine lives in one file; the journeys -live elsewhere and are written per product. - -## What are the three parts, in dumb terms? - -1. **The draft** — `let draft = {}`. A plain JavaScript object sitting in - memory, created once when the page loads. Every step writes something into - it; the final step reads everything out of it. -2. **Step tools** — each step becomes its own normal, callable WebMCP tool - (`document-trip-search-places`). When the agent calls one, it runs the - step's logic, stuffs the result into the draft, and replies "Stored. Still - needed: X" or "Stored. Ready — call submit." That reply is literally how the - agent knows what to do next. There's no hidden orchestration — the agent - figures out the order by reading those messages, one call at a time. -3. **The submit gate** — one final tool (`document-trip-submit`) that does four - things in strict order: check nothing's missing (refuse if it is) → ask the - human to confirm → only on a yes, call the real backend with the draft's - data → clear the draft. +No, and this is the most common confusion. That file never creates a journey. It +is the factory: a reusable `createJourney()` function that journey files call. +Think of a Lego baseplate and the models built on it. `journey.webmcp.ts` is the +baseplate, shared and owned by the generator. `book-ride.webmcp.ts` is a model +someone builds on it. The engine lives in one file; the journeys live elsewhere +and are written per product. + +## What are the three parts, in plain terms? + +1. **The draft** is one plain object in memory, created when the page loads. + Steps write into it; the submit reads everything out of it. +2. **Step tools** are ordinary WebMCP tools, one per step + (`book-ride-resolve-destination`). When the agent calls one, it does the + step's work, stores the result on the draft, and replies "Stored. Still + needed: X" or "Stored. The journey is ready." That reply is how the agent + knows what to do next. There is no hidden orchestration; the agent works one + call at a time and reads the messages. +3. **The submit gate** is one final tool (`book-ride-submit`) that does four + things in order: check nothing is missing and refuse if it is, ask the human + to confirm, run the real write with the finished draft, and clear the draft. ## Why is it called a "draft"? -Same reason as a draft email. It's the version-in-progress — built up piece by -piece, and nothing is sent until the very end. It's called a draft specifically -because it is *not yet the real thing*: nothing reaches the app's backend from -it until the submit step. +Same reason as a draft email. It is the version in progress, built up piece by +piece, and nothing is sent until the end. It is a draft because it is *not* the +real thing: nothing reaches your backend from it until the submit runs. ## What does "storing something in the draft" actually mean? -Literally adding a labeled value to that one object — `title: "Lisbon Trip"`, -`locationObject: {...}`. Like writing another line on a sticky note. No -database, no file, no network call is involved in the storing itself. +Adding a labeled value to that one object, like `destination: {...}` or +`pickupAt: "2026-03-14T19:30:00Z"`. It is like writing another line on a sticky +note. There is no database, no file, and no network call in the storing itself. -## What does "in-memory" mean if I'm not technical? +## What does "in memory" mean if I am not technical? Picture a whiteboard in a room versus a filing cabinet down the hall. The -whiteboard is right there — fast, everyone in the room can read it instantly — -but the moment someone wipes it or the room closes, it's gone forever, no copy -anywhere. The filing cabinet keeps things after everyone leaves. The draft is -the whiteboard: it lives only inside the browser tab's running memory. Close -the tab, refresh, or navigate away, and it's wiped. Nothing durable, ever. +whiteboard is right there, fast, and everyone in the room can read it, but the +moment the room closes it is gone. The filing cabinet keeps things after +everyone leaves. The draft is the whiteboard: it lives only inside the browser +tab's running memory. Close the tab, refresh, or navigate away and it is wiped. +Nothing durable, ever. -## What's the shape of this store? +## What is the shape of the draft? -The simplest possible: a flat bag of labeled values — -`{ locationObject: {...}, title: "Lisbon Trip" }`. No nesting enforced, any -step can write any field. One caveat: if two steps use the same field name, -the second silently overwrites the first. Avoiding that is on whoever writes -the journey. +The simplest possible: a flat bag of labeled values, for example +`{ destination: {...}, pickupAt: "2026-03-14T19:30:00Z" }`. Nothing enforces +nesting, and any step can write any field. One caveat: if two steps use the same +field name, the second silently overwrites the first. Avoiding that is on +whoever writes the journey. ## Will it hold data reliably for a session? -Depends what you mean by session: +It depends what you mean by session: -- **Within one continuous page view** (tab open, agent working, no refresh) — - yes, completely. It's a variable in memory; nothing can drop it mid-flow. -- **Across a reload, a closed tab, or a new tab** — no, and that's deliberate. - It is never meant to outlive one uninterrupted visit. +- **Within one continuous page view** (tab open, agent working, no refresh): yes, + completely. It is a variable in memory and nothing drops it mid-flow. +- **Across a reload, a closed tab, or a new tab:** no, deliberately. It is never + meant to outlive one uninterrupted visit. ## How does it store results from the real backend? -Subtle but important: **not every step touches the backend.** A step like -`set-details` has no logic — it stores whatever the agent typed, verbatim, no -network call at all. A step like `search-places` calls the app's real search -API and stores the real answer. So the draft ends up holding a mix: some -fields are "what the agent said," some are "what the server said." Either way, -nothing is created or changed for real until the final submit. +Not every step touches the backend. A step like `set-pickup-time` has no logic; +it stores whatever the agent passed in, verbatim, with no network call. A step +like `resolve-destination` calls the app's real geocoder and stores the real +answer. So the draft ends up holding a mix: some fields are "what the agent +said", some are "what the server said". Either way, nothing is created or +changed for real until the submit. ## Why is the draft created only once per page load? -Because `createJourney()` — the function that sets up `let draft = {}` — runs -once, at page load, not per attempt. So there's one shared scratchpad per -journey per page visit. Fine for the realistic case (one agent, one user, one -thing at a time). The unhandled edge case: two parallel attempts at the *same* -journey on one page would share and overwrite the same draft. Accepted, not -handled. +Because `createJourney()`, the function that sets up the draft, runs once at +page load, not per attempt. There is one shared scratchpad per journey per page +visit. That is fine for the realistic case: one agent, one user, one thing at a +time. The unhandled edge case is two parallel attempts at the *same* journey on +one page, which would share and overwrite the same draft. Accepted, not handled. -## What is a "half-finished journey," and why would it happen? +## What is a "half-finished journey", and why would it happen? -The agent completes `search-places`, then the person closes the laptop, gets -distracted, the tab crashes — before the remaining steps and submit. Some steps -done, nobody submitted: half-finished. Keeping the draft memory-only means a -reload always starts completely clean — no stale, half-filled attempt from -three days ago reappearing with an outdated search result. Throwing away -incomplete progress is the feature, not a gap. +The agent resolves the destination, then the person closes the laptop or the tab +crashes, before the rest of the steps and the submit. Some steps done, nothing +sent. Keeping the draft in memory means a reload always starts clean, with no +stale, half-filled attempt from three days ago reappearing with an outdated +result. Throwing away incomplete progress is the feature, not a gap. ## Does a journey reuse the tools codegen already generated? -Yes, directly — it never reimplements anything. A tool-backed step calls the -same underlying request the standalone tool uses; the submit's `run` is the -existing write tool's `execute`. The journey just wraps them with the gate and -confirmation. Bonus: a risky write like `create-trip` — normally generated -withheld — never needs to be switched on as a standalone tool at all. The -journey's submit becomes its only door, with prerequisite checks and a human -confirmation built in. Wrapping a risky tool in a journey is a *safer* way to +Yes, directly, and it never reimplements anything. A tool-backed step calls the +same underlying request the standalone tool uses, and the submit's `run` is the +existing write tool's execute. The journey wraps them with the gate and the +confirmation. Bonus: a risky write like `request-ride`, normally generated +withheld, never has to be enabled as a standalone tool at all. The journey's +submit becomes its only door, with the prerequisite checks and a human +confirmation built in. Wrapping a risky tool in a journey is a safer way to expose it than enabling it directly. ## How does the agent know these tools exist? -No special channel — ordinary WebMCP discovery. On page load, every registered -tool (normal ones, journey steps, journey submits) lands in one flat list the -browser exposes. The agent asks the page "what can I do here," gets the whole -list, with zero formal distinction between "normal tool" and "part of a -journey." The only grouping signal is baked into the text: each step's name -carries the journey prefix (`document-trip-…`), and each description ends with -*"Part of document-trip: Record a trip you've been on…"* The agent reads that -and infers the connection — the same way it figures out anything else: by -reading. The stitching is done by the factory, mechanically, so it can't be -forgotten. - -## What does the agent do vs. what do I do? - -The agent's job is orchestration and conversation: figure out what to call -next from the reply messages, ask you for anything it can't get from the -backend. Your job is exactly one thing: the final yes/no on the confirmation -dialog before anything real happens. The agent cannot click that dialog for -you — a human click is the only way through. +There is no special channel: ordinary WebMCP discovery. On page load, every +registered tool, including journey steps and submits, lands in one flat list the +browser exposes. The agent asks the page what it can do and gets the whole list, +with no formal distinction between a normal tool and part of a journey. The only +grouping signal is in the text: each step's name carries the journey prefix +(`book-ride-…`), and each description ends with something like *"Part of +book-ride: Book a ride to a destination the user gives."* The agent reads that +and infers the connection, the same way it figures out anything else. The +factory stitches it mechanically, so it cannot be forgotten. + +## What does the agent do, and what do I do? + +The agent's job is orchestration and conversation: work out what to call next +from the replies, and ask the user for anything it cannot get from the backend. +Your job is exactly one thing: the final yes or no on the confirmation dialog +before anything real happens. The agent cannot click that dialog for you. A +human click is the only way through. ## Can the agent call the steps out of order? -The order isn't enforced by code — an agent could call `set-details` before -`search-places`. It mostly self-corrects: a step that needs an earlier step's -draft field fails with a readable error ("Not ready…" or a failed call naming -what it needed), which nudges the agent back. One honest caveat: that -self-correction depends on the step failing *usefully* when called early — -which is on whoever writes the journey, not something the shared machinery -guarantees. +The order is not enforced by code, so an agent could call `set-pickup-time` +before `resolve-destination`. It mostly self-corrects: a step that needs an +earlier step's draft field fails with a readable message naming what is missing, +which nudges the agent back. One honest caveat: that self-correction depends on +the step failing *usefully* when called early, which is on whoever writes the +journey, not something the shared machinery guarantees. ## What happens if I decline the confirmation? -The agent is told "The user declined this action," and the draft is left -as-is — not wiped. You can change your mind, adjust something, and submit +The agent is told "The user declined this action", and the draft is left +untouched, not wiped. You can change your mind, adjust something, and submit later without starting over. ## Why can't codegen just generate the journeys for me? -Because the flow isn't in your API spec. An OpenAPI file says `POST /v1/trips` -exists; it does not say that creating a trip is really search → set details → -confirm, or that your product documents past trips rather than planning future -ones. That knowledge lives in the product. A CLI guessing flows produces -plausible-sounding garbage — tools that pass every mechanical check and are -still wrong. So codegen stops at a hint in its report ("these endpoints look -like one flow — declare a journey?") and the bundled skill file teaches your -own coding agent how to write the journey file well. Your agent can ask you -about the product; our CLI can't. +Because the flow is not in your API spec. An OpenAPI file says `POST /v1/rides` +exists; it does not say that booking a ride is really resolve the destination, +then set the pickup time, then confirm. It also cannot know that your product +wants a person to approve the ride rather than let an agent request it +directly. That knowledge lives in the product. A CLI guessing flows produces +plausible-sounding garbage, tools that pass every mechanical check and are still +wrong. So codegen stops at a hint in its report and the bundled skill file +teaches your own coding agent how to write the journey file well. Your agent can +ask you about the product; the CLI cannot. ## Won't journeys make my tool list even bigger? -They can, if you let them. `verify` now counts journey tools in the surface -total (one per step plus the submit gate) and warns when the whole surface grows -past what's good for agents. It's a warning, not a failed build: the browser has -no limit on how many tools you register, only agent quality does. +They can, if you let them. `verify` counts journey tools in the surface total, +one per step plus the submit gate, and warns when the whole surface grows past +what is good for agents. It is a warning, not a failed build: the browser has no +limit on how many tools you register, only agent quality does. Keeping the surface small is a convention you follow, not something the generator does for you. A read tool that a journey uses still registers on its @@ -178,5 +170,5 @@ them, so leave them withheld and the submit is their only door. ## What if two steps use the same draft field name? The second overwrites the first, silently. The draft is a flat bag with no -protection — give each step's fields distinct names when you write the -journey. (Same answer as the shape question; it bites twice.) +protection, so give each step's fields distinct names when you write the +journey. diff --git a/site/content/docs/journeys.mdx b/site/content/docs/journeys.mdx index 0ef4391..dcc4fd3 100644 --- a/site/content/docs/journeys.mdx +++ b/site/content/docs/journeys.mdx @@ -1,167 +1,212 @@ --- title: Journeys -description: Some goals take several tool calls with shared state and one confirmed write at the end. A journey is that shape — here's exactly what ships, what you write, and what a real agent session looks like. +description: A few steps that share state, ending in one write a person confirms. Journeys are how you describe a goal that takes more than one tool call. --- -# Journeys - -Some things a user asks an agent to do can't be one tool call. "Log the trip I -took to Lisbon" means: find the real place, set the title and dates, then create -the trip — and only a human should be able to say yes to that last part. A -journey is that shape: **a few steps that share state, and one guarded submit at -the end.** - -A journey is made of three pieces: - -1. **The draft** — one plain object in the page's memory. Think whiteboard, not - filing cabinet: steps write their results on it, the submit reads from it, - and it's wiped clean when the page reloads. No database, no files, nothing - saved anywhere. -2. **Step tools** — each step becomes its own ordinary WebMCP tool. Calling one - stores what it produced into the draft and replies with what's still missing, - so the agent always knows the next move. -3. **The submit gate** — one final tool that refuses until every step has - delivered, asks the human to confirm, and only then makes the real write — - using only what's in the draft. Afterward the draft resets to empty. - -## A real session, start to finish - -beenthere.page is a journal for trips you've been on. Its journey for the core -flow is called `document-trip`, with two steps and a submit. Here's the complete -session — a signed-in user on their profile page, an agent in the browser, and -one sentence from the user: *"Log the trip I took to Lisbon last March."* - -**Page load.** The page registers its tools. Alongside the ordinary ones, three -journey tools appear: `document-trip-search-places`, `document-trip-set-details`, -and `document-trip-submit`. There's no special channel for journeys — the agent -sees one flat list and learns these belong together from their names and -descriptions, which the machinery stamps mechanically: *"Search real places and -store the pick. Part of document-trip: Record a trip you've been on…"* - -**Step 1.** The agent calls `document-trip-search-places` with -`{ input: "Lisbon" }`. That hits the real backend search, and the resolved place -— a structured object with a `placeId`, address, country — lands in the draft. -This matters more than it looks: the create-trip endpoint needs that object, and -an agent can't invent it. Now it comes from the server, not from a guess. The -tool replies: *"Stored. Still needed: document-trip-set-details."* - -**Step 2.** The agent calls `document-trip-set-details` with -`{ title: "Lisbon, March 2026", startDate: "2026-03-14" }`. No backend call — -the values are stored on the draft as-is. The tool replies: *"Stored. The -journey is ready — call document-trip-submit."* - -**The submit.** The agent calls `document-trip-submit`. The gate checks the -draft: `locationObject` present, `title` present — nothing missing. Then the -page shows a real confirmation dialog: *"Allow the agent to: Create the trip and -open it in the editor."* Only a human click can pass this — the agent cannot -approve it for you. - -**Approved.** The real `create-trip` call fires, with input assembled from the -draft and nothing else. The draft resets to empty. The app does what it does -when you click the button yourself: the trip is created and the editor opens on -it. The agent turns back to the user: "Created and opened — want me to draft the -story from your photos?" - -If the human declines, nothing is wiped: the draft stays, and the agent is told -the user declined. And if the tab closes mid-journey, the draft simply -evaporates — a half-finished journey never lingers. - -The division of labor, the whole session: the agent orchestrates and talks to -the user; the human does exactly one thing — approve or deny at the end. - -## What codegen ships vs. what you write - -Codegen ships the machinery, never the journeys: - -- **`journey.webmcp.ts`** — the `createJourney` helper, copied into your repo on - every `generate`, next to `runtime.webmcp.ts`. It's generator-owned: don't - hand-edit it; regeneration overwrites it. The draft, the step registration, - the gate, the confirmation — all of it lives here, once, for every journey. -- **Registration** — the generated `index.ts` auto-imports everything in the - `journeys/` folder inside your tools directory (e.g. - `src/webmcp/journeys/`) and registers it. Drop in a journey file, re-run - `generate`, it's live. -- **Checks** — `verify` validates journey files: submit gate present, steps - described within budget, step count small enough for an agent to track, and no - direct `fetch` bypassing your generated tools. - -What codegen never writes is the journey itself — an OpenAPI spec can't reveal -that "create a trip" is really search → details → confirm. Only the product side -knows the flow. So you write one small file per journey (or your coding agent -does, guided by the bundled skill file): +## When one tool is not enough + +Most tools do one thing and forget it: `list-trips`, `cancel-order`, +`search-products`. Some goals do not fit that shape. Booking a ride is not one +call, it is: + +1. turn the destination the user said into the coordinates the API needs, +2. ask when they want to be picked up, +3. actually request the ride, after the person agrees to it. + +Three things make this a different kind of problem: + +- **The steps depend on each other.** The ride request needs what the first two + steps produced. +- **The agent cannot invent a step's output.** It knows "SFO airport" as words. + The ride API needs coordinates, and those have to come from a real geocoder. + Ask a model to make them up and your server rejects the request. +- **The last step changes something real.** Requesting a ride should not happen + because an agent decided it should. A person should say yes. + +A **journey** is how you describe that goal once: a shared place to collect the +answers, one tool per step, and a submit that a human confirms. + +## The mental model: a form, filled in steps + +The easiest way to hold journeys in your head is a form. + +- **The draft is the form.** One object in the page's memory, for the life of + the page. Steps write on it; the submit reads from it. It is not a database + and nothing is saved to disk. +- **Each step fills in part of the form.** A step either calls one of your + existing tools and stores what it returns (the destination's coordinates), or + just takes the agent's input and stores it (the pickup time). Every step + replies with what is still blank, so the agent always knows the next move. +- **The submit is the Send button.** It stays disabled until every field is + filled, it shows the person what is about to happen, and only then does it run + the real tool. + +From the agent's side there is no hidden machinery. It sees a small flat list of +tools and fills them in one at a time, reading the "still needed" reply after +each call. Journeys do not coordinate the agent. They give it a form. + +## The whole thing in one example + +A rideshare app has two generated tools: `geocode-address` (a read) and +`request-ride` (a write, so it is withheld until enabled). The journey that ties +them into one goal is a single file: ```ts -// src/webmcp/journeys/document-trip.webmcp.ts +// src/webmcp/journeys/book-ride.webmcp.ts import { createJourney } from "../journey.webmcp"; -import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; -import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; +import { geocodeAddressTool, fetchGeocodeAddress } from "../geocode-address.webmcp"; +import { executeRequestRide, type RequestRideInput } from "../request-ride.webmcp"; + +export const bookRide = createJourney({ + name: "book-ride", + goal: "Book a ride to a destination the user gives", -export const documentTrip = createJourney({ - name: "document-trip", - goal: "Record a trip you've been on and open the editor to write its story", steps: { - // A tool-backed step: inherits the generated tool's description and - // schema, calls its raw caller. You write only what lands in the draft. - "search-places": { - tool: getAutocompleteTool, - call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), - store: (places) => ({ locationObject: places }), // the resolved pick - provides: ["locationObject"], + // A tool-backed step. The agent gives an address as text; the real + // geocoder turns it into coordinates, and the step stores the result. + "resolve-destination": { + tool: geocodeAddressTool, + call: (input, signal) => fetchGeocodeAddress(input as never, signal), + store: (place) => ({ destination: place }), + provides: ["destination"], }, - // A freeform step: no backend call — collects input into the draft. - "set-details": { - description: "Set the trip's title and dates.", + + // A free step. No API call: the pickup time the agent passed in is + // stored on the draft as-is. + "set-pickup-time": { + description: "Set when the user wants to be picked up.", input: { type: "object", - properties: { title: { type: "string" }, startDate: { type: "string" } }, - required: ["title"], + properties: { + pickupAt: { type: "string", description: "The pickup time, in ISO 8601." }, + }, + required: ["pickupAt"], }, - provides: ["title"], + provides: ["pickupAt"], }, }, + submit: { - description: "Create the trip and open it in the editor.", - build: (draft) => draft as CreateTripInput, // assemble the real tool's input - run: executeCreateTrip, // the existing tool does the work + description: "Request the ride and show the driver's details.", + build: (draft) => draft as RequestRideInput, // the draft becomes the tool's input + run: executeRequestRide, // the real write runs here, after the human confirms }, }); ``` -## Rules worth knowing +That is the whole feature. There is no config block and no new language: a +journey is TypeScript that imports the tools you already generated. -(Every "wait, but what if…" these rules raise is answered plainly in -[Journey questions, answered](/docs/journeys-faq).) - -- **Journeys reuse your generated tools; they don't replace them.** A - tool-backed step inherits the tool's contract and calls the same underlying - request — nothing is reimplemented. Edit the tool, regenerate, and the journey - follows. -- **A tool that only makes sense inside a flow shouldn't be registered on its - own.** `get-trip-stamp-eligibility` is meaningless outside the stamp flow, so - it lives only as a journey step. And a risky write — like `create-trip` — - stays withheld as a standalone tool forever; the journey's submit gate becomes - its only public door. Journeys should shrink the surface an agent sees, not - grow it. -- **The submit gate can't be edited away, but a step is still your code.** The - gate, the confirmation, and the draft clearing live in the shared helper, so a - journey file can't remove them. A step's `call` and `run`, though, are code you - or your agent write. Keep steps to reads and draft writes; put anything that - changes real data in the submit, where the human is asked first. `verify` - only marks a step read-only when the tool it composes is a read, but it can't - inspect arbitrary code, so the discipline is still yours. -- **The draft is per page load, per journey.** Within one continuous page view - it's fully reliable. Across a reload, it's gone — deliberately. Two parallel - attempts at the same journey on one page would share one draft; in practice - agents work one thing at a time, so this is accepted, not handled. -- **Keep journeys short.** Two to five steps. If you're past five, you're - describing two journeys. +## What the agent sees -## Grouping is a different thing +On page load, `createJourney` registers three ordinary tools. They look like any +other tool; nothing marks them as special to the agent. -Don't confuse journeys with what the generator's grouping does. Grouping -*merges*: endpoints that are one action split by API shape (a request-upload -call and a complete-upload call) become a single coarse tool — it happens at -generate time, straight from the spec. Journeys *chain*: different decisions -with shared state and a guarded submit, written per-product after generation. -Same action split across calls → that's grouping. Different decisions along the -way → that's a journey. +```text +book-ride-resolve-destination Fills the destination. Part of "book-ride": Book a ride... +book-ride-set-pickup-time Set when the user wants to be picked up. Part of "book-ride"... +book-ride-submit Request the ride and show the driver's details. +``` + +The agent fills the form by calling them in order, reading each reply: + +```text +agent book-ride-resolve-destination { "address": "SFO airport" } +tool Stored. Still needed: book-ride-set-pickup-time, book-ride-submit. + +agent book-ride-set-pickup-time { "pickupAt": "2026-03-14T19:30:00Z" } +tool Stored. The journey is ready — call book-ride-submit. + +agent book-ride-submit + (the page asks the human: "Allow the agent to: Request the ride and show the driver's details.") +human approves +tool ride requested +``` + +Two details are doing a lot of work here. The "still needed" replies are how the +agent learns the order without being told it in prose. And the submit cannot run +until the draft is full, so an agent that skips a step is refused with the list +of what is missing, not a failed request. + +## How state moves between steps + +Three fields move the data: + +- **`call`** does the work. For a tool-backed step it calls the raw caller your + generated file exports (`fetchGeocodeAddress`), not the agent-facing wrapper. + You reuse one request, and the schema stays single-sourced. +- **`store`** decides what lands on the draft. It receives the call's result and + returns the fields to save. This is where a resolved object becomes a named + form field. +- **`provides`** names the draft fields the step is responsible for. The submit + refuses until every promise in every step is on the draft. + +The submit's **`build`** is the reverse: it takes the finished draft and +assembles the real tool's input. Because the only path to the real input is +`build(draft)`, an agent cannot call the write with a half-filled form. + +## Rules worth knowing + +- **Keep journeys short.** Two to five steps. Past five, you are usually + describing two journeys. +- **Steps read, submit writes.** A step should call a read tool or fill the + draft. Put anything that changes real data in the submit, where the human is + asked first. `verify` marks a step read-only only when the tool it composes is + a read, and it cannot inspect code you write by hand. +- **Reuse, do not re-implement.** A tool-backed step inherits the generated + tool's description and schema and calls its raw caller. Edit the tool, + regenerate, and the journey follows. Never write a direct `fetch` in a journey + file; `verify` flags it. +- **Do not double-register.** A tool that only makes sense inside a flow should + not also register standalone. Withhold it in `.webmcp-codegen.json`, or exclude + it, so the journey is its only door. Journeys should shrink the surface, not + grow it. +- **The draft is per page load, per journey.** It is reliable within one page + view and gone on reload, on purpose. Two parallel attempts at the same journey + on one page share a draft; in practice agents work one thing at a time. + +The gate itself cannot be edited away. `journey.webmcp.ts` owns the confirmation +and the draft clearing, and it is regenerated on every `generate`. The step +logic is your code, which is why the rules above exist. + +## What ships, and what you write + +`generate` writes two things into your tools directory: + +- **`journey.webmcp.ts`**, the factory that defines the draft, registers the + steps, and holds the submit gate. Generator-owned, regenerated every run. +- **`index.ts`** imports every file in `journeys/` and registers whatever + `createJourney` returns. Drop a journey file in, re-run `generate`, and it is + live. + +What it never writes is the journey itself, because an OpenAPI spec cannot tell +it that "book a ride" is really *resolve, then schedule, then confirm*. That is +product knowledge. You write it, or your coding agent does, guided by the +[skill file](/docs/working-with-your-agent). + +## Journeys are not grouping + +Two features sound similar and are not: + +- **Grouping** merges endpoints that are one action split across two calls (a + request-upload plus a complete-upload) into a single tool. It happens at + generate time, straight from the spec, with no human input. +- **Journeys** chain different decisions that share state and end in one guarded + submit. They are written per product, after generation. + +Same action split across calls: group it. Several decisions along the way: a +journey. + +## Next + + + + How to prompt an agent to write a journey for your product. + + + The draft, the lifecycle, and the edge cases, in plain terms. + + + Where journeys fit in the wider loop of shipping a safe surface. + + diff --git a/site/content/docs/working-with-your-agent.mdx b/site/content/docs/working-with-your-agent.mdx index 7943b63..1fcaa8b 100644 --- a/site/content/docs/working-with-your-agent.mdx +++ b/site/content/docs/working-with-your-agent.mdx @@ -63,16 +63,15 @@ the goal, the data the agent cannot invent, and the moment that needs a human: > Read `.agents/skills/webmcp-tools/SKILL.md` and the generated tools in > `src/webmcp`. > -> Write a journey called `document-trip` in `src/webmcp/journeys`. Goal: let a -> user record a trip they have already taken and open the editor to write its -> story. +> Write a journey called `book-ride` in `src/webmcp/journeys`. Goal: let a user +> book a ride to a destination they give. > -> - The trip needs a resolved `locationObject`, which only the autocomplete -> endpoint can provide. Compose the generated `get-autocomplete` tool for -> that step and store the result on the draft. -> - The user sets a title and dates in a second step. -> - Creating the trip is a write, so it goes through the submit gate, not a -> step. Use the real `create-trip` execute as the submit's `run`. +> - The ride API needs coordinates, which only the geocode endpoint can +> provide. Compose the generated `geocode-address` tool for that step and +> store the result on the draft as `destination`. +> - The user sets a pickup time in a second step. +> - Requesting the ride is a write, so it goes through the submit gate, not a +> step. Use the real `request-ride` execute as the submit's `run`. > > Run `npx @webmcp-stack/codegen verify` when you are done and fix what it > reports. @@ -98,10 +97,10 @@ its work: step calls something that writes, the write is not behind the submit gate, and the human confirmation is not in the path. Ask the agent to move it into the submit. -- **Check the name against your product.** `plan-trip` on a memories product - passes every mechanical check and is still wrong. This is the one no linter - catches, and it is why the skill file keeps asking the agent to understand the - product first. +- **Check the name against your product.** A name can pass every mechanical + check and still be wrong for what the flow actually does. This is the one no + linter catches, and it is why the skill file keeps asking the agent to + understand the product first. A journey is only as safe as the code in it. The submit gate, the From 19105cc29f2f86b8213f0e632b4a99846657dc6a Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 23:59:09 +0530 Subject: [PATCH 28/41] docs(site): register the standard Fumadocs components Steps for procedures, Tabs for alternatives, Accordions for optional detail, and Files for directory trees, registered once so docs pages use them without an import. --- site/components/mdx.tsx | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/site/components/mdx.tsx b/site/components/mdx.tsx index 0b47c13..0fd7716 100644 --- a/site/components/mdx.tsx +++ b/site/components/mdx.tsx @@ -1,9 +1,25 @@ +import { Accordion, Accordions } from "fumadocs-ui/components/accordion"; +import { File, Files, Folder } from "fumadocs-ui/components/files"; +import { Step, Steps } from "fumadocs-ui/components/steps"; +import { Tab, Tabs } from "fumadocs-ui/components/tabs"; import defaultMdxComponents from "fumadocs-ui/mdx"; import type { MDXComponents } from "mdx/types"; export function getMDXComponents(components?: MDXComponents) { return { ...defaultMdxComponents, + // Standard Fumadocs pieces, registered once so docs pages can use them + // without an import: Steps for procedures, Tabs for alternatives, + // Accordions for optional detail, Files for a directory tree. + Accordion, + Accordions, + File, + Files, + Folder, + Step, + Steps, + Tab, + Tabs, ...components, } satisfies MDXComponents; } From 6c45c8b5e719cdfff09a1d7c53709c5a2d38c87a Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 23:59:10 +0530 Subject: [PATCH 29/41] docs: add the prompt cookbook, shape guide, and testing guide Three practical gaps: - Prompt cookbook: copy-paste prompts for the recurring jobs (improve a description, write a journey, review the surface, fix verify, do a safety pass), each with what to check afterwards. - "Tool, group, or journey?": the decision guide for which shape a capability takes, with signs for and against, and the edge cases in accordions. - Testing your tools: how to test a generated tool without a browser, what throws and why, how to test a journey's store and build, and the one thing that still needs a browser. --- site/content/docs/meta.json | 5 +- site/content/docs/prompt-cookbook.mdx | 145 +++++++++++++++++++ site/content/docs/testing.mdx | 201 ++++++++++++++++++++++++++ site/content/docs/tool-or-journey.mdx | 105 ++++++++++++++ 4 files changed, 455 insertions(+), 1 deletion(-) create mode 100644 site/content/docs/prompt-cookbook.mdx create mode 100644 site/content/docs/testing.mdx create mode 100644 site/content/docs/tool-or-journey.mdx diff --git a/site/content/docs/meta.json b/site/content/docs/meta.json index 707c7d1..72d7cea 100644 --- a/site/content/docs/meta.json +++ b/site/content/docs/meta.json @@ -5,9 +5,12 @@ "why-agent-tools", "quickstart", "after-you-generate", - "working-with-your-agent", + "tool-or-journey", "journeys", "journeys-faq", + "working-with-your-agent", + "prompt-cookbook", + "testing", "safety", "regeneration", "visible-effects", diff --git a/site/content/docs/prompt-cookbook.mdx b/site/content/docs/prompt-cookbook.mdx new file mode 100644 index 0000000..28ecebd --- /dev/null +++ b/site/content/docs/prompt-cookbook.mdx @@ -0,0 +1,145 @@ +--- +title: Prompt cookbook +description: Copy-paste prompts for the jobs that come up again and again on an agent surface, with what to check afterwards. +--- + +Every `generate` run writes `.agents/skills/webmcp-tools/SKILL.md`, so a coding +agent that reads your repo already knows the naming rules, the budgets, and the +journey pattern. What it cannot know is your product. These prompts give it that +context. + +Copy one, replace the names in it with yours, and run it. You do not need a +coding agent to use them; each one also works as a checklist for doing the task +by hand. + + + Read the diff before you accept it, and do not let an agent enable a write on + its own. Enabling a write is a human decision about what a model may do while + acting as your user. + + +## Improve one description + +Descriptions are the highest-leverage edit on the surface, because the model +reads them as part of the prompt. + +```text +Read .agents/skills/webmcp-tools/SKILL.md and src/webmcp/list-trips.webmcp.ts. +The description reads mechanical. Rewrite it so an agent knows what the tool +does, when to use it, and what it returns, within the skill's budget. Put the +new text in .webmcp-codegen.json, not inside the generated region. +``` + +**Then:** run `verify`, and check that the override landed in +`.webmcp-codegen.json` rather than in the generated file. + +## Write a journey + +A good journey prompt names the goal, the value the agent cannot invent, and the +step that needs a human. + +```text +Read .agents/skills/webmcp-tools/SKILL.md and the generated tools in src/webmcp. + +Write a journey called book-ride in src/webmcp/journeys. Goal: let a user book a +ride to a destination they give. + +- The ride API needs coordinates, which only the geocode endpoint can provide. + Compose the generated geocode-address tool for that step and store the result + on the draft as destination. +- The user sets a pickup time in a second step. +- Requesting the ride is a write, so it goes through the submit gate, not a + step. Use the real request-ride execute as the submit's run. + +Run npx @webmcp-stack/codegen verify when you are done and fix what it reports. +``` + +**Then:** confirm every step is a read, and the write sits in `submit`. + +## Review the whole surface + +Useful before a release or after a big spec change. + +```text +Read .agents/skills/webmcp-tools/SKILL.md and every file in src/webmcp. For each +tool, answer: is the name intent-shaped, does the description say what the tool +returns, and should an agent be able to call it at all? Do not enable any +withheld tool. Return a table with the tool, the verdict, and the reason. +``` + +**Then:** turn the table into issues or edits. An agent suggesting a write be +enabled is a question for you, not an action. + +## Name a tool or journey + +Naming is where an agent without product context goes wrong most quietly. + +```text +Before naming anything, read what this product is: [one or two sentences about +who uses it and when]. Then suggest names for [the tool or flow] that say what +the user is doing, not what the endpoint is. Give me three options and the +reasoning for each. +``` + +**Then:** pick the name yourself. A name can pass every mechanical check and +still be wrong for the product. + +## Fix a failing verify + +```text +Run npx @webmcp-stack/codegen verify and fix every error. For warnings, list +them and propose a fix for each; do not change a description's meaning just to +make it shorter. Re-run verify when you are done. +``` + +**Then:** read the warnings yourself. The trim is a judgment call, not a +mechanical one. + +## Do a safety pass + +```text +Read .agents/skills/webmcp-tools/SKILL.md. Review src/webmcp for three things: +output fields that look like personal data, descriptions that instruct the agent +instead of describing the tool, and read tools that actually perform writes. +Report findings with the file and line, and change nothing. +``` + +**Then:** decide what to fix. The report is the deliverable. + +## Add a tool that is not in the spec + +```text +I want agents to be able to [action]. Look at the tools in src/webmcp and the +source in codegen.config.mjs. Add a tool for it in the style the skill file +expects, or tell me if it should be a journey instead. Do not enable any write. +``` + +**Then:** check the schema and description against the skill's rules, and run +`verify`. + +## Enable a withheld write + +Do this one yourself, or have the agent prepare it while you approve. + +```text +I want to allow agents to [action]. Find the tool, show me what it does and +whether it confirms with the user before running, then enable that one tool in +.webmcp-codegen.json. Do not enable anything else. +``` + +**Then:** read the diff, then test the tool in the browser and go through the +confirmation dialog yourself. + +## Next + + + + What the skill file teaches, and how to check an agent's work. + + + Which shape a given problem should take. + + + How to test the generated code and a journey's logic. + + diff --git a/site/content/docs/testing.mdx b/site/content/docs/testing.mdx new file mode 100644 index 0000000..4a6ceb0 --- /dev/null +++ b/site/content/docs/testing.mdx @@ -0,0 +1,201 @@ +--- +title: Testing your tools +description: Generated tools are ordinary TypeScript. Here is how to test them without a browser, and the one thing that still needs a browser. +--- + +The generated files are plain TypeScript in your repo, and they export the +pieces you want to test: a raw caller (`fetchGetTrip`) and an agent-facing +function (`executeGetTrip`). That means most of your surface can be tested the +way you test the rest of your app. + + + A journey's draft and submit gate are library code you did not write and do + not own. Test the `store` and `build` functions you wrote, and test the whole + flow in a browser. Do not test the generated region. + + + + +### Test the request without a browser + +`fetchGetTrip` calls your API through `callApi`, which uses `fetch`. Mock +`fetch` and assert both the request and the parsed body. This is the test that +catches a wrong path, method, or body long before an agent does. + +```ts title="src/webmcp/get-trip.test.ts" +// @vitest-environment happy-dom +import { afterEach, expect, it, vi } from "vitest"; +import { fetchGetTrip } from "./get-trip.webmcp"; + +afterEach(() => vi.unstubAllGlobals()); + +it("calls the endpoint and returns the parsed body", async () => { + const calls: string[] = []; + vi.stubGlobal( + "fetch", + vi.fn(async (input: RequestInfo) => { + calls.push(String(input)); + return new Response(JSON.stringify({ id: "trip_1", title: "Lisbon" }), { + status: 200, + headers: { "content-type": "application/json" }, + }); + }), + ); + + const trip = await fetchGetTrip({ id: "trip_1" }); + + expect(calls[0]).toContain("/v1/trips/trip_1"); + expect(trip).toMatchObject({ id: "trip_1" }); +}); +``` + +The `happy-dom` environment matters: `callApi` resolves a relative path against +`window.location.origin`. + + + +### Test the result the agent reads + +`executeGetTrip` wraps the response with `toolResult`, which is the text an +agent actually sees. Assert the shape, not the whole string. + +```ts +import { executeGetTrip } from "./get-trip.webmcp"; + +it("wraps the result for the agent", async () => { + vi.stubGlobal( + "fetch", + vi.fn(async () => new Response(JSON.stringify({ id: "trip_1" }), { status: 200 })), + ); + + const result = await executeGetTrip({ id: "trip_1" }); + + expect(result.isError).toBeUndefined(); + expect(result.content[0]?.text).toContain("trip_1"); +}); +``` + + + +### Know what throws + +`fetchGetTrip` and `executeGetTrip` throw on a failed request. The readable +error an agent sees is produced one level up, in the generated registration +wrapper, which catches the throw and returns `asToolError(error)`. + +So a unit test of the raw caller should assert the throw: + +```ts +it("throws on a failed response", async () => { + vi.stubGlobal( + "fetch", + vi.fn(async () => new Response("nope", { status: 500, statusText: "Server Error" })), + ); + + await expect(fetchGetTrip({ id: "trip_1" })).rejects.toThrow("500"); +}); +``` + + + +### Test a journey's logic + +The draft and the gate belong to `journey.webmcp.ts`, so leave them alone. +Export the two functions you wrote and test those on their own. + +```ts title="src/webmcp/journeys/book-ride.webmcp.ts" +import { createJourney } from "../journey.webmcp"; +import { geocodeAddressTool, fetchGeocodeAddress } from "../geocode-address.webmcp"; +import { executeRequestRide, type RequestRideInput } from "../request-ride.webmcp"; + +export const storeDestination = (place: unknown) => ({ destination: place }); +export const buildRideInput = (draft: Readonly>) => + draft as RequestRideInput; + +export const bookRide = createJourney({ + name: "book-ride", + goal: "Book a ride to a destination the user gives", + steps: { + "resolve-destination": { + tool: geocodeAddressTool, + call: (input, signal) => fetchGeocodeAddress(input as never, signal), + store: storeDestination, + provides: ["destination"], + }, + // ... + }, + submit: { + description: "Request the ride and show the driver's details.", + build: buildRideInput, + run: executeRequestRide, + }, +}); +``` + +Now the parts with logic are ordinary functions: + +```ts title="src/webmcp/journeys/book-ride.test.ts" +import { expect, it } from "vitest"; +import { buildRideInput, storeDestination } from "./book-ride.webmcp"; + +it("stores the resolved place under its draft field", () => { + const place = { lat: 37.6, lng: -122.4, label: "SFO" }; + expect(storeDestination(place)).toEqual({ destination: place }); +}); + +it("assembles the ride request from a full draft", () => { + const input = buildRideInput({ + destination: { lat: 37.6, lng: -122.4 }, + pickupAt: "2026-03-14T19:30:00Z", + }); + expect(input).toMatchObject({ pickupAt: "2026-03-14T19:30:00Z" }); +}); +``` + + + +### Test the whole thing in a browser + +The one thing unit tests cannot cover is the real thing: registration, the +browser validating input against the schema, the confirmation dialog, and the +page updating. That needs a browser with the signed-in session. + +Open your app with WebMCP enabled, use the DevTools panel to call the tools by +hand, and go all the way through the submit gate, including the confirmation. +See [Chrome DevTools WebMCP panel](/docs/devtools). + + + + +## What not to test + +- **The generated region.** It is derived from your spec and regenerated. Test + the spec indirectly through `verify`, and test your `execute` bodies. +- **The runtime helpers.** `callApi`, `toolResult`, and `asToolError` are + library code. If you find a bug in them, fix it in your repo and send it + upstream; do not snapshot their output. +- **The draft's lifetime.** That the draft resets on reload is a documented + behavior, not something to assert in a unit test. + +## Put the mechanical checks in CI + +Unit tests cover logic. `verify` covers the surface itself: names, description +budgets, the journey gate. Run both in CI. + +```bash +npx @webmcp-stack/codegen verify +``` + +## Next + + + + Where testing fits in the wider loop. + + + What to check on screen when a tool runs. + + + The common first-run failures. + + diff --git a/site/content/docs/tool-or-journey.mdx b/site/content/docs/tool-or-journey.mdx new file mode 100644 index 0000000..d666cfe --- /dev/null +++ b/site/content/docs/tool-or-journey.mdx @@ -0,0 +1,105 @@ +--- +title: Tool, group, or journey? +description: Three shapes for three different problems. Pick the wrong one and the agent gets more surface than it can use. +--- + +The surface an agent uses well is small and intent-shaped. When you add a +capability, it takes one of three shapes. This page is how to tell which. + +## The short answer + +| The problem | The shape | What it looks like | +| --- | --- | --- | +| One action, one call | **A tool** | `cancel-order`, `search-products` | +| One action the API split into two calls | **A grouped tool** | `upload-media` wrapping request-upload and complete-upload | +| A goal that takes several calls, shares state, and ends in one write | **A journey** | `book-ride`: resolve, schedule, confirm | + +## A tool + +Most things are a tool. If an agent can express the whole action as one call +with the inputs it already has, you are done. The generator makes these from +your spec, and reads work immediately. + +The question to ask is not "does this endpoint exist" but "would a helpful +person do this in one step". If yes, it is a tool. + +## Grouping + +Some endpoints only make sense together. Requesting an upload and completing it +are one action that the API happens to split in two calls. An agent should never +have to orchestrate that handshake, so the generator detects the pair and emits +one coarse, withheld tool next to the members. + +You do not usually decide this. Grouping happens at generate time from the spec, +merges only when the rules are exact, and skips anything fuzzy with a note. You +adopt the proposal by enabling it. + +## A journey + +Reach for a journey when all of these are true: + +- **The goal takes more than one call.** Resolve a destination, then set a time, + then request the ride. +- **The steps share state.** A later step or the submit needs what an earlier + step produced. +- **At least one input cannot be invented.** The ride needs coordinates, which + only the geocoder can provide. An agent that guesses gets rejected. +- **The end is a real write that a person should confirm.** + +If a goal has the first two but not the last, you may still want a journey: it +is how you keep a multi-call flow from becoming several loose tools the agent +has to sequence itself. + +The [journeys page](/docs/journeys) is the walkthrough. + +## Signs it should not be a journey + +- **One call.** That is a tool. A one-step journey is a tool with extra steps. +- **No shared state.** If each step is independent, the agent can just call the + tools. Wrapping them adds surface without adding safety. +- **More than five steps.** Past five, you are usually describing two journeys. +- **It is really about navigation.** Moving the page around is app behavior, not + a tool call. Keep it in your UI. + +## Signs it should not be a tool + +- **The name mirrors the route.** `post-v1-trips-id-cancel` is an endpoint + wearing a name. The intent is `cancel-trip`. +- **It only makes sense inside a flow.** If calling it alone leaves the app in a + broken state, it belongs to a journey, not the public surface. +- **Two tools are really one decision.** If an agent always calls them together, + they are one tool. + + + + The draft is one object per page load, so two parallel attempts at the same + journey share it and overwrite each other. That is accepted, not handled. + Agents work one thing at a time in practice, and a reload starts clean. + + + Read it from that journey's draft is not supported, because drafts are + private to a journey. If two flows genuinely share state, they are probably + one journey. If they do not, pass the value from the first submit's result + into the second journey's step as normal input. + + + Whether a step navigates is app behavior, not a tool concern. The journey's + submit runs your real tool, so whatever that tool does to the page when a + person clicks the button is what should happen here too. See + [Make the effect visible](/docs/visible-effects). + + + +## Next + + + + The multi-step primitive, with a full example. + + + How classification and withholding decide what agents can reach. + + + The loop that turns the surface into one you trust. + + From 533133994d3e3a6e6afb6696a462974f3882c2b2 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 23:59:17 +0530 Subject: [PATCH 30/41] docs: put the standard components to work in the guides - Quickstart: Tabs for the Next.js and Vite registration edits. - After you generate: a Files tree of the tools directory. - Journeys: inline code annotations on store, provides, and build, the lines the state-movement section talks about. - Working with your coding agent: point at the prompt cookbook. --- site/content/docs/after-you-generate.mdx | 16 ++++++++++++++++ site/content/docs/journeys.mdx | 11 ++++++----- site/content/docs/quickstart.mdx | 16 +++++++++++++--- site/content/docs/working-with-your-agent.mdx | 4 ++++ 4 files changed, 39 insertions(+), 8 deletions(-) diff --git a/site/content/docs/after-you-generate.mdx b/site/content/docs/after-you-generate.mdx index 2e7062c..6ee2034 100644 --- a/site/content/docs/after-you-generate.mdx +++ b/site/content/docs/after-you-generate.mdx @@ -25,6 +25,22 @@ The dashboard is for understanding, not decorating: every tool with its route, risk label, and findings. `verify` tells you where the surface is weak. Fix errors first; they block generation. Warnings can wait until the tool matters. +A generated project looks like this. One file per tool, plus the shared runtime, +the journey factory, and the barrel that registers everything: + + + + + + + + + + + + + + ## 2. Decide what agents may actually do Reads are registered so you can see the surface. Writes and destructive tools diff --git a/site/content/docs/journeys.mdx b/site/content/docs/journeys.mdx index dcc4fd3..c114af3 100644 --- a/site/content/docs/journeys.mdx +++ b/site/content/docs/journeys.mdx @@ -67,8 +67,8 @@ export const bookRide = createJourney({ "resolve-destination": { tool: geocodeAddressTool, call: (input, signal) => fetchGeocodeAddress(input as never, signal), - store: (place) => ({ destination: place }), - provides: ["destination"], + store: (place) => ({ destination: place }), // [!code highlight] + provides: ["destination"], // [!code highlight] }, // A free step. No API call: the pickup time the agent passed in is @@ -82,14 +82,15 @@ export const bookRide = createJourney({ }, required: ["pickupAt"], }, - provides: ["pickupAt"], + provides: ["pickupAt"], // [!code highlight] }, }, submit: { description: "Request the ride and show the driver's details.", - build: (draft) => draft as RequestRideInput, // the draft becomes the tool's input - run: executeRequestRide, // the real write runs here, after the human confirms + // The draft becomes the real tool's input; the write runs after the human confirms. + build: (draft) => draft as RequestRideInput, // [!code highlight] + run: executeRequestRide, }, }); ``` diff --git a/site/content/docs/quickstart.mdx b/site/content/docs/quickstart.mdx index df623ac..1afa764 100644 --- a/site/content/docs/quickstart.mdx +++ b/site/content/docs/quickstart.mdx @@ -37,9 +37,19 @@ npx @webmcp-stack/codegen generate --dry-run Generated files do nothing until your app registers them once at startup. The CLI does this edit for you: -- **Next.js (app router):** creates `src/webmcp/register.tsx` (a small client component) - and adds an import plus `` to your root layout. -- **Vite + React:** adds an import and one `registerAllTools()` call to `main.tsx`. + + + +Creates `src/webmcp/register.tsx`, a small client component, and adds an import plus +`` to your root layout. + + + + +Adds an import and one `registerAllTools()` call to `main.tsx`. + + + Edits are additive only, idempotent, and printed in the report with undo instructions. If the CLI can't find your entry file confidently, it prints the two lines instead of diff --git a/site/content/docs/working-with-your-agent.mdx b/site/content/docs/working-with-your-agent.mdx index 1fcaa8b..884606d 100644 --- a/site/content/docs/working-with-your-agent.mdx +++ b/site/content/docs/working-with-your-agent.mdx @@ -34,6 +34,10 @@ skills stack, so your project rules sit on top of these. You do not have to do anything to activate it. A capable coding agent opens the repo, and the rules are in front of it. Your job is to ask for the right thing. +There is a page of ready-to-use prompts for the recurring jobs in the +[Prompt cookbook](/docs/prompt-cookbook). The rest of this page explains why +they work and how to check the result. + ## What to give the agent Three things turn a vague request into a good tool or journey: From c9d74d7f23fc3e3b53beedda12ad1205655928c7 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Fri, 11 Sep 2026 23:59:17 +0530 Subject: [PATCH 31/41] docs(site): strip code annotations from llms-full.txt The full-text feed is raw markdown, so Fumadocs' presentation-only "// [!code highlight]" markers were leaking into it. --- site/app/llms-full.txt/route.ts | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/site/app/llms-full.txt/route.ts b/site/app/llms-full.txt/route.ts index a992e38..52f7288 100644 --- a/site/app/llms-full.txt/route.ts +++ b/site/app/llms-full.txt/route.ts @@ -24,7 +24,12 @@ export async function GET() { for (const slug of meta.pages) { const raw = await readFile(join(dir, `${slug}.mdx`), "utf8"); - const body = raw.replace(/^---[\s\S]*?---\s*/, "").trim(); + const body = raw + .replace(/^---[\s\S]*?---\s*/, "") + // Fumadocs code annotations are presentation only; a reader of the raw + // markdown should not see "// [!code highlight]". + .replace(/ ?\/\/ ?\[!code[^\]]*\]/g, "") + .trim(); parts.push(body, "", "---", ""); } From dc6d90dc3ca9a67168059c8ef22ae932fd17daaf Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 00:41:25 +0530 Subject: [PATCH 32/41] docs(site): centralize the site origin and move to the new domain The origin was hardcoded in four files. It now lives in lib/site-url.ts, overridable with NEXT_PUBLIC_SITE_URL, so page metadata, sitemap, robots, and the llms.txt feeds cannot drift apart again. --- site/app/layout.tsx | 3 ++- site/app/llms.txt/route.ts | 3 ++- site/app/robots.ts | 3 ++- site/app/sitemap.ts | 3 ++- site/lib/site-url.ts | 7 +++++++ 5 files changed, 15 insertions(+), 4 deletions(-) create mode 100644 site/lib/site-url.ts diff --git a/site/app/layout.tsx b/site/app/layout.tsx index f1b72ea..811b659 100644 --- a/site/app/layout.tsx +++ b/site/app/layout.tsx @@ -3,6 +3,7 @@ import { RootProvider } from "fumadocs-ui/provider"; import type { Metadata, Viewport } from "next"; import { Caveat, Inter, JetBrains_Mono, Space_Grotesk } from "next/font/google"; import type { ReactNode } from "react"; +import { SITE_URL } from "@/lib/site-url"; const inter = Inter({ subsets: ["latin"], variable: "--font-inter" }); const spaceGrotesk = Space_Grotesk({ subsets: ["latin"], variable: "--font-space-grotesk" }); @@ -13,7 +14,7 @@ const DESCRIPTION = "Turn an OpenAPI spec into safe, typed, human-reviewed WebMCP tools. Real files in your repo: contracts regenerate, your code survives, safety audit built in."; export const metadata: Metadata = { - metadataBase: new URL("https://webmcp-stack.vercel.app"), + metadataBase: new URL(SITE_URL), title: { template: "%s | webmcp-stack", default: "webmcp-stack: generate WebMCP tools from the API spec you already have", diff --git a/site/app/llms.txt/route.ts b/site/app/llms.txt/route.ts index 8ebf21a..b086d85 100644 --- a/site/app/llms.txt/route.ts +++ b/site/app/llms.txt/route.ts @@ -1,9 +1,10 @@ +import { SITE_URL } from "@/lib/site-url"; import { source } from "@/lib/source"; /** Generated at build time, so it is a static file on the deployed site. */ export const dynamic = "force-static"; -const BASE = "https://webmcp-stack.vercel.app"; +const BASE = SITE_URL; /** * The llms.txt index: a short, curated map of the docs for an agent to read diff --git a/site/app/robots.ts b/site/app/robots.ts index 41e8706..2f9fc3c 100644 --- a/site/app/robots.ts +++ b/site/app/robots.ts @@ -1,6 +1,7 @@ import type { MetadataRoute } from "next"; +import { SITE_URL } from "@/lib/site-url"; -const BASE = "https://webmcp-stack.vercel.app"; +const BASE = SITE_URL; /** * Everything here is meant to be read, including by AI agents. The docs also diff --git a/site/app/sitemap.ts b/site/app/sitemap.ts index c13d643..f4bb4ee 100644 --- a/site/app/sitemap.ts +++ b/site/app/sitemap.ts @@ -1,7 +1,8 @@ import type { MetadataRoute } from "next"; +import { SITE_URL } from "@/lib/site-url"; import { source } from "@/lib/source"; -const BASE = "https://webmcp-stack.vercel.app"; +const BASE = SITE_URL; export default function sitemap(): MetadataRoute.Sitemap { const pages = source.getPages().map((page) => ({ diff --git a/site/lib/site-url.ts b/site/lib/site-url.ts new file mode 100644 index 0000000..d4d8572 --- /dev/null +++ b/site/lib/site-url.ts @@ -0,0 +1,7 @@ +/** + * The canonical production origin. One place to change it: page metadata, the + * sitemap, robots.txt, and the llms.txt feeds all read from here. + * + * Set NEXT_PUBLIC_SITE_URL to override it (previews, a custom domain move). + */ +export const SITE_URL = process.env.NEXT_PUBLIC_SITE_URL ?? "https://webmcp.souravinsights.com"; From f75b90598ea0ead32b4e23d013ada6c34559fe8d Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 00:41:26 +0530 Subject: [PATCH 33/41] chore: point the CLI and README docs links at the new domain The production site moved to https://webmcp.souravinsights.com, so the docs links printed by the CLI and shown in both READMEs follow it. --- .changeset/new-docs-domain.md | 6 ++++++ README.md | 6 +++--- packages/codegen/README.md | 4 ++-- packages/codegen/src/cli-output.ts | 4 ++-- packages/codegen/src/cli.ts | 4 ++-- 5 files changed, 15 insertions(+), 9 deletions(-) create mode 100644 .changeset/new-docs-domain.md diff --git a/.changeset/new-docs-domain.md b/.changeset/new-docs-domain.md new file mode 100644 index 0000000..eae5c87 --- /dev/null +++ b/.changeset/new-docs-domain.md @@ -0,0 +1,6 @@ +--- +"@webmcp-stack/codegen": patch +--- + +Point the CLI's docs links at the new production site, +https://webmcp.souravinsights.com. diff --git a/README.md b/README.md index b852d62..c5567ed 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@

diff --git a/packages/codegen/README.md b/packages/codegen/README.md index d6eaf56..fd87667 100644 --- a/packages/codegen/README.md +++ b/packages/codegen/README.md @@ -1,5 +1,5 @@
- + webmcp-stack

@webmcp-stack/codegen

@@ -9,7 +9,7 @@ MIT license

- Docs | + Docs | GitHub | Issues

diff --git a/packages/codegen/src/cli-output.ts b/packages/codegen/src/cli-output.ts index 93eae6e..15d36c6 100644 --- a/packages/codegen/src/cli-output.ts +++ b/packages/codegen/src/cli-output.ts @@ -243,7 +243,7 @@ export function renderSummary( console.log(` ${bold("Next:")} ${c.cyan("npx @webmcp-stack/codegen dev")}`); console.log(dim(" Review your tools, edit descriptions, test them live")); console.log(""); - console.log(dim(` Docs: https://webmcp-stack.vercel.app/docs`)); + console.log(dim(` Docs: https://webmcp.souravinsights.com/docs`)); console.log(""); } @@ -347,6 +347,6 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string } console.log(dim(`Files: ${setup.config.outputs[0]?.outDir ?? "src/webmcp"}`)); - console.log(dim(`Docs: https://webmcp-stack.vercel.app/docs`)); + console.log(dim(`Docs: https://webmcp.souravinsights.com/docs`)); console.log(""); } diff --git a/packages/codegen/src/cli.ts b/packages/codegen/src/cli.ts index a758617..07a6293 100644 --- a/packages/codegen/src/cli.ts +++ b/packages/codegen/src/cli.ts @@ -85,7 +85,7 @@ Examples npx @webmcp-stack/codegen generate --verbose # see all 72 tools listed Docs - https://webmcp-stack.vercel.app/docs + https://webmcp.souravinsights.com/docs `; export interface CliFlags { @@ -216,7 +216,7 @@ async function init(): Promise { info("schemas; see the commented block in the config."); } info("\nEdit it to add sources, change the output directory, or set safety options."); - info("Docs: https://webmcp-stack.vercel.app/docs/configuration\n"); + info("Docs: https://webmcp.souravinsights.com/docs/configuration\n"); return 0; } From 962f2dd706f8eb43a9a55b71f762f38d5b3285ee Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 12:46:50 +0530 Subject: [PATCH 34/41] docs: keep the docs and generated output to plain ASCII Replace special punctuation with characters any keyboard can type, across the public docs, the CLI output, and the files the generator writes. The generated region marker becomes plain hyphens; reads still accept the old box-drawing marker so files generated by an earlier version migrate on the next run instead of looking hand-edited. Fixture and example regenerated to match. --- README.md | 4 +- .../.agents/skills/webmcp-tools/SKILL.md | 130 +++++++++ examples/openapi-petstore/README.md | 4 +- examples/openapi-petstore/openapi.yaml | 2 +- .../src/webmcp/adopt-pet.webmcp.ts | 25 +- .../src/webmcp/create-pet.webmcp.ts | 17 +- .../src/webmcp/delete-pet.webmcp.ts | 17 +- .../src/webmcp/get-pet.webmcp.ts | 17 +- examples/openapi-petstore/src/webmcp/index.ts | 2 + .../src/webmcp/journey.webmcp.ts | 267 ++++++++++++++++++ .../src/webmcp/list-pets.webmcp.ts | 13 +- .../src/webmcp/runtime.webmcp.ts | 57 +++- packages/codegen/README.md | 10 +- packages/codegen/assets/journey.webmcp.ts | 27 +- packages/codegen/assets/skill/SKILL.md | 24 +- packages/codegen/evals/skill/README.md | 20 +- packages/codegen/evals/skill/cases.json | 2 +- .../.agents/skills/webmcp-tools/SKILL.md | 24 +- .../codegen/evals/skill/fixture/README.md | 2 +- .../fixture/src/webmcp/create-trip.webmcp.ts | 4 +- .../src/webmcp/get-autocomplete.webmcp.ts | 4 +- .../fixture/src/webmcp/journey.webmcp.ts | 27 +- .../fixture/src/webmcp/list-trips.webmcp.ts | 4 +- .../fixture/src/webmcp/runtime.webmcp.ts | 2 +- packages/codegen/evals/skill/run.mjs | 14 +- packages/codegen/src/cli-output.ts | 36 +-- packages/codegen/src/cli.ts | 16 +- packages/codegen/src/data-file.ts | 2 +- packages/codegen/src/describe.test.ts | 6 +- packages/codegen/src/describe.ts | 19 +- packages/codegen/src/detect-app.ts | 6 +- packages/codegen/src/dev/server.ts | 6 +- packages/codegen/src/dev/ui.ts | 24 +- packages/codegen/src/group.ts | 10 +- packages/codegen/src/index.ts | 2 +- packages/codegen/src/json-schema.ts | 4 +- packages/codegen/src/logger.ts | 2 +- packages/codegen/src/naming.test.ts | 2 +- packages/codegen/src/naming.ts | 64 ++--- packages/codegen/src/outputs/assets.ts | 6 +- .../codegen/src/outputs/tools-templates.ts | 18 +- packages/codegen/src/outputs/tools.test.ts | 8 +- packages/codegen/src/outputs/tools.ts | 41 ++- packages/codegen/src/pipeline.ts | 12 +- packages/codegen/src/safety.ts | 2 +- packages/codegen/src/sources/openapi.test.ts | 2 +- packages/codegen/src/sources/openapi.ts | 10 +- packages/codegen/src/sources/schema.ts | 4 +- packages/codegen/src/types.ts | 2 +- packages/codegen/src/verify.test.ts | 2 +- packages/codegen/src/verify.ts | 42 +-- packages/codegen/src/wire.ts | 20 +- site/content/docs/cli.mdx | 10 +- site/content/docs/devtools.mdx | 6 +- site/content/docs/journeys-faq.mdx | 2 +- site/content/docs/journeys.mdx | 2 +- site/content/docs/quickstart.mdx | 2 +- site/content/docs/regeneration.mdx | 4 +- 58 files changed, 812 insertions(+), 301 deletions(-) create mode 100644 examples/openapi-petstore/.agents/skills/webmcp-tools/SKILL.md create mode 100644 examples/openapi-petstore/src/webmcp/journey.webmcp.ts diff --git a/README.md b/README.md index c5567ed..d4d72d5 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,7 @@ You can, and it works. What you get back is different every time, and nothing ch A real tool from a real app: `create-trip`, one of 70+ tools generated for [beenthere.page](https://beenthere.page) from its OpenAPI spec, shortened for the README: ```ts -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * Create a new trip. Returns the trip. * Source: POST /v1/trips/ (openapi). Risk: write-confirm. @@ -89,7 +89,7 @@ export async function fetchCreateTrip(input: CreateTripInput, signal?: AbortSign // const confirmed = await requestUserConfirmation( // "Allow the agent to: Create a new trip. Returns the trip.", // ); -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- export async function executeCreateTrip(input: CreateTripInput, signal?: AbortSignal) { return toolDisabled("create-trip.webmcp.ts"); diff --git a/examples/openapi-petstore/.agents/skills/webmcp-tools/SKILL.md b/examples/openapi-petstore/.agents/skills/webmcp-tools/SKILL.md new file mode 100644 index 0000000..356ca92 --- /dev/null +++ b/examples/openapi-petstore/.agents/skills/webmcp-tools/SKILL.md @@ -0,0 +1,130 @@ +--- +name: webmcp-tools +description: Build, improve, or review this repo's WebMCP tools and journeys - the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. +--- + + + +# WebMCP tools in this repo + +WebMCP tools run in the visitor's browser, with the signed-in user's session, +while the agent calling them may also be reading attacker-influenced page +content. Every tool you write is both an API and a security surface. The rules +below exist so an agent-facing tool is correct by default - follow them even +when the user's request is casual. + +## Naming + +- Verb-first, intent-shaped, max 30 characters: `list-trips`, + `add-bucket-list-destination`. Never method-first (`get-v1-trips`), never + numbered (`get-pricing-2`). +- Name the user's intent, not the endpoint. `POST /search` is a read named + `search-...`; `POST /orders/{id}/cancel` is destructive named `cancel-...`. +- **Understand the product before naming anything.** What is it, who uses it, + and when in the user's life does this action happen? If that is not written + down in the repo, ask the user before naming intent-level tools or journeys. + Wrong-tense or wrong-role names pass every mechanical check and are still + wrong: on a journal for trips you've *been on*, the flow is `document-trip`, + never `plan-trip`. This is the failure no linter can catch - it is your job. + +## Descriptions + +- Say what the tool does, when to use it, and what it returns: + "Create a new trip. Returns the trip." +- Max 500 characters for the tool, max 150 per parameter. Turn constraints + into sentences: "A number from 30 to 600." +- Never instruct the agent or encode flow control in a description + ("always call X first") - that is steering. Prerequisites belong in a + journey, not in prose. +- If a field's value can only come from another tool (a resolved place object, + a server id), say so in that field's description. + +## Safety and exposure + +- Reads are registered immediately. Writes and destructive tools stay + withheld - generated but not registered - until the user deliberately + enables one. Never enable a write tool without being asked. +- Mutating tools confirm each call with the human via + `requestUserConfirmation`. That call lives in the generated region; never + move or remove it. +- Free-text outputs get `untrustedContentHint: true` - the agent must not + treat user-written content as the site speaking. +- **The schema is not the security boundary.** `execute` must call the app's + real endpoint or action layer, so server-side validation runs on every + call. Never wire `execute` to return canned data or bypass the app's own + flow (cache invalidation, navigation, stores). + +## The execute contract + +- Never throw for failure. The browser maps a rejected `execute` to a bare + `UnknownError` and discards your message. Return `toolError(message)` / + `asToolError(error)` so the agent can read and recover. (Cancellation is + the one exception: let `AbortError` propagate.) +- Return via `toolResult(data)` and keep outputs under ~1.5K characters - + summarize or paginate rather than dumping. +- When a call changes what is on screen, make it visible: navigate, + invalidate a query, dispatch an event. The human is watching the page. + +## Editing generated files + +- Each `*.webmcp.ts` has a generated region between the + `webmcp-codegen` markers - never edit inside it; regeneration rewrites it. + Your work goes below the marker (the `execute` body) or in + `.webmcp-codegen.json` (description, enabled, and field-text overrides, which + survive regeneration and always win over generated text). +- After editing tools, run `npx @webmcp-stack/codegen verify` and fix what it + reports. + +## Journeys (multi-step flows) + +Reach for a journey when a goal takes several calls with shared state, when +an input can't be invented (a resolved place object), or when a spend should +be gated (an eligibility check before a paid generation). Journey files live +in the `journeys/` folder inside the generated tools directory +(`src/webmcp/journeys/document-trip.webmcp.ts`); re-run `verify` after +writing one. Pattern: + +```ts +// src/webmcp/journeys/document-trip.webmcp.ts +import { createJourney } from "../journey.webmcp"; +import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; +import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; + +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + // Tool-backed step: inherits the generated tool's description and schema; + // you write only what lands in the draft. + "search-places": { + tool: getAutocompleteTool, + call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), + store: (places) => ({ locationObject: places }), // store the resolved pick + provides: ["locationObject"], + }, + // Freeform step: no backend call - collects input straight into the draft. + "set-details": { + description: "Set the trip's title and dates.", + input: { type: "object", properties: { title: { type: "string" } }, required: ["title"] }, + provides: ["title"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft as CreateTripInput, // assemble the real tool's input + run: executeCreateTrip, // the existing tool does the work + }, +}); +``` + +- 2-5 steps. More means two journeys. +- Tool-backed steps reuse the generated tool's contract and its raw caller + (`fetchX`); the submit's `run` is the real write tool's `execute`, so its + confirmation and validation still apply. Never write a direct `fetch` in a + journey file - `verify` flags it. +- Keep real writes in the submit gate. A tool-backed step inherits the composed + tool's read-only hint, and a free step that only stores input is read-only + too. Never route a mutating call through a step: it would be advertised as + safe and skip the confirmation the submit performs. diff --git a/examples/openapi-petstore/README.md b/examples/openapi-petstore/README.md index 7593a4e..11b8409 100644 --- a/examples/openapi-petstore/README.md +++ b/examples/openapi-petstore/README.md @@ -1,4 +1,4 @@ -# Example: OpenAPI → WebMCP tools +# Example: OpenAPI -> WebMCP tools The smallest possible end-to-end demo of `webmcp-codegen`: a trimmed Petstore OpenAPI spec, one config file, and the generated output (committed so you can @@ -30,7 +30,7 @@ node ../../packages/codegen/dist/cli.js generate ## The regeneration promise -Edit the `throw new Error("Not implemented…")` in any tool's `execute()` to +Edit the `throw new Error("Not implemented...")` in any tool's `execute()` to return something real, then change a description in `openapi.yaml` and re-run `generate`. The description updates; your `execute()` is untouched. That split is the point of the tool. diff --git a/examples/openapi-petstore/openapi.yaml b/examples/openapi-petstore/openapi.yaml index 25a60ba..b664068 100644 --- a/examples/openapi-petstore/openapi.yaml +++ b/examples/openapi-petstore/openapi.yaml @@ -72,7 +72,7 @@ paths: /pets/{id}/adopt: post: operationId: adoptPet - summary: Adopt a pet — this finalizes the adoption paperwork + summary: Adopt a pet - this finalizes the adoption paperwork parameters: - name: id in: path diff --git a/examples/openapi-petstore/src/webmcp/adopt-pet.webmcp.ts b/examples/openapi-petstore/src/webmcp/adopt-pet.webmcp.ts index 9c62766..33dc471 100644 --- a/examples/openapi-petstore/src/webmcp/adopt-pet.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/adopt-pet.webmcp.ts @@ -1,8 +1,8 @@ -import { toolDisabled } from "./runtime.webmcp"; +import { callApi, toolDisabled } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** - * Adopt a pet — this finalizes the adoption paperwork. Returns the pet. + * Adopt a pet - this finalizes the adoption paperwork. Returns the pet. * * Source: POST /pets/{id}/adopt (openapi). Risk: write-confirm. * Starts withheld: not registered until you enable it (see registerAdoptPet below). @@ -15,7 +15,7 @@ export const adoptPetInputSchema = { "properties": { "id": { "type": "string", - "description": "Id." + "description": "The unique identifier of the pet." } }, "required": [ @@ -32,14 +32,23 @@ export const adoptPetHints = {"readOnlyHint":false,"destructiveHint":false,"idem /** The tool definition, minus `execute` (which is yours, below the marker). */ export const adoptPetTool = { name: "adopt-pet", - description: "Adopt a pet — this finalizes the adoption paperwork. Returns the pet.", + title: "Adopt Pet", + description: "Adopt a pet - this finalizes the adoption paperwork. Returns the pet.", inputSchema: adoptPetInputSchema, annotations: { readOnlyHint: false, untrustedContentHint: true, + consequentialHint: false, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeAdoptPet is the agent-facing one. */ +export async function fetchAdoptPet(input: AdoptPetInput, signal?: AbortSignal) { + const data = await callApi(`/pets/${input.id}/adopt`, { method: "POST", signal }); + return data; +} + /** * Withheld: this tool is not registered, so agents cannot see or pick * it. The registration below stays commented until you enable the tool @@ -59,7 +68,7 @@ export async function registerAdoptPet(signal?: AbortSignal): Promise { // // This tool changes things, so the user is always asked first. The // // confirmation lives in the generated region: it cannot be edited away. // const confirmed = await requestUserConfirmation( - // "Allow the agent to: Adopt a pet — this finalizes the adoption paperwork. Returns the pet.", + // "Allow the agent to: Adopt a pet - this finalizes the adoption paperwork. Returns the pet.", // ); // if (!confirmed) { // return { @@ -80,7 +89,7 @@ export async function registerAdoptPet(signal?: AbortSignal): Promise { // ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "adopt-pet". @@ -93,7 +102,7 @@ export async function registerAdoptPet(signal?: AbortSignal): Promise { * The user is asked to confirm every call (built into the generated region). */ // -// ⚠ webmcp-codegen flagged these response fields as likely PII: owner.email. +// ! webmcp-codegen flagged these response fields as likely PII: owner.email. // Everything you return reaches the agent. Leave those fields out of what you // return unless the agent genuinely needs them, and say so in a comment if you keep them. export async function executeAdoptPet(input: AdoptPetInput) { diff --git a/examples/openapi-petstore/src/webmcp/create-pet.webmcp.ts b/examples/openapi-petstore/src/webmcp/create-pet.webmcp.ts index 6018ac3..b3c5f23 100644 --- a/examples/openapi-petstore/src/webmcp/create-pet.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/create-pet.webmcp.ts @@ -1,6 +1,6 @@ -import { toolDisabled } from "./runtime.webmcp"; +import { callApi, toolDisabled } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * Add a new pet to the store. Returns the pet. * @@ -36,14 +36,23 @@ export const createPetHints = {"readOnlyHint":false,"destructiveHint":false,"ide /** The tool definition, minus `execute` (which is yours, below the marker). */ export const createPetTool = { name: "create-pet", + title: "Create Pet", description: "Add a new pet to the store. Returns the pet.", inputSchema: createPetInputSchema, annotations: { readOnlyHint: false, untrustedContentHint: true, + consequentialHint: false, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeCreatePet is the agent-facing one. */ +export async function fetchCreatePet(input: CreatePetInput, signal?: AbortSignal) { + const data = await callApi("/pets", { method: "POST", body: { name: input.name, tag: input.tag }, signal }); + return data; +} + /** * Withheld: this tool is not registered, so agents cannot see or pick * it. The registration below stays commented until you enable the tool @@ -84,7 +93,7 @@ export async function registerCreatePet(signal?: AbortSignal): Promise { // ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "create-pet". @@ -97,7 +106,7 @@ export async function registerCreatePet(signal?: AbortSignal): Promise { * The user is asked to confirm every call (built into the generated region). */ // -// ⚠ webmcp-codegen flagged these response fields as likely PII: owner.email. +// ! webmcp-codegen flagged these response fields as likely PII: owner.email. // Everything you return reaches the agent. Leave those fields out of what you // return unless the agent genuinely needs them, and say so in a comment if you keep them. export async function executeCreatePet(input: CreatePetInput) { diff --git a/examples/openapi-petstore/src/webmcp/delete-pet.webmcp.ts b/examples/openapi-petstore/src/webmcp/delete-pet.webmcp.ts index 57b556b..d2425a2 100644 --- a/examples/openapi-petstore/src/webmcp/delete-pet.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/delete-pet.webmcp.ts @@ -1,6 +1,6 @@ -import { toolDisabled } from "./runtime.webmcp"; +import { callApi, toolDisabled } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * Remove a pet from the store permanently * @@ -15,7 +15,7 @@ export const deletePetInputSchema = { "properties": { "id": { "type": "string", - "description": "Id." + "description": "The unique identifier of the pet." } }, "required": [ @@ -32,14 +32,23 @@ export const deletePetHints = {"readOnlyHint":false,"destructiveHint":true,"idem /** The tool definition, minus `execute` (which is yours, below the marker). */ export const deletePetTool = { name: "delete-pet", + title: "Delete Pet", description: "Remove a pet from the store permanently", inputSchema: deletePetInputSchema, annotations: { readOnlyHint: false, untrustedContentHint: false, + consequentialHint: true, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeDeletePet is the agent-facing one. */ +export async function fetchDeletePet(input: DeletePetInput, signal?: AbortSignal) { + const data = await callApi(`/pets/${input.id}`, { method: "DELETE", signal }); + return data; +} + /** * Withheld: this tool is not registered, so agents cannot see or pick * it. The registration below stays commented until you enable the tool @@ -80,7 +89,7 @@ export async function registerDeletePet(signal?: AbortSignal): Promise { // ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "delete-pet". diff --git a/examples/openapi-petstore/src/webmcp/get-pet.webmcp.ts b/examples/openapi-petstore/src/webmcp/get-pet.webmcp.ts index 608f0e1..1238b39 100644 --- a/examples/openapi-petstore/src/webmcp/get-pet.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/get-pet.webmcp.ts @@ -1,6 +1,6 @@ import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * Get one pet, including its owner's contact details. Returns the pet. * @@ -15,7 +15,7 @@ export const getPetInputSchema = { "properties": { "id": { "type": "string", - "description": "Id." + "description": "The unique identifier of the pet." } }, "required": [ @@ -32,14 +32,23 @@ export const getPetHints = {"readOnlyHint":true,"destructiveHint":false,"idempot /** The tool definition, minus `execute` (which is yours, below the marker). */ export const getPetTool = { name: "get-pet", + title: "Get Pet", description: "Get one pet, including its owner's contact details. Returns the pet.", inputSchema: getPetInputSchema, annotations: { readOnlyHint: true, untrustedContentHint: true, + consequentialHint: false, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeGetPet is the agent-facing one. */ +export async function fetchGetPet(input: GetPetInput, signal?: AbortSignal) { + const data = await callApi(`/pets/${input.id}`, { method: "GET", signal }); + return data; +} + /** * Register this tool with WebMCP. Call it once on page load, or use * registerAllTools() from the generated index.ts. Skips quietly when the @@ -67,7 +76,7 @@ export async function registerGetPet(signal?: AbortSignal): Promise { ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "get-pet". @@ -77,7 +86,7 @@ export async function registerGetPet(signal?: AbortSignal): Promise { * whenever you like; the contract above never changes. */ // -// ⚠ webmcp-codegen flagged these response fields as likely PII: owner.email. +// ! webmcp-codegen flagged these response fields as likely PII: owner.email. // Everything you return reaches the agent. Leave those fields out of what you // return unless the agent genuinely needs them, and say so in a comment if you keep them. export async function executeGetPet(input: GetPetInput) { diff --git a/examples/openapi-petstore/src/webmcp/index.ts b/examples/openapi-petstore/src/webmcp/index.ts index 516b540..7f38ef6 100644 --- a/examples/openapi-petstore/src/webmcp/index.ts +++ b/examples/openapi-petstore/src/webmcp/index.ts @@ -33,4 +33,6 @@ export async function registerAllTools(signal?: AbortSignal): Promise { console.warn("[webmcp-codegen] a tool failed to register:", error); } } + // Drop journey definitions into ./journeys/ and re-run `generate`: + // the next barrel registers every createJourney() export it finds there. } diff --git a/examples/openapi-petstore/src/webmcp/journey.webmcp.ts b/examples/openapi-petstore/src/webmcp/journey.webmcp.ts new file mode 100644 index 0000000..a67ca24 --- /dev/null +++ b/examples/openapi-petstore/src/webmcp/journey.webmcp.ts @@ -0,0 +1,267 @@ +/** + * Written by webmcp-codegen on every `generate` run. Do not edit by hand; + * your changes will be lost. This file is fully ours - journey definitions + * (your code) live in journeys/*.webmcp.ts and import createJourney from here. + * + * createJourney: multi-step agent flows with a shared draft and one submit. + * + * The three pieces, literally: + * + * 1. THE DRAFT - one plain object per page load (`let draft = {}` below). + * Nothing fancier: steps write their results into it, the submit reads + * from it. It dies with the page; a half-finished journey does not + * survive a reload, which is what you want. + * + * 2. STEP TOOLS - ordinary registered WebMCP tools, one per step, named + * "-" (e.g. "document-trip-search-places"). A step's + * execute stores what it produced into the draft, then replies with what + * is still missing, so the agent always knows the next move. + * + * 3. THE SUBMIT GATE - one more registered tool, "-submit". Its + * execute, in order: refuses with the list of missing steps, asks the + * human to confirm, runs the real tool's execute with the assembled + * input, clears the draft. There is no way to submit around it, because + * the real input only exists inside build(draft). + */ + +import { + asToolError, + getModelContext, + requestUserConfirmation, + toolError, + toolResult, + type WebMcpToolResult, +} from "./runtime.webmcp"; + +type Json = Record; + +/** + * A step backed by an existing generated tool. The step inherits the tool's + * description and input schema - the definition lives in one place - and + * calls the raw caller the generated file exports (fetchGetAutocomplete, + * not the agent-facing execute wrapper). You write only what's new: which + * slice of the result lands in the draft. + */ +export interface ToolStep { + /** The generated tool object, e.g. getAutocompleteTool. */ + tool: { + description?: string; + inputSchema?: Json; + /** The generated tool's annotations. The step inherits readOnlyHint from it. */ + annotations?: { readOnlyHint?: boolean }; + }; + /** + * The raw caller the generated file exports. Receives the step's input + * plus the draft so far, so a later step can feed on an earlier one's + * stored fields. + */ + call: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** Map the call's result into the draft fields this step leaves behind. */ + store: (result: unknown) => Json; + /** Draft fields this step leaves behind. Submit waits for all of them. */ + provides: string[]; + /** Override the agent-facing description. Default: the tool's own. */ + description?: string; + /** Override the agent-facing input schema. Default: the tool's own. */ + input?: Json; + /** + * Override the read-only hint. Default: the composed tool's own hint, so a + * step that composes a read stays read-only and a step that composes a + * write does not. Set this only when you know the step's behavior differs. + */ + readOnly?: boolean; +} + +/** + * A step with no backend call of its own - it collects input into the draft + * ("set the title and dates"). With a `run`, it can do work first; whatever + * `run` returns is stored. Without one, the input is stored verbatim. + */ +export interface FreeStep { + description: string; + input: Json; + provides: string[]; + run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** + * Override the read-only hint. Default: true when the step has no `run` + * (it only stores its input in the draft), false when `run` can do work. + */ + readOnly?: boolean; +} + +export type JourneyStep = ToolStep | FreeStep; + +export interface JourneyDef { + /** Journey name; step tools derive from it ("document-trip-search-places"). */ + name: string; + /** The one sentence every step repeats to the agent, so the goal survives. */ + goal: string; + steps: Record; + submit: { + /** What the human confirms, e.g. "Create the trip and open the editor." */ + description: string; + /** Assemble the real tool's input from the draft. This is your code. */ + build: (draft: Readonly) => unknown; + /** The existing tool's execute - your real endpoint runs here. */ + run: (input: never, signal?: AbortSignal) => Promise; + }; +} + +function isToolStep(step: JourneyStep): step is ToolStep { + return "tool" in step; +} + +/** Chrome's published budget for one tool description. */ +const TOOL_DESCRIPTION_MAX = 500; + +/** The ASCII cut marker; fitText reserves its length before slicing. */ +const ELLIPSIS = "..."; + +/** Fit machine-composed text to a budget, reserving room for the ellipsis. */ +function fitText(text: string, budget: number): string { + if (text.length <= budget) return text; + const slice = text.slice(0, Math.max(ELLIPSIS.length, budget - ELLIPSIS.length)); + const wordEnd = slice.lastIndexOf(" "); + return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}${ELLIPSIS}`; +} + +/** + * Whether calling this step can change anything outside the draft. A step + * that composes a generated read tool inherits the tool's readOnlyHint; a + * free step that only stores input is a read; everything else defaults to + * "not read-only", because a step's `call`/`run` is user code we cannot + * inspect. We never advertise a write as safe just because it is a step. + */ +function stepReadOnly(step: JourneyStep): boolean { + if (step.readOnly !== undefined) return step.readOnly; + if (isToolStep(step)) return step.tool.annotations?.readOnlyHint === true; + return step.run === undefined; +} + +/** "document-trip-search-places" -> "Document Trip Search Places" (native UIs). */ +function toTitle(kebab: string): string { + return kebab + .split("-") + .filter(Boolean) + .map((part) => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); +} + +export function createJourney(def: JourneyDef) { + /** The shared draft. Page-scoped on purpose: reloads start clean. */ + let draft: Json = {}; + + /** Every (step, field) pair the draft still lacks, as readable text. */ + function missing(): string[] { + return Object.entries(def.steps).flatMap(([key, step]) => + step.provides + .filter((field) => draft[field] === undefined) + .map((field) => `${def.name}-${key} (stores "${field}")`), + ); + } + + function stepDescription(step: JourneyStep): string { + const base = + step.description ?? (isToolStep(step) ? step.tool.description : undefined) ?? "Journey step."; + const suffix = ` Part of "${def.name}": ${def.goal}`; + if (base.length + suffix.length <= TOOL_DESCRIPTION_MAX) return base + suffix; + // The step's own sentence matters more than repeating the whole goal. + // Keep the journey name (the grouping signal), drop the goal, and fit the + // base so the composed text never exceeds the budget. + const tag = ` Part of "${def.name}".`; + return `${fitText(base, TOOL_DESCRIPTION_MAX - tag.length)}${tag}`; + } + + function stepInput(step: JourneyStep): Json { + if (step.input) return step.input; + if (isToolStep(step) && step.tool.inputSchema) return step.tool.inputSchema; + return { type: "object", properties: {} }; + } + + /** Run one step and store what it produced. */ + async function runStep(step: JourneyStep, input: Json, signal?: AbortSignal): Promise { + if (isToolStep(step)) { + const result = await step.call(input, signal, { ...draft }); + Object.assign(draft, step.store(result)); + return; + } + const stored = step.run ? await step.run(input, signal, { ...draft }) : input; + Object.assign(draft, stored); + } + + async function registerSteps(signal?: AbortSignal): Promise { + // Resolve the context here, not at createJourney() time: a browser or + // polyfill that installs WebMCP after this module loads must still get the + // journey registered. Generated tools look it up the same way. + const modelContext = getModelContext(); + if (!modelContext) return; + for (const [key, step] of Object.entries(def.steps)) { + await modelContext.registerTool( + { + name: `${def.name}-${key}`, + title: toTitle(`${def.name}-${key}`), + description: stepDescription(step), + inputSchema: stepInput(step), + annotations: { readOnlyHint: stepReadOnly(step) }, + execute: async (input, context) => { + try { + await runStep(step, input as Json, context?.signal); + const left = missing(); + return toolResult( + left.length === 0 + ? `Stored. The journey is ready - call ${def.name}-submit.` + : `Stored. Still needed: ${left.join(", ")}.`, + ); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + } + + async function registerSubmit(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); + if (!modelContext) return; + await modelContext.registerTool( + { + name: `${def.name}-submit`, + title: toTitle(`${def.name}-submit`), + description: def.submit.description, + inputSchema: { type: "object", properties: {} }, + annotations: { readOnlyHint: false, consequentialHint: true }, + execute: async (_input, context) => { + context?.signal?.throwIfAborted(); + const left = missing(); + if (left.length > 0) { + return toolError(`Not ready to submit. Call these first: ${left.join(", ")}.`); + } + const confirmed = await requestUserConfirmation( + `Allow the agent to: ${def.submit.description}`, + ); + if (!confirmed) return toolError("The user declined this action."); + try { + const result = await def.submit.run(def.submit.build(draft) as never, context?.signal); + draft = {}; // a submitted journey starts clean + return result as WebMcpToolResult; + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + + return { + /** Call once on page load, next to registerAllTools(). */ + async register(signal?: AbortSignal): Promise { + await registerSteps(signal); + await registerSubmit(signal); + }, + /** What's in the draft right now - for the dashboard and for tests. */ + inspectDraft: (): Json => ({ ...draft }), + }; +} diff --git a/examples/openapi-petstore/src/webmcp/list-pets.webmcp.ts b/examples/openapi-petstore/src/webmcp/list-pets.webmcp.ts index 470b9a9..9568cdf 100644 --- a/examples/openapi-petstore/src/webmcp/list-pets.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/list-pets.webmcp.ts @@ -1,6 +1,6 @@ import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * List all pets in the store. Returns an array of pets. * @@ -34,14 +34,23 @@ export const listPetsHints = {"readOnlyHint":true,"destructiveHint":false,"idemp /** The tool definition, minus `execute` (which is yours, below the marker). */ export const listPetsTool = { name: "list-pets", + title: "List Pets", description: "List all pets in the store. Returns an array of pets.", inputSchema: listPetsInputSchema, annotations: { readOnlyHint: true, untrustedContentHint: true, + consequentialHint: false, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeListPets is the agent-facing one. */ +export async function fetchListPets(input: ListPetsInput, signal?: AbortSignal) { + const data = await callApi("/pets", { method: "GET", query: { status: input.status }, signal }); + return data; +} + /** * Register this tool with WebMCP. Call it once on page load, or use * registerAllTools() from the generated index.ts. Skips quietly when the @@ -69,7 +78,7 @@ export async function registerListPets(signal?: AbortSignal): Promise { ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "list-pets". diff --git a/examples/openapi-petstore/src/webmcp/runtime.webmcp.ts b/examples/openapi-petstore/src/webmcp/runtime.webmcp.ts index ae9fb54..01067b1 100644 --- a/examples/openapi-petstore/src/webmcp/runtime.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/runtime.webmcp.ts @@ -13,10 +13,16 @@ export interface WebMcpToolResult { /** A tool as the browser runtime understands it. */ export interface WebMcpToolDefinition { name: string; + /** A human-facing label for native UIs (the spec's USVString title). */ + title?: string; description: string; inputSchema?: Record; /** Hints the agent reads to decide how careful to be with this tool. */ - annotations?: { readOnlyHint?: boolean; untrustedContentHint?: boolean }; + annotations?: { + readOnlyHint?: boolean; + untrustedContentHint?: boolean; + consequentialHint?: boolean; + }; execute: ( input: Record, context?: { signal?: AbortSignal }, @@ -27,7 +33,7 @@ export interface WebMcpToolDefinition { export interface ModelContext { registerTool( tool: WebMcpToolDefinition, - options?: { signal?: AbortSignal }, + options?: { signal?: AbortSignal; exposedTo?: string[] }, ): Promise; } @@ -52,6 +58,30 @@ export function getModelContext(): ModelContext | null { return modelContext ?? null; } +/** + * Register every journey exported from the modules the barrel found in + * journeys/. Anything with a .register() method counts (createJourney's + * return shape); anything else is skipped quietly. One journey failing never + * takes the others down with it. + */ +export async function registerJourneys( + modules: Record[], + signal?: AbortSignal, +): Promise { + for (const module of modules) { + for (const value of Object.values(module)) { + const journey = value as { register?: unknown } | null; + if (journey !== null && typeof journey === "object" && typeof journey.register === "function") { + try { + await (journey.register as (signal?: AbortSignal) => Promise)(signal); + } catch (error) { + console.warn("[webmcp-codegen] a journey failed to register:", error); + } + } + } + } +} + /** * Call your API from the page. Same origin by default (pass a full URL when * the API lives on another host), always with the signed-in user's session @@ -91,12 +121,27 @@ export async function callApi( } } -/** Wrap a result in the MCP shape, so tool bodies stay one line. */ +/** Chrome's output budget: one tool result stays under ~1.5K characters. */ +const TOOL_OUTPUT_MAX = 1536; + +const TRUNCATED_NOTICE = + "\n... [truncated to fit the 1.5K output budget - return a smaller slice or paginate]"; + +/** + * Wrap a result in the MCP shape, so tool bodies stay one line. The result + * text is capped at Chrome's ~1.5K per-call output budget: oversized payloads + * cost the agent context and can trip guardrails, so they are cut with a + * notice rather than delivered whole. The cap lives here in the shared + * runtime, so it cannot be edited away per tool. + */ export function toolResult(data: unknown): WebMcpToolResult { + const text = typeof data === "string" ? data : JSON.stringify(data, null, 2); + const fitted = + text.length <= TOOL_OUTPUT_MAX + ? text + : text.slice(0, TOOL_OUTPUT_MAX - TRUNCATED_NOTICE.length) + TRUNCATED_NOTICE; return { - content: [ - { type: "text", text: typeof data === "string" ? data : JSON.stringify(data, null, 2) }, - ], + content: [{ type: "text", text: fitted }], }; } diff --git a/packages/codegen/README.md b/packages/codegen/README.md index fd87667..a300415 100644 --- a/packages/codegen/README.md +++ b/packages/codegen/README.md @@ -72,7 +72,7 @@ Working with a literal `` instead? The `form` output annotates it in place ## Safety is part of generation -This is not a dumb API → WebMCP converter. Giving agents access to application actions is a new security surface, so the generator analyzes what every endpoint actually is: read-only, write, destructive, auth-boundary, or sensitive/PII-related. Every tool gets a safety classification and WebMCP hints, the audit pass runs inside `generate` (errors block, exit codes for CI), and higher-risk tools are generated disabled so you explicitly decide what agents can touch. The goal is that you stay in control of the agent-facing surface instead of blindly exposing every endpoint. +This is not a dumb API -> WebMCP converter. Giving agents access to application actions is a new security surface, so the generator analyzes what every endpoint actually is: read-only, write, destructive, auth-boundary, or sensitive/PII-related. Every tool gets a safety classification and WebMCP hints, the audit pass runs inside `generate` (errors block, exit codes for CI), and higher-risk tools are generated disabled so you explicitly decide what agents can touch. The goal is that you stay in control of the agent-facing surface instead of blindly exposing every endpoint. ## The dashboard @@ -87,13 +87,13 @@ A local control panel for your WebMCP surface, the way Scalar is for APIs or Sto One file per endpoint, like `delete-pet.webmcp.ts`: ```ts -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- export const deletePetInputSchema = { /* derived from your spec */ }; export type DeletePetInput = { id: string }; export async function registerDeletePet(signal?: AbortSignal) { // Registers the tool; mutations ask the user to confirm, always. } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- export async function executeDeletePet(input: DeletePetInput) { // This tool starts disabled: it changes things. To enable it, delete the @@ -148,8 +148,8 @@ export default defineConfig({ ## Requirements -- Node.js ≥ 20 -- To *use* the generated tools in a browser: enable `chrome://flags/#enable-webmcp-testing` for local development (Chrome 149+, Edge 150+); production pages join the WebMCP origin trial — or use the WebMCP polyfill +- Node.js 20 or newer +- To *use* the generated tools in a browser: enable `chrome://flags/#enable-webmcp-testing` for local development (Chrome 149+, Edge 150+); production pages join the WebMCP origin trial - or use the WebMCP polyfill ## License diff --git a/packages/codegen/assets/journey.webmcp.ts b/packages/codegen/assets/journey.webmcp.ts index 718e25f..a67ca24 100644 --- a/packages/codegen/assets/journey.webmcp.ts +++ b/packages/codegen/assets/journey.webmcp.ts @@ -1,23 +1,23 @@ /** * Written by webmcp-codegen on every `generate` run. Do not edit by hand; - * your changes will be lost. This file is fully ours — journey definitions + * your changes will be lost. This file is fully ours - journey definitions * (your code) live in journeys/*.webmcp.ts and import createJourney from here. * * createJourney: multi-step agent flows with a shared draft and one submit. * * The three pieces, literally: * - * 1. THE DRAFT — one plain object per page load (`let draft = {}` below). + * 1. THE DRAFT - one plain object per page load (`let draft = {}` below). * Nothing fancier: steps write their results into it, the submit reads * from it. It dies with the page; a half-finished journey does not * survive a reload, which is what you want. * - * 2. STEP TOOLS — ordinary registered WebMCP tools, one per step, named + * 2. STEP TOOLS - ordinary registered WebMCP tools, one per step, named * "-" (e.g. "document-trip-search-places"). A step's * execute stores what it produced into the draft, then replies with what * is still missing, so the agent always knows the next move. * - * 3. THE SUBMIT GATE — one more registered tool, "-submit". Its + * 3. THE SUBMIT GATE - one more registered tool, "-submit". Its * execute, in order: refuses with the list of missing steps, asks the * human to confirm, runs the real tool's execute with the assembled * input, clears the draft. There is no way to submit around it, because @@ -37,7 +37,7 @@ type Json = Record; /** * A step backed by an existing generated tool. The step inherits the tool's - * description and input schema — the definition lives in one place — and + * description and input schema - the definition lives in one place - and * calls the raw caller the generated file exports (fetchGetAutocomplete, * not the agent-facing execute wrapper). You write only what's new: which * slice of the result lands in the draft. @@ -73,7 +73,7 @@ export interface ToolStep { } /** - * A step with no backend call of its own — it collects input into the draft + * A step with no backend call of its own - it collects input into the draft * ("set the title and dates"). With a `run`, it can do work first; whatever * `run` returns is stored. Without one, the input is stored verbatim. */ @@ -102,7 +102,7 @@ export interface JourneyDef { description: string; /** Assemble the real tool's input from the draft. This is your code. */ build: (draft: Readonly) => unknown; - /** The existing tool's execute — your real endpoint runs here. */ + /** The existing tool's execute - your real endpoint runs here. */ run: (input: never, signal?: AbortSignal) => Promise; }; } @@ -114,12 +114,15 @@ function isToolStep(step: JourneyStep): step is ToolStep { /** Chrome's published budget for one tool description. */ const TOOL_DESCRIPTION_MAX = 500; +/** The ASCII cut marker; fitText reserves its length before slicing. */ +const ELLIPSIS = "..."; + /** Fit machine-composed text to a budget, reserving room for the ellipsis. */ function fitText(text: string, budget: number): string { if (text.length <= budget) return text; - const slice = text.slice(0, Math.max(1, budget - 1)); + const slice = text.slice(0, Math.max(ELLIPSIS.length, budget - ELLIPSIS.length)); const wordEnd = slice.lastIndexOf(" "); - return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}…`; + return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}${ELLIPSIS}`; } /** @@ -135,7 +138,7 @@ function stepReadOnly(step: JourneyStep): boolean { return step.run === undefined; } -/** "document-trip-search-places" → "Document Trip Search Places" (native UIs). */ +/** "document-trip-search-places" -> "Document Trip Search Places" (native UIs). */ function toTitle(kebab: string): string { return kebab .split("-") @@ -206,7 +209,7 @@ export function createJourney(def: JourneyDef) { const left = missing(); return toolResult( left.length === 0 - ? `Stored. The journey is ready — call ${def.name}-submit.` + ? `Stored. The journey is ready - call ${def.name}-submit.` : `Stored. Still needed: ${left.join(", ")}.`, ); } catch (error) { @@ -258,7 +261,7 @@ export function createJourney(def: JourneyDef) { await registerSteps(signal); await registerSubmit(signal); }, - /** What's in the draft right now — for the dashboard and for tests. */ + /** What's in the draft right now - for the dashboard and for tests. */ inspectDraft: (): Json => ({ ...draft }), }; } diff --git a/packages/codegen/assets/skill/SKILL.md b/packages/codegen/assets/skill/SKILL.md index 3edf9c1..356ca92 100644 --- a/packages/codegen/assets/skill/SKILL.md +++ b/packages/codegen/assets/skill/SKILL.md @@ -1,10 +1,10 @@ --- name: webmcp-tools -description: Build, improve, or review this repo's WebMCP tools and journeys — the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. +description: Build, improve, or review this repo's WebMCP tools and journeys - the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. --- # WebMCP tools in this repo @@ -12,7 +12,7 @@ description: Build, improve, or review this repo's WebMCP tools and journeys — WebMCP tools run in the visitor's browser, with the signed-in user's session, while the agent calling them may also be reading attacker-influenced page content. Every tool you write is both an API and a security surface. The rules -below exist so an agent-facing tool is correct by default — follow them even +below exist so an agent-facing tool is correct by default - follow them even when the user's request is casual. ## Naming @@ -27,7 +27,7 @@ when the user's request is casual. down in the repo, ask the user before naming intent-level tools or journeys. Wrong-tense or wrong-role names pass every mechanical check and are still wrong: on a journal for trips you've *been on*, the flow is `document-trip`, - never `plan-trip`. This is the failure no linter can catch — it is your job. + never `plan-trip`. This is the failure no linter can catch - it is your job. ## Descriptions @@ -36,7 +36,7 @@ when the user's request is casual. - Max 500 characters for the tool, max 150 per parameter. Turn constraints into sentences: "A number from 30 to 600." - Never instruct the agent or encode flow control in a description - ("always call X first") — that is steering. Prerequisites belong in a + ("always call X first") - that is steering. Prerequisites belong in a journey, not in prose. - If a field's value can only come from another tool (a resolved place object, a server id), say so in that field's description. @@ -44,12 +44,12 @@ when the user's request is casual. ## Safety and exposure - Reads are registered immediately. Writes and destructive tools stay - withheld — generated but not registered — until the user deliberately + withheld - generated but not registered - until the user deliberately enables one. Never enable a write tool without being asked. - Mutating tools confirm each call with the human via `requestUserConfirmation`. That call lives in the generated region; never move or remove it. -- Free-text outputs get `untrustedContentHint: true` — the agent must not +- Free-text outputs get `untrustedContentHint: true` - the agent must not treat user-written content as the site speaking. - **The schema is not the security boundary.** `execute` must call the app's real endpoint or action layer, so server-side validation runs on every @@ -62,7 +62,7 @@ when the user's request is casual. `UnknownError` and discards your message. Return `toolError(message)` / `asToolError(error)` so the agent can read and recover. (Cancellation is the one exception: let `AbortError` propagate.) -- Return via `toolResult(data)` and keep outputs under ~1.5K characters — +- Return via `toolResult(data)` and keep outputs under ~1.5K characters - summarize or paginate rather than dumping. - When a call changes what is on screen, make it visible: navigate, invalidate a query, dispatch an event. The human is watching the page. @@ -70,7 +70,7 @@ when the user's request is casual. ## Editing generated files - Each `*.webmcp.ts` has a generated region between the - `webmcp-codegen` markers — never edit inside it; regeneration rewrites it. + `webmcp-codegen` markers - never edit inside it; regeneration rewrites it. Your work goes below the marker (the `execute` body) or in `.webmcp-codegen.json` (description, enabled, and field-text overrides, which survive regeneration and always win over generated text). @@ -104,7 +104,7 @@ export const documentTrip = createJourney({ store: (places) => ({ locationObject: places }), // store the resolved pick provides: ["locationObject"], }, - // Freeform step: no backend call — collects input straight into the draft. + // Freeform step: no backend call - collects input straight into the draft. "set-details": { description: "Set the trip's title and dates.", input: { type: "object", properties: { title: { type: "string" } }, required: ["title"] }, @@ -119,11 +119,11 @@ export const documentTrip = createJourney({ }); ``` -- 2–5 steps. More means two journeys. +- 2-5 steps. More means two journeys. - Tool-backed steps reuse the generated tool's contract and its raw caller (`fetchX`); the submit's `run` is the real write tool's `execute`, so its confirmation and validation still apply. Never write a direct `fetch` in a - journey file — `verify` flags it. + journey file - `verify` flags it. - Keep real writes in the submit gate. A tool-backed step inherits the composed tool's read-only hint, and a free step that only stores input is read-only too. Never route a mutating call through a step: it would be advertised as diff --git a/packages/codegen/evals/skill/README.md b/packages/codegen/evals/skill/README.md index b5419fa..19df1d0 100644 --- a/packages/codegen/evals/skill/README.md +++ b/packages/codegen/evals/skill/README.md @@ -4,22 +4,22 @@ The skill file (`assets/skill/SKILL.md`) is treated like code: a wording change that makes agents write worse WebMCP tools should show up as a failed test, not a vibe. This directory is that test rig, following the OpenAI eval-skills writeup and philschmid's testing-skills post: -**prompt → captured run → checks → score, over multiple trials.** +**prompt -> captured run -> checks -> score, over multiple trials.** ## Layout -- `fixture/` — beenthere-lite, a real generated surface (place search, trip +- `fixture/` - beenthere-lite, a real generated surface (place search, trip list, withheld trip creation) with the skill file installed at `.agents/skills/webmcp-tools/`. Every case starts from a fresh copy. -- `cases.json` — the prompt set. `core` cases gate; `negative` cases prove +- `cases.json` - the prompt set. `core` cases gate; `negative` cases prove the skill doesn't leak into unrelated work; `control` cases are informational only. -- `run.mjs` — the harness and its deterministic graders. -- `results/` — timestamped JSON reports (gitignored). +- `run.mjs` - the harness and its deterministic graders. +- `results/` - timestamped JSON reports (gitignored). ## Running -You need an agent CLI on PATH (Codex, Claude Code, …). The command template +You need an agent CLI on PATH (Codex, Claude Code, ...). The command template is `AGENT_CMD`, with `{PROMPT}` substituted and shell-quoted: ```sh @@ -38,7 +38,7 @@ never gate. a *retrospective* (been-there) product. The right answer composes the generated tools into `createJourney(...)`, names it like `document-trip`, and gates creation behind the human. `plan-trip` / `book-trip` match a -forbidden pattern — that naming failure passes every mechanical lint yet is +forbidden pattern - that naming failure passes every mechanical lint yet is semantically wrong, which is exactly the class of mistake the skill file exists to prevent. @@ -52,13 +52,13 @@ and the skill can retire. Graders look at files, not transcripts: expected files satisfying include/exclude regexes, the generated region's marker preserved byte-for-byte, description budgets measured, the tree unchanged on unrelated prompts. Adding -an LLM-as-judge pass is possible later — constrain it to a structured schema -(`overall_pass`, per-check results) so scores diff across runs — but the +an LLM-as-judge pass is possible later - constrain it to a structured schema +(`overall_pass`, per-check results) so scores diff across runs - but the fixtures were chosen so regex + structure carry the verdict. ## The operating loop -1. A real failure — from a user report, a docs change, a model regression — +1. A real failure - from a user report, a docs change, a model regression - becomes a case here first. 2. Tune the skill until the case is at ~100% pass rate across trials. 3. The case joins the regression set permanently. diff --git a/packages/codegen/evals/skill/cases.json b/packages/codegen/evals/skill/cases.json index b198ba4..54b74ec 100644 --- a/packages/codegen/evals/skill/cases.json +++ b/packages/codegen/evals/skill/cases.json @@ -52,7 +52,7 @@ { "id": "improve-description", "kind": "core", - "comment": "Descriptions live in the generated region (do-not-edit) or the overrides file — the skill teaches overrides, so the marker must survive either way.", + "comment": "Descriptions live in the generated region (do-not-edit) or the overrides file - the skill teaches overrides, so the marker must survive either way.", "prompt": "The create-trip description doesn't tell agents what comes back. Improve it.", "checks": [ { diff --git a/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md b/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md index 3edf9c1..356ca92 100644 --- a/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md +++ b/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md @@ -1,10 +1,10 @@ --- name: webmcp-tools -description: Build, improve, or review this repo's WebMCP tools and journeys — the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. +description: Build, improve, or review this repo's WebMCP tools and journeys - the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. --- # WebMCP tools in this repo @@ -12,7 +12,7 @@ description: Build, improve, or review this repo's WebMCP tools and journeys — WebMCP tools run in the visitor's browser, with the signed-in user's session, while the agent calling them may also be reading attacker-influenced page content. Every tool you write is both an API and a security surface. The rules -below exist so an agent-facing tool is correct by default — follow them even +below exist so an agent-facing tool is correct by default - follow them even when the user's request is casual. ## Naming @@ -27,7 +27,7 @@ when the user's request is casual. down in the repo, ask the user before naming intent-level tools or journeys. Wrong-tense or wrong-role names pass every mechanical check and are still wrong: on a journal for trips you've *been on*, the flow is `document-trip`, - never `plan-trip`. This is the failure no linter can catch — it is your job. + never `plan-trip`. This is the failure no linter can catch - it is your job. ## Descriptions @@ -36,7 +36,7 @@ when the user's request is casual. - Max 500 characters for the tool, max 150 per parameter. Turn constraints into sentences: "A number from 30 to 600." - Never instruct the agent or encode flow control in a description - ("always call X first") — that is steering. Prerequisites belong in a + ("always call X first") - that is steering. Prerequisites belong in a journey, not in prose. - If a field's value can only come from another tool (a resolved place object, a server id), say so in that field's description. @@ -44,12 +44,12 @@ when the user's request is casual. ## Safety and exposure - Reads are registered immediately. Writes and destructive tools stay - withheld — generated but not registered — until the user deliberately + withheld - generated but not registered - until the user deliberately enables one. Never enable a write tool without being asked. - Mutating tools confirm each call with the human via `requestUserConfirmation`. That call lives in the generated region; never move or remove it. -- Free-text outputs get `untrustedContentHint: true` — the agent must not +- Free-text outputs get `untrustedContentHint: true` - the agent must not treat user-written content as the site speaking. - **The schema is not the security boundary.** `execute` must call the app's real endpoint or action layer, so server-side validation runs on every @@ -62,7 +62,7 @@ when the user's request is casual. `UnknownError` and discards your message. Return `toolError(message)` / `asToolError(error)` so the agent can read and recover. (Cancellation is the one exception: let `AbortError` propagate.) -- Return via `toolResult(data)` and keep outputs under ~1.5K characters — +- Return via `toolResult(data)` and keep outputs under ~1.5K characters - summarize or paginate rather than dumping. - When a call changes what is on screen, make it visible: navigate, invalidate a query, dispatch an event. The human is watching the page. @@ -70,7 +70,7 @@ when the user's request is casual. ## Editing generated files - Each `*.webmcp.ts` has a generated region between the - `webmcp-codegen` markers — never edit inside it; regeneration rewrites it. + `webmcp-codegen` markers - never edit inside it; regeneration rewrites it. Your work goes below the marker (the `execute` body) or in `.webmcp-codegen.json` (description, enabled, and field-text overrides, which survive regeneration and always win over generated text). @@ -104,7 +104,7 @@ export const documentTrip = createJourney({ store: (places) => ({ locationObject: places }), // store the resolved pick provides: ["locationObject"], }, - // Freeform step: no backend call — collects input straight into the draft. + // Freeform step: no backend call - collects input straight into the draft. "set-details": { description: "Set the trip's title and dates.", input: { type: "object", properties: { title: { type: "string" } }, required: ["title"] }, @@ -119,11 +119,11 @@ export const documentTrip = createJourney({ }); ``` -- 2–5 steps. More means two journeys. +- 2-5 steps. More means two journeys. - Tool-backed steps reuse the generated tool's contract and its raw caller (`fetchX`); the submit's `run` is the real write tool's `execute`, so its confirmation and validation still apply. Never write a direct `fetch` in a - journey file — `verify` flags it. + journey file - `verify` flags it. - Keep real writes in the submit gate. A tool-backed step inherits the composed tool's read-only hint, and a free step that only stores input is read-only too. Never route a mutating call through a step: it would be advertised as diff --git a/packages/codegen/evals/skill/fixture/README.md b/packages/codegen/evals/skill/fixture/README.md index 868ca24..d13cb69 100644 --- a/packages/codegen/evals/skill/fixture/README.md +++ b/packages/codegen/evals/skill/fixture/README.md @@ -1,6 +1,6 @@ # beenthere-lite (eval fixture) -A memory-journal app for trips you've *been on* — deliberately NOT a travel +A memory-journal app for trips you've *been on* - deliberately NOT a travel planner. Generated by `@webmcp-stack/codegen generate --spec spec.json` and committed as the baseline every eval case starts from. diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts index 7811276..8c10748 100644 --- a/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts +++ b/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts @@ -1,6 +1,6 @@ import { callApi, toolDisabled } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * Create a new trip. Returns the trip. * @@ -141,7 +141,7 @@ export async function registerCreateTrip(signal?: AbortSignal): Promise { // ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "create-trip". diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts index f2f7df7..94bac69 100644 --- a/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts +++ b/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts @@ -1,6 +1,6 @@ import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * Search places by free text. Returns an array of autocomplete. * @@ -76,7 +76,7 @@ export async function registerGetAutocomplete(signal?: AbortSignal): Promise-" (e.g. "document-trip-search-places"). A step's * execute stores what it produced into the draft, then replies with what * is still missing, so the agent always knows the next move. * - * 3. THE SUBMIT GATE — one more registered tool, "-submit". Its + * 3. THE SUBMIT GATE - one more registered tool, "-submit". Its * execute, in order: refuses with the list of missing steps, asks the * human to confirm, runs the real tool's execute with the assembled * input, clears the draft. There is no way to submit around it, because @@ -37,7 +37,7 @@ type Json = Record; /** * A step backed by an existing generated tool. The step inherits the tool's - * description and input schema — the definition lives in one place — and + * description and input schema - the definition lives in one place - and * calls the raw caller the generated file exports (fetchGetAutocomplete, * not the agent-facing execute wrapper). You write only what's new: which * slice of the result lands in the draft. @@ -73,7 +73,7 @@ export interface ToolStep { } /** - * A step with no backend call of its own — it collects input into the draft + * A step with no backend call of its own - it collects input into the draft * ("set the title and dates"). With a `run`, it can do work first; whatever * `run` returns is stored. Without one, the input is stored verbatim. */ @@ -102,7 +102,7 @@ export interface JourneyDef { description: string; /** Assemble the real tool's input from the draft. This is your code. */ build: (draft: Readonly) => unknown; - /** The existing tool's execute — your real endpoint runs here. */ + /** The existing tool's execute - your real endpoint runs here. */ run: (input: never, signal?: AbortSignal) => Promise; }; } @@ -114,12 +114,15 @@ function isToolStep(step: JourneyStep): step is ToolStep { /** Chrome's published budget for one tool description. */ const TOOL_DESCRIPTION_MAX = 500; +/** The ASCII cut marker; fitText reserves its length before slicing. */ +const ELLIPSIS = "..."; + /** Fit machine-composed text to a budget, reserving room for the ellipsis. */ function fitText(text: string, budget: number): string { if (text.length <= budget) return text; - const slice = text.slice(0, Math.max(1, budget - 1)); + const slice = text.slice(0, Math.max(ELLIPSIS.length, budget - ELLIPSIS.length)); const wordEnd = slice.lastIndexOf(" "); - return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}…`; + return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}${ELLIPSIS}`; } /** @@ -135,7 +138,7 @@ function stepReadOnly(step: JourneyStep): boolean { return step.run === undefined; } -/** "document-trip-search-places" → "Document Trip Search Places" (native UIs). */ +/** "document-trip-search-places" -> "Document Trip Search Places" (native UIs). */ function toTitle(kebab: string): string { return kebab .split("-") @@ -206,7 +209,7 @@ export function createJourney(def: JourneyDef) { const left = missing(); return toolResult( left.length === 0 - ? `Stored. The journey is ready — call ${def.name}-submit.` + ? `Stored. The journey is ready - call ${def.name}-submit.` : `Stored. Still needed: ${left.join(", ")}.`, ); } catch (error) { @@ -258,7 +261,7 @@ export function createJourney(def: JourneyDef) { await registerSteps(signal); await registerSubmit(signal); }, - /** What's in the draft right now — for the dashboard and for tests. */ + /** What's in the draft right now - for the dashboard and for tests. */ inspectDraft: (): Json => ({ ...draft }), }; } diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts index 1092fe2..07c5ea5 100644 --- a/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts +++ b/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts @@ -1,6 +1,6 @@ import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * List the signed-in user's trips. Returns an array of trips. * @@ -69,7 +69,7 @@ export async function registerListTrips(signal?: AbortSignal): Promise { ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "list-trips". diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts index 9939c09..01067b1 100644 --- a/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts +++ b/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts @@ -125,7 +125,7 @@ export async function callApi( const TOOL_OUTPUT_MAX = 1536; const TRUNCATED_NOTICE = - "\n… [truncated to fit the 1.5K output budget — return a smaller slice or paginate]"; + "\n... [truncated to fit the 1.5K output budget - return a smaller slice or paginate]"; /** * Wrap a result in the MCP shape, so tool bodies stay one line. The result diff --git a/packages/codegen/evals/skill/run.mjs b/packages/codegen/evals/skill/run.mjs index faf4745..29f0409 100644 --- a/packages/codegen/evals/skill/run.mjs +++ b/packages/codegen/evals/skill/run.mjs @@ -1,10 +1,10 @@ #!/usr/bin/env node /** - * The skill-file eval runner: prompt → captured run → checks → score. + * The skill-file eval runner: prompt -> captured run -> checks -> score. * * Each case gets a fresh copy of fixture/ (plus the skill file, unless the * case removes it), an agent runs the case's prompt inside it headlessly, - * and deterministic checks grade what the agent left behind — files, not + * and deterministic checks grade what the agent left behind - files, not * transcripts, wherever possible. Behavior is nondeterministic: run trials * and report pass RATES, never a single run's verdict. * @@ -30,7 +30,7 @@ import { fileURLToPath } from "node:url"; const here = dirname(fileURLToPath(import.meta.url)); const FIXTURE = join(here, "fixture"); const GENERATED_END = - "// ─── webmcp-codegen: end generated. Your code below survives regeneration. ───"; + "// --- webmcp-codegen: end generated. Your code below survives regeneration. ---"; // ---------------------------------------------------------------- graders // Every grader: async (ctx) => null on pass, or a sentence on failure. @@ -95,7 +95,7 @@ const graders = { }; return headOf(after) === headOf(before) ? null - : `${check.path}: the generated region was edited — regeneration will clobber it; descriptions and names move through .webmcp-codegen.json`; + : `${check.path}: the generated region was edited - regeneration will clobber it; descriptions and names move through .webmcp-codegen.json`; }, /** Nothing under src/webmcp may differ from the fixture (negative cases). */ @@ -226,7 +226,7 @@ async function runCase(caseDef, command, trials, report) { results, }); console.log( - ` ${gate === true ? "✓" : gate === "informational" ? "ℹ" : "✖"} ${caseDef.id}: ${passed}/${results.length} passed${caseDef.kind === "control" ? " (control)" : ""}`, + ` ${gate === true ? "ok" : gate === "informational" ? "i" : "x"} ${caseDef.id}: ${passed}/${results.length} passed${caseDef.kind === "control" ? " (control)" : ""}`, ); for (const result of results.filter((r) => !r.pass)) { for (const line of result.failures) console.log(` trial ${result.trial + 1}: ${line}`); @@ -252,7 +252,7 @@ async function selftest() { }); const gotFail = failure !== null; console.log( - ` ${gotFail === expectFail ? "✓" : "✖"} ${label}${gotFail && expectFail ? ` (${failure})` : ""}`, + ` ${gotFail === expectFail ? "ok" : "x"} ${label}${gotFail && expectFail ? ` (${failure})` : ""}`, ); await rm(dir, { recursive: true, force: true }); return gotFail === expectFail; @@ -384,7 +384,7 @@ async function main() { const gated = report.cases.filter((c) => c.kind !== "control"); const allPassed = gated.every((c) => c.gate === true); console.log( - `\n ${gated.filter((c) => c.gate === true).length}/${gated.length} gated cases fully passing — report: ${relative(process.cwd(), outPath)}`, + `\n ${gated.filter((c) => c.gate === true).length}/${gated.length} gated cases fully passing - report: ${relative(process.cwd(), outPath)}`, ); process.exit(allPassed ? 0 : 1); } diff --git a/packages/codegen/src/cli-output.ts b/packages/codegen/src/cli-output.ts index 15d36c6..f15fbdb 100644 --- a/packages/codegen/src/cli-output.ts +++ b/packages/codegen/src/cli-output.ts @@ -31,7 +31,7 @@ function termWidth(): number { } /** - * The banner, printed once at the start of every command — a banner leads, + * The banner, printed once at the start of every command - a banner leads, * it never sits in the middle of a report. The full wordmark in cfonts * "tiny": solid block letters, small enough that webmcp-stack fits in 51 * columns. The color is the site's accent (#58a6ff), on a real terminal @@ -153,7 +153,7 @@ function wrapNames(names: string[], indent: string): string[] { return lines; } -/** The default summary output — written for humans, not machines. */ +/** The default summary output - written for humans, not machines. */ export function renderSummary( result: GenerateResult, setup: Setup, @@ -171,7 +171,7 @@ export function renderSummary( console.log(""); // What happened - console.log(` ${c.green("✓")} ${bold(`${tools.length} tools generated`)}`); + console.log(` ${c.green("ok")} ${bold(`${tools.length} tools generated`)}`); const parts = [`${enabled} ready to use`]; if (withheld > 0) parts.push(`${withheld} withheld until you enable them`); if (gated > 0) parts.push(`${gated} visible but disabled`); @@ -182,15 +182,15 @@ export function renderSummary( console.log(""); // Blocking errors first, one line each, in red. They stop the write, and a - // red wall that says exactly what to fix beats a green ✓ that doesn't mean it. + // red wall that says exactly what to fix beats a green ok that doesn't mean it. const errorFindings = findings.filter((f) => f.level === "error"); if (errorFindings.length > 0) { console.log( - ` ${c.red("✖")} ${bold(`${errorFindings.length} error${errorFindings.length === 1 ? "" : "s"}, nothing written`)}`, + ` ${c.red("x")} ${bold(`${errorFindings.length} error${errorFindings.length === 1 ? "" : "s"}, nothing written`)}`, ); for (const f of errorFindings) { const where = f.tool ? dim(` (${f.tool})`) : ""; - console.log(` ${c.red("✖")} ${f.message}${where}`); + console.log(` ${c.red("x")} ${f.message}${where}`); } console.log(""); } @@ -206,7 +206,7 @@ export function renderSummary( for (const group of groups) { const heading = group.heading.length > maxHeading - ? `${group.heading.slice(0, maxHeading - 1)}…` + ? `${group.heading.slice(0, maxHeading - 1)}...` : group.heading; console.log(dim(` ${heading}`)); } @@ -216,9 +216,9 @@ export function renderSummary( // Where things went const outDir = setup.config.outputs[0]?.outDir ?? "src/webmcp"; - console.log(` ${c.cyan("→")} ${bold("Files")} ${outDir}`); + console.log(` ${c.cyan("->")} ${bold("Files")} ${outDir}`); if (wiring && !wiring.alreadyWired) { - console.log(` ${c.cyan("→")} ${bold("Registration")} wired into your app`); + console.log(` ${c.cyan("->")} ${bold("Registration")} wired into your app`); } // Per-file notes (kept attributes, added names, unmatched controls). Capped // in the summary; --verbose lists them all. @@ -227,15 +227,15 @@ export function renderSummary( console.log(dim(` ${note}`)); } if (fileNotes.length > 6) { - console.log(dim(` …and ${fileNotes.length - 6} more (run with --verbose)`)); + console.log(dim(` ...and ${fileNotes.length - 6} more (run with --verbose)`)); } // Pipeline proposals (groupings, renames): the run's "look at this" lines. for (const note of result.notes.slice(0, 6)) { - console.log(` ${c.cyan("◦")} ${note}`); + console.log(` ${c.cyan("-")} ${note}`); } if (result.notes.length > 6) { - console.log(dim(` …and ${result.notes.length - 6} more (run with --verbose)`)); + console.log(dim(` ...and ${result.notes.length - 6} more (run with --verbose)`)); } console.log(""); @@ -247,7 +247,7 @@ export function renderSummary( console.log(""); } -/** Verbose output — every tool, for when you want the full list. */ +/** Verbose output - every tool, for when you want the full list. */ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string): void { const { tools, findings, skipped } = result; @@ -300,14 +300,14 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string for (const tool of group) { const description = tool.description || "(no description)"; table.push([ - tool.name.length > nameWidth ? `${tool.name.slice(0, nameWidth - 1)}…` : tool.name, + tool.name.length > nameWidth ? `${tool.name.slice(0, nameWidth - 1)}...` : tool.name, tool.enabledByDefault ? c.green("enabled") : tool.withheld ? dim("withheld") : dim("disabled"), description.length > descWidth - 2 - ? `${description.slice(0, descWidth - 3)}…` + ? `${description.slice(0, descWidth - 3)}...` : description, ]); } @@ -316,7 +316,7 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string } // Safety notes: grouped by what the person should do, with the affected - // tools listed compactly under each. 60 identical ⚠ lines teach nothing; + // tools listed compactly under each. 60 identical ! lines teach nothing; // 5 headed groups do. const errorFindings = findings.filter((f) => f.level === "error"); const groups = groupFindings(findings); @@ -325,11 +325,11 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string console.log(""); for (const f of errorFindings) { const where = f.tool ? dim(` (${f.tool})`) : ""; - console.log(` ${c.red("✖")} ${f.message}${where}`); + console.log(` ${c.red("x")} ${f.message}${where}`); } if (errorFindings.length > 0) console.log(""); for (const group of groups) { - console.log(` ${c.yellow("⚠")} ${group.heading}`); + console.log(` ${c.yellow("!")} ${group.heading}`); for (const line of wrapNames(group.items, " ")) { console.log(dim(line)); } diff --git a/packages/codegen/src/cli.ts b/packages/codegen/src/cli.ts index 07a6293..3028471 100644 --- a/packages/codegen/src/cli.ts +++ b/packages/codegen/src/cli.ts @@ -55,7 +55,7 @@ import { import { applyWiring, planWiring, type WirePlan } from "./wire.js"; const HELP = ` -webmcp-codegen — generate WebMCP tools from your OpenAPI spec +webmcp-codegen - generate WebMCP tools from your OpenAPI spec Usage npx @webmcp-stack/codegen [command] [flags] @@ -313,7 +313,7 @@ async function dev(port: number): Promise { /** * The tool standard, measured locally: runs the pipeline exactly as generate * would (nothing written), then reports how the registered surface holds up - * — names, descriptions, field text, annotations, surface size. Exits 1 on + * - names, descriptions, field text, annotations, surface size. Exits 1 on * error-level findings so CI can gate on it. */ async function verify(flags: CliFlags): Promise { @@ -341,7 +341,7 @@ async function verify(flags: CliFlags): Promise { const registered = result.tools.filter((tool) => !tool.withheld); // Journey files are the user's code, so verify can't get them from the - // pipeline's tool list — it reads the journeys/ folder of each tools + // pipeline's tool list - it reads the journeys/ folder of each tools // output itself and lints what it finds there. Read them first: the surface // count needs to include the tools they register at runtime. const journeyInputs: JourneyFileInput[] = []; @@ -370,13 +370,13 @@ async function verify(flags: CliFlags): Promise { info(` ${setup.label}: ${result.tools.length} tools, ${registered.length} registered`); info(""); for (const check of checks) { - const mark = check.level === "ok" ? "✓" : check.level === "error" ? "✖" : "!"; + const mark = check.level === "ok" ? "ok" : check.level === "error" ? "x" : "!"; info(` ${mark} ${check.area}: ${check.summary}`); for (const finding of check.findings.slice(0, 8)) { info(` ${finding}`); } if (check.findings.length > 8) { - info(dim(` …and ${check.findings.length - 8} more`)); + info(dim(` ...and ${check.findings.length - 8} more`)); } } @@ -388,7 +388,7 @@ async function verify(flags: CliFlags): Promise { info(` ! ${finding.message}`); } if (pageFindings.length === 0) { - info(" ✓ Page checks passed"); + info(" ok Page checks passed"); } } @@ -425,7 +425,7 @@ async function generate(flags: CliFlags): Promise { const progress = flags.verbose ? (msg: string) => debug(msg) : undefined; // The data file carries the developer's dashboard edits and the last run's - // names. Both must reach the pipeline here — the dashboard is where edits + // names. Both must reach the pipeline here - the dashboard is where edits // are made, but this command is where files are written, and an edit that // only one of them reads does not survive. const data = await loadDataFile(cwd); @@ -441,7 +441,7 @@ async function generate(flags: CliFlags): Promise { }); if (!flags.verbose && result.tools.length > 0) { - info(` → Read ${result.tools.length + result.skipped.length} operations`); + info(` -> Read ${result.tools.length + result.skipped.length} operations`); } // Audit errors block the write. Instead of requiring --force on a re-run, diff --git a/packages/codegen/src/data-file.ts b/packages/codegen/src/data-file.ts index fb40ef1..4bd758e 100644 --- a/packages/codegen/src/data-file.ts +++ b/packages/codegen/src/data-file.ts @@ -1,7 +1,7 @@ /** * The remembered-choices file: `.webmcp-codegen.json` at the project root. * - * It is plain data — never code — so it works in the pure-npx flow (no + * It is plain data - never code - so it works in the pure-npx flow (no * install needed) and can be read and written safely by the CLI and the dev * dashboard alike. It holds two kinds of things: * diff --git a/packages/codegen/src/describe.test.ts b/packages/codegen/src/describe.test.ts index 8a0b411..d6aa95a 100644 --- a/packages/codegen/src/describe.test.ts +++ b/packages/codegen/src/describe.test.ts @@ -27,16 +27,16 @@ describe("fitBudget", () => { const text = `${"a b c d e f g h i j k l m n o p q r s t u v w x y z ".repeat(6)}end`; const fitted = fitBudget(text, 150); expect(fitted.length).toBeLessThanOrEqual(150); - expect(fitted.endsWith("…")).toBe(true); + expect(fitted.endsWith("...")).toBe(true); expect(fitted).not.toContain("end"); }); it("never exceeds the budget for a single unbroken token", () => { // The regression that started this: a URL or token with no spaces used to - // come back at budget + 1 because the ellipsis was appended after the cut. + // come back over budget because the cut marker was appended after the cut. const fitted = fitBudget("x".repeat(300), 150); expect(fitted.length).toBeLessThanOrEqual(150); - expect(fitted.endsWith("…")).toBe(true); + expect(fitted.endsWith("...")).toBe(true); }); }); diff --git a/packages/codegen/src/describe.ts b/packages/codegen/src/describe.ts index db3e241..9fbfd4f 100644 --- a/packages/codegen/src/describe.ts +++ b/packages/codegen/src/describe.ts @@ -39,6 +39,9 @@ import type { CandidateTool, JsonSchema } from "./types.js"; export const TOOL_DESCRIPTION_MAX = 500; export const FIELD_DESCRIPTION_MAX = 150; +/** The ASCII cut marker; fit functions reserve its length before slicing. */ +const ELLIPSIS = "..."; + /** * Fit machine-drafted text to a character budget. A text that fits passes * through untouched. One that overflows is cut at the last sentence boundary @@ -52,9 +55,9 @@ export const FIELD_DESCRIPTION_MAX = 150; */ export function fitBudget(text: string, budget: number): string { if (text.length <= budget) return text; - // Leave one character for the ellipsis, so the result can never be - // budget + 1 when there is no space to cut at. - const slice = text.slice(0, budget - 1); + // Reserve room for the cut marker, so the result can never come back over + // budget when there is no space to cut at. + const slice = text.slice(0, budget - ELLIPSIS.length); const sentenceEnd = Math.max( slice.lastIndexOf(". "), slice.lastIndexOf("! "), @@ -63,7 +66,7 @@ export function fitBudget(text: string, budget: number): string { if (sentenceEnd >= Math.floor(budget / 2)) return slice.slice(0, sentenceEnd + 1); const wordEnd = slice.lastIndexOf(" "); const body = wordEnd > 0 ? slice.slice(0, wordEnd) : slice; - return `${body.trimEnd()}…`; + return `${body.trimEnd()}${ELLIPSIS}`; } /** @@ -192,7 +195,7 @@ function alreadyStatesConstraints(text: string, schema: JsonSchema): boolean { return values.length > 0 && values.every((value) => text.includes(value)); } -/** "purchaseDate" / "purchase_date" / "purchase-date" → "Purchase date". +/** "purchaseDate" / "purchase_date" / "purchase-date" -> "Purchase date". * Sentence case, matching the field text in Chrome's WebMCP examples; these * are machine drafts that the audit flags, not final copy. */ function humanizeFieldName(name: string): string { @@ -230,7 +233,7 @@ function patternSentence(name: string, schema: JsonSchema, noun?: string): strin if (last === "url" && subject) return `The URL of the ${subject}.`; if (last === "at" && words.length > 1) { // The stem's last word is the event ("captured"); anything before it is - // what it happened to ("email verified at" → the email). + // what it happened to ("email verified at" -> the email). const happenedTo = words.slice(0, -2).join(" ") || noun; if (happenedTo) return `When the ${happenedTo} was ${words[words.length - 2]}.`; } @@ -243,7 +246,7 @@ function patternSentence(name: string, schema: JsonSchema, noun?: string): strin } /** - * The noun a tool acts on, from its name: "create-trip" → "trip". Our own + * The noun a tool acts on, from its name: "create-trip" -> "trip". Our own * naming rules put the verb first, so the next segment that means something * is the noun. Only a fallback subject for pattern sentences. */ @@ -434,7 +437,7 @@ const PHRASAL_VERBS = new Set(["sign-up", "sign-in", "sign-out", "log-in", "log- function returnShapeSentence(toolName: string, output: JsonSchema): string { const words = toolName.split("-"); - // The noun is what the verb leaves behind — and a phrasal verb is two words. + // The noun is what the verb leaves behind - and a phrasal verb is two words. const firstTwo = words.slice(0, 2).join("-"); const nounWords = PHRASAL_VERBS.has(firstTwo) ? words.slice(2) : words.slice(1); const nounPhrase = nounWords.join(" ").replace(/ by \w+$/, ""); diff --git a/packages/codegen/src/detect-app.ts b/packages/codegen/src/detect-app.ts index 77a9170..5a8539a 100644 --- a/packages/codegen/src/detect-app.ts +++ b/packages/codegen/src/detect-app.ts @@ -2,12 +2,12 @@ * Web-app detection: where the generated tools should live. * * The tools are browser code, so they belong in whichever package *is* the - * web app — not next to the spec, and not wherever the command happened to + * web app - not next to the spec, and not wherever the command happened to * run. In a monorepo like: * * apps/ - * ├── server/ (has the openapi.json) - * └── web/ (has next in its package.json) ← tools go here + * |-- server/ (has the openapi.json) + * `-- web/ (has next in its package.json) <- tools go here * * detection means reading package.json files and looking for a browser * framework. One candidate: we use it and say so. Several: the CLI asks diff --git a/packages/codegen/src/dev/server.ts b/packages/codegen/src/dev/server.ts index 92319a8..07c9540 100644 --- a/packages/codegen/src/dev/server.ts +++ b/packages/codegen/src/dev/server.ts @@ -8,7 +8,7 @@ * shows is what a generate run would write. * * It exists only while the command is running, listens on localhost only, - * and nothing about it ever touches the user's app bundle — by design, so + * and nothing about it ever touches the user's app bundle - by design, so * this dev tool can never leak into production. */ @@ -200,7 +200,7 @@ function toUiTool( ? `schema: ${tool.source.ref}` : ""; // The dry run already holds every file's contents in memory, so the - // dashboard can show the real generated source per tool — the same + // dashboard can show the real generated source per tool - the same // progressive disclosure the site's demo has, against live output. const file = files.find((f) => basename(f.path) === `${tool.name}.webmcp.ts`); return { @@ -233,7 +233,7 @@ function toUiTool( * The direct "run it" test: call the endpoint the way the generated * execute() would, but server-side. Two honest limitations the UI states: * there is no browser session here (auth cookies do not apply), and the - * call needs an absolute base URL — the spec's servers entry or one the + * call needs an absolute base URL - the spec's servers entry or one the * developer types in. */ async function runEndpoint( diff --git a/packages/codegen/src/dev/ui.ts b/packages/codegen/src/dev/ui.ts index 1e878d5..2db2f4e 100644 --- a/packages/codegen/src/dev/ui.ts +++ b/packages/codegen/src/dev/ui.ts @@ -74,7 +74,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean } .app.detail-open .main { transform: translateX(0); } .main .placeholder { display: none; } - /* The back control is a bare chevron — a tap target, not a button. It + /* The back control is a bare chevron - a tap target, not a button. It sits inline at the left of the title row, so no vertical space is spent on navigation. */ .back-btn { @@ -140,7 +140,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean background: var(--surface); } /* The drag handle on the sidebar's right edge. Invisible until you - hover near it, then a 2px accent line — the Vercel/Linear idiom. */ + hover near it, then a 2px accent line - the Vercel/Linear idiom. */ .sidebar-resize { position: absolute; top: 0; @@ -402,7 +402,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean } /* The per-tool source disclosure: the generated file, revealed on demand. - A quiet row that expands into the code — the dashboard is the disclosure, + A quiet row that expands into the code - the dashboard is the disclosure, not a separate view. */ .source-disclosure { margin-bottom: 28px; @@ -753,7 +753,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean ::-webkit-scrollbar-thumb:hover { background: var(--ghost); } /* Narrow-screen content density. This block comes after the base .detail - so it actually wins on source order — an earlier media query lost to the + so it actually wins on source order - an earlier media query lost to the desktop rule, which is why the padding never changed. */ @media (max-width: 640px) { .detail { padding: 14px 16px; max-width: none; } @@ -766,7 +766,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean ? ` /* Scoped mode: mounted inside a shadow root on the marketing site, where document-level selectors never match and vh/dvh would measure the page - viewport, not the host — which is exactly what clipped the demo's scroll + viewport, not the host - which is exactly what clipped the demo's scroll region before. This block comes LAST so it overrides the base rules: :host plays the body role and the app fills it, not the viewport. */ :host { @@ -784,7 +784,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean position: relative; overflow: hidden; } - /* The demo's default sidebar width — narrower than the real dashboard's + /* The demo's default sidebar width - narrower than the real dashboard's 320px, so the detail pane gets the room in the embedded frame. The drag handle sets an inline width, which still wins over this; the mobile full-width sidebar rule carries !important and is unaffected. */ @@ -815,7 +815,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean
- +
@@ -826,7 +826,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean

Select a tool to view details

- ↑ ↓ to navigate  ·  ⌘K to search + Up Down to navigate  -  CmdK to search

@@ -960,7 +960,7 @@ var EMBEDDED_STATE = ${embeddedState ? JSON.stringify(embeddedState) : "null"}; ].filter(Boolean).join(""); var findings = (tool.findings || []).map(function (finding) { - var icon = finding.level === "error" ? "✖" : "⚠"; + var icon = finding.level === "error" ? "x" : "!"; return '
' + icon + "" + esc(finding.message) + "
"; }).join(""); @@ -1050,7 +1050,7 @@ var EMBEDDED_STATE = ${embeddedState ? JSON.stringify(embeddedState) : "null"}; '' + 'Saved' + "" + - '
Agents pick tools by this text. Saved to .webmcp-codegen.json, so it survives regeneration. ⌘S to save.
' + + '
Agents pick tools by this text. Saved to .webmcp-codegen.json, so it survives regeneration. CmdS to save.
' + "" + (fieldRows @@ -1079,7 +1079,7 @@ var EMBEDDED_STATE = ${embeddedState ? JSON.stringify(embeddedState) : "null"}; '

Run this tool

server-side, no browser session
' + '
' + (tool.requiresAuth - ? '
⚠ This endpoint requires a browser session. The dashboard runs server-side, so you will get a 401. Test it in Chrome DevTools where you are signed in.
' + ? '
! This endpoint requires a browser session. The dashboard runs server-side, so you will get a 401. Test it in Chrome DevTools where you are signed in.
' : "") + '' + (fields || '
This tool takes no inputs.
') + @@ -1235,7 +1235,7 @@ var EMBEDDED_STATE = ${embeddedState ? JSON.stringify(embeddedState) : "null"}; } if ((event.metaKey || event.ctrlKey) && event.key === "s") { event.preventDefault(); - // ⌘S saves whatever is being edited: an open field row, else the + // CmdS saves whatever is being edited: an open field row, else the // tool description. if (document.getElementById("field-edit-input")) saveFieldEdit(); else saveDescription(); diff --git a/packages/codegen/src/group.ts b/packages/codegen/src/group.ts index 42f4d27..3179111 100644 --- a/packages/codegen/src/group.ts +++ b/packages/codegen/src/group.ts @@ -21,8 +21,8 @@ * * Threading (how the merged call feeds the second request from the first * response) is exact-name only: the right side's path params must each match - * a property on the left's response schema (`{uploadId}` ← `uploadId`). If a - * path param can't be threaded, the pair is skipped — guessing data flow is + * a property on the left's response schema (`{uploadId}` <- `uploadId`). If a + * path param can't be threaded, the pair is skipped - guessing data flow is * how plausible garbage gets shipped. * * The merged tool is withheld like any write: the merge is a proposal the @@ -162,7 +162,7 @@ export function groupHandshakes(candidates: CandidateTool[]): GroupResult { const threaded = threadingFor(left, right); if (threaded === undefined) { notes.push( - `${left.name} + ${right.name} look like one flow, but "${right.name}" takes path params the first response doesn't provide — left as separate tools.`, + `${left.name} + ${right.name} look like one flow, but "${right.name}" takes path params the first response doesn't provide - left as separate tools.`, ); continue; } @@ -170,7 +170,7 @@ export function groupHandshakes(candidates: CandidateTool[]): GroupResult { const name = pickMergedName(noun, resource, taken); if (!name) { notes.push( - `${left.name} + ${right.name} look like one flow, but every merged name collided — left as separate tools.`, + `${left.name} + ${right.name} look like one flow, but every merged name collided - left as separate tools.`, ); continue; } @@ -208,7 +208,7 @@ export function groupHandshakes(candidates: CandidateTool[]): GroupResult { }, }); notes.push( - `Grouped ${left.name} + ${right.name} into ${name} — one action the API split in two calls. It starts withheld like its members; enable ${name} instead of the pair when you're satisfied.`, + `Grouped ${left.name} + ${right.name} into ${name} - one action the API split in two calls. It starts withheld like its members; enable ${name} instead of the pair when you're satisfied.`, ); } diff --git a/packages/codegen/src/index.ts b/packages/codegen/src/index.ts index 45a6adc..206e8b6 100644 --- a/packages/codegen/src/index.ts +++ b/packages/codegen/src/index.ts @@ -2,7 +2,7 @@ * webmcp-codegen public API. * * Most people only ever need `defineConfig`. Sources and outputs live - * behind their own subpaths ("@webmcp-stack/codegen/sources", "…/outputs") + * behind their own subpaths ("@webmcp-stack/codegen/sources", ".../outputs") * so the top-level import stays small. */ diff --git a/packages/codegen/src/json-schema.ts b/packages/codegen/src/json-schema.ts index 876ded0..6128bc7 100644 --- a/packages/codegen/src/json-schema.ts +++ b/packages/codegen/src/json-schema.ts @@ -56,7 +56,7 @@ export function deref(schema: JsonSchema, spec: unknown): JsonSchema { * self-contained JSON Schema. The browser has no idea what * "#/components/schemas/Order" means, so refs must not survive codegen. * - * Recursive models (Order → LineItem → Order) would loop forever, so a ref + * Recursive models (Order -> LineItem -> Order) would loop forever, so a ref * that points back to one of its own ancestors resolves to a plain object * with a note instead. The tool schema stays finite and honest. */ @@ -152,7 +152,7 @@ export function jsonSchemaToTs(schema: JsonSchema, spec: unknown): string { } } -/** "get-order-status" → "GetOrderStatus" (for generated type names). */ +/** "get-order-status" -> "GetOrderStatus" (for generated type names). */ export function pascalCase(name: string): string { return name .split(/[-_]/) diff --git a/packages/codegen/src/logger.ts b/packages/codegen/src/logger.ts index d7d6469..dc9dca2 100644 --- a/packages/codegen/src/logger.ts +++ b/packages/codegen/src/logger.ts @@ -1,7 +1,7 @@ /** * Logging via pino, pretty-printed through one in-process stream in both * TTY and CI: the report is the product's output, and a human reads CI logs - * too — piped output is the same text with the colors stripped, never JSON + * too - piped output is the same text with the colors stripped, never JSON * envelopes. (If a machine-readable mode is ever needed, it is a --json * flag, not a silent format change on pipe.) * diff --git a/packages/codegen/src/naming.test.ts b/packages/codegen/src/naming.test.ts index d3d4fad..4ff78df 100644 --- a/packages/codegen/src/naming.test.ts +++ b/packages/codegen/src/naming.test.ts @@ -92,7 +92,7 @@ describe("analyzeRoute", () => { ["patch", "/v1/users/me", "update-current-user"], ]; - it.each(cases)("%s %s → %s", (method, path, expected) => { + it.each(cases)("%s %s -> %s", (method, path, expected) => { expect(analyzeRoute(method, path).base).toBe(expected); }); diff --git a/packages/codegen/src/naming.ts b/packages/codegen/src/naming.ts index 3780783..2c76f46 100644 --- a/packages/codegen/src/naming.ts +++ b/packages/codegen/src/naming.ts @@ -6,13 +6,13 @@ * ("post-trips-trip-id-story-generate"). Every tool gets its name from the * best signal available, in order: * - * 1. An explicit override (config or .webmcp-codegen.json) — always wins. + * 1. An explicit override (config or .webmcp-codegen.json) - always wins. * 2. A cleaned operationId, when the spec has one. * 3. An intent-shaped name derived from the route (analyzeRoute). - * 4. The plain method+path concat — the total fallback that can never fail. + * 4. The plain method+path concat - the total fallback that can never fail. * * resolveNames() runs the set-level pass: collisions are deepened with - * parent context ("generate-story" → "generate-trip-story"), and every + * parent context ("generate-story" -> "generate-trip-story"), and every * rename is reported. */ @@ -25,17 +25,17 @@ export const TOOL_NAME_PATTERN = /^[a-zA-Z][a-zA-Z0-9_-]*$/; * Turn an operationId or route fragment into a valid, readable tool name. * * Examples: - * "getOrderStatus" → "get-order-status" - * "GET /orders/{id}" → "get-orders-id" - * "list_pets" → "list-pets" + * "getOrderStatus" -> "get-order-status" + * "GET /orders/{id}" -> "get-orders-id" + * "list_pets" -> "list-pets" */ export function toToolName(raw: string): string { const name = raw - // Split acronym boundaries first: "getHTTPStatus" → "get-HTTPStatus" + // Split acronym boundaries first: "getHTTPStatus" -> "get-HTTPStatus" .replace(/([A-Z]+)([A-Z][a-z])/g, "$1-$2") - // Then camelCase and PascalCase boundaries: "getOrder" → "get-Order" + // Then camelCase and PascalCase boundaries: "getOrder" -> "get-Order" .replace(/([a-z0-9])([A-Z])/g, "$1-$2") - // Path placeholders and separators become dashes: "/orders/{id}" → "-orders-id" + // Path placeholders and separators become dashes: "/orders/{id}" -> "-orders-id" .replace(/[{}/_\s.]+/g, "-") // Anything left that isn't a letter, digit or dash is dropped .replace(/[^a-zA-Z0-9-]/g, "") @@ -72,8 +72,8 @@ export function nameFromRoute(method: string, path: string): string { export function cleanOperationId(operationId: string): string { let id = operationId; - // Drop a leading framework prefix: "TripsController_create" → "create", - // "user.service.getProfile" → "getProfile". Everything up to and including + // Drop a leading framework prefix: "TripsController_create" -> "create", + // "user.service.getProfile" -> "getProfile". Everything up to and including // the first scaffolding marker goes, and only when something remains. const segments = id.split(/_|\./).filter(Boolean); const marker = segments.findIndex((s) => /(controller|service|api|handler)$/i.test(s)); @@ -81,7 +81,7 @@ export function cleanOperationId(operationId: string): string { id = segments.slice(marker + 1).join("_"); } - // Drop a trailing version marker: "createTrip_v2" → "createTrip". + // Drop a trailing version marker: "createTrip_v2" -> "createTrip". id = id.replace(/[._-]?v\d+$/i, ""); return toToolName(id); @@ -205,7 +205,7 @@ export interface RouteAnalysis { base: string; /** * "intent" names can absorb parent context on collision ("generate-story" - * → "generate-trip-story"); "fallback" names are fixed strings. + * -> "generate-trip-story"); "fallback" names are fixed strings. */ tier: "intent" | "fallback"; /** The verb and noun the name is built from: verb + context + noun. */ @@ -225,7 +225,7 @@ export interface RouteAnalysis { droppedVersion: boolean; } -/** Singularize the last word of a phrase: "saved-places" → "saved-place". */ +/** Singularize the last word of a phrase: "saved-places" -> "saved-place". */ function singularPhrase(phrase: string): string { const words = phrase.split("-"); words[words.length - 1] = pluralize.singular(words[words.length - 1] as string); @@ -260,7 +260,7 @@ function intentName( reserve: string[] = [], ): RouteAnalysis { // Context is stored nearest-first, but names read outermost-first: - // "list-explore-destination-candidates", not "list-destination-explore-…". + // "list-explore-destination-candidates", not "list-destination-explore-...". const used = context.slice(0, minDepth).reverse(); const base = [verb, ...used, noun].filter(Boolean).join("-"); return { base, tier: "intent", verb, noun, context, minDepth, droppedVersion, reserve }; @@ -271,18 +271,18 @@ function intentName( * * Two shapes are recognized: * - * Action endpoints — the last segment starts with a known verb, so the - * name is the action: POST /trips/{id}/story/generate → "generate-story". + * Action endpoints - the last segment starts with a known verb, so the + * name is the action: POST /trips/{id}/story/generate -> "generate-story". * Action names stay minimal; parent context is spent only on collision. * - * Plain REST — the method implies the verb and the path shape says member - * or collection: POST /trips → "create-trip", GET /trips/{id} → - * "get-trip", GET /trips/{id}/blocks → "list-trip-blocks". Nested reads + * Plain REST - the method implies the verb and the path shape says member + * or collection: POST /trips -> "create-trip", GET /trips/{id} -> + * "get-trip", GET /trips/{id}/blocks -> "list-trip-blocks". Nested reads * and writes include the immediate parent, because every API nests list * and get somewhere and parentless names collide constantly. * * Anything else falls back to the method+path concat, which can never fail - * to produce a name — it can only produce a boring one. + * to produce a name - it can only produce a boring one. */ export function analyzeRoute(method: string, path: string): RouteAnalysis { const segments = path.split("/").filter(Boolean); @@ -290,11 +290,11 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { const droppedVersion = rest.length !== segments.length; // Walk the path once, keeping resource segments in order and noting where - // params sit between them — a param between two resources is what makes a + // params sit between them - a param between two resources is what makes a // name nested ("trips/{id}/blocks" is nested; "trips/{id}" is not). // A member word mid-path merges into its parent as a scope marker: - // "/users/me/stamps" is read as "current-user → stamps", not "users, me, - // stamps" — "me" names no resource of its own. + // "/users/me/stamps" is read as "current-user -> stamps", not "users, me, + // stamps" - "me" names no resource of its own. const resources: { phrase: string; nestedUnderMember: boolean }[] = []; let sawParam = false; for (const segment of rest) { @@ -344,8 +344,8 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { // "batch" is a modifier, never the verb: "batch-trip-blocks" reads as a // tool about batching, not as blocks being written. The verb comes from - // the rest of the segment ("batch-delete" → "delete-media-batch") or from - // the method ("…/blocks/batch" → "update-trip-blocks-batch"). + // the rest of the segment ("batch-delete" -> "delete-media-batch") or from + // the method (".../blocks/batch" -> "update-trip-blocks-batch"). if (firstWord === "batch" && resources.length > 1) { const prev = resources[resources.length - 2] as (typeof resources)[number]; const rest = last.phrase.split("-").slice(1).join("-"); @@ -374,7 +374,7 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { // Plain REST: the method supplies the verb, the path shape the noun. const verbs = METHOD_VERBS[method.toLowerCase()]; - if (!verbs) return fallback; // HEAD, OPTIONS, WebDAV — honest concat. + if (!verbs) return fallback; // HEAD, OPTIONS, WebDAV - honest concat. // A trailing "all" scopes the parent collection rather than naming one: // "GET /pricing/all" is "list-all-pricing", never "get-all". @@ -392,9 +392,9 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { } if (lastIsParam) { - // GET /trips/{id} → the member named by the last resource segment. Two + // GET /trips/{id} -> the member named by the last resource segment. Two // or more trailing params are a lookup by the last one: - // GET /trips/{username}/{slug} → "get-trip-by-slug". + // GET /trips/{username}/{slug} -> "get-trip-by-slug". const noun = lastParam ? `${singularPhrase(last.phrase)}-by-${lastParam}` : singularPhrase(last.phrase); @@ -410,7 +410,7 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { } // A trailing resource word that reads plural is a collection; a singular - // one is a singleton sub-resource (GET /auth/session → "get-session"). + // one is a singleton sub-resource (GET /auth/session -> "get-session"). const member = !pluralize.isPlural(last.phrase.split("-").pop() as string); // POST with a trailing resource always creates one of it, whether the word // reads singular or plural: "create-trip-template", "create-trip-block". @@ -460,7 +460,7 @@ export interface ResolvedNames { * Assign final names to a whole tool set. * * Route-derived names start minimal ("generate-story") and absorb parent - * context on collision ("generate-trip-story") — minimal first because the + * context on collision ("generate-trip-story") - minimal first because the * short name is usually unique, deepening because a bare noun stops saying * which resource once a second API has one. Ties break in order: parent * context, grouping words kept in reserve ("admin" beats a number), the @@ -555,7 +555,7 @@ export function resolveNames(inputs: NameInput[]): ResolvedNames { // Nobody can deepen. The declared name (or the first arrival) keeps the // spot; the rest take a method suffix, then a counter. A method suffix // that repeats the verb says nothing ("get-trip-get"), so those go - // straight to the counter — and the counter is always an error, because + // straight to the counter - and the counter is always an error, because // a numbered name means the spec needs a human's word, not our digit. for (const group of colliding) { const ordered = [...group].sort( diff --git a/packages/codegen/src/outputs/assets.ts b/packages/codegen/src/outputs/assets.ts index 234b899..06552fd 100644 --- a/packages/codegen/src/outputs/assets.ts +++ b/packages/codegen/src/outputs/assets.ts @@ -2,11 +2,11 @@ * Read a bundled asset (the skill file, the journey helper). * * These ship as files in the published package rather than as template - * strings in source because they are also the reviewable artifacts — the + * strings in source because they are also the reviewable artifacts - the * design docs and the docs site point at assets/ directly, and two sources * of truth would drift. The package publishes dist + assets; the two - * candidate roots cover the built layout (dist/x.js → ../assets) and the - * source tree under test (src/outputs/x.ts → ../../assets). + * candidate roots cover the built layout (dist/x.js -> ../assets) and the + * source tree under test (src/outputs/x.ts -> ../../assets). */ import { readFile } from "node:fs/promises"; diff --git a/packages/codegen/src/outputs/tools-templates.ts b/packages/codegen/src/outputs/tools-templates.ts index 3cf1ef8..7d943a0 100644 --- a/packages/codegen/src/outputs/tools-templates.ts +++ b/packages/codegen/src/outputs/tools-templates.ts @@ -281,7 +281,7 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { if (tool.piiInOutput.length > 0) { lines.push( `//`, - `// ⚠ webmcp-codegen flagged these response fields as likely PII: ${tool.piiInOutput.join(", ")}.`, + `// ! webmcp-codegen flagged these response fields as likely PII: ${tool.piiInOutput.join(", ")}.`, `// Everything you return reaches the agent. Leave those fields out of what you`, `// return unless the agent genuinely needs them, and say so in a comment if you keep them.`, ); @@ -343,7 +343,7 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { * knows: the path template becomes a template literal, query params become * the search string, body fields become the JSON body. * - * "/pets/{id}" + DELETE → const data = await callApi(`/pets/${input.id}`, { method: "DELETE" }); + * "/pets/{id}" + DELETE -> const data = await callApi(`/pets/${input.id}`, { method: "DELETE" }); * * When the source carries no route information, we fall back to an honest * TODO instead of inventing a URL. @@ -353,7 +353,7 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { * is frequently a local dev URL (http://localhost:3001); baking that into the * generated fetch makes every deployed tool call the visitor's own machine. * So: a non-local absolute URL is kept (the API genuinely lives elsewhere), - * a local one returns undefined so the tool falls back to same-origin — + * a local one returns undefined so the tool falls back to same-origin - * which is where a deployed app's API actually is. */ export function resolveApiBase(serverUrl: string | undefined): string | undefined { @@ -372,7 +372,7 @@ export function resolveApiBase(serverUrl: string | undefined): string | undefine /** * The arguments to one callApi(...): the path expression (template params * interpolated, non-local server URLs kept) plus method/query/body/signal. - * `pathRef` says where each path param's value comes from — ordinary tools + * `pathRef` says where each path param's value comes from - ordinary tools * read `input`, the second call of a composed tool reads the first result. */ function buildCallExpr(options: { @@ -395,7 +395,7 @@ function buildCallExpr(options: { const queryParams = queryParamsAll.filter((name) => !skipFields.has(name)); const bodyParams = bodyParamsAll.filter((name) => !skipFields.has(name)); - // "/pets/{id}" → `/pets/${input.id}`. Params the schema knows by name. + // "/pets/{id}" -> `/pets/${input.id}`. Params the schema knows by name. let pathExpr = `\`${pathTemplate.replace(/\{([^}]+)\}/g, (_m, param: string) => `\${${pathRef(param)}}`)}\``; if (pathParams.length === 0) pathExpr = JSON.stringify(pathTemplate); @@ -453,7 +453,7 @@ function firstResultRef(param: string): string { * The two calls of a grouped handshake tool: the first request runs, its * response fields fill the second request's path params by exact name, and * the second response is the tool's result. Threaded fields never reach the - * agent-facing input — that's the point of the composition. + * agent-facing input - that's the point of the composition. */ function composedFetchBody(plan: NonNullable): string[] { const threaded = new Set(Object.keys(plan.threaded)); @@ -616,7 +616,7 @@ export async function callApi( const TOOL_OUTPUT_MAX = 1536; const TRUNCATED_NOTICE = - "\\n… [truncated to fit the 1.5K output budget — return a smaller slice or paginate]"; + "\\n... [truncated to fit the 1.5K output budget - return a smaller slice or paginate]"; /** * Wrap a result in the MCP shape, so tool bodies stay one line. The result @@ -735,9 +735,9 @@ export async function registerAllTools(signal?: AbortSignal): Promise { `; } -/** "GetOrderStatus" → "getOrderStatus" (for the generated const names). */ +/** "GetOrderStatus" -> "getOrderStatus" (for the generated const names). */ /** - * The spec's human-facing `title`: "list-trips" → "List Trips". Derived from + * The spec's human-facing `title`: "list-trips" -> "List Trips". Derived from * the name so the two never disagree. */ function titleFromName(name: string): string { diff --git a/packages/codegen/src/outputs/tools.test.ts b/packages/codegen/src/outputs/tools.test.ts index bda2123..a37aca2 100644 --- a/packages/codegen/src/outputs/tools.test.ts +++ b/packages/codegen/src/outputs/tools.test.ts @@ -207,7 +207,7 @@ describe("js generator", () => { // indentation preserved, so uncommenting restores working code. expect(tool?.contents).toContain("// ...cancelOrderTool,"); // The import reflects it: getModelContext is part of the fence, not the - // file — but callApi stays live, because the raw caller (fetchX) the + // file - but callApi stays live, because the raw caller (fetchX) the // generated region emits is live too: journeys compose withheld tools. expect(tool?.contents).toContain('import { callApi, toolDisabled } from "./runtime.webmcp";'); expect(tool?.contents).toContain("export async function fetchCancelOrder("); @@ -333,7 +333,7 @@ describe("js generator", () => { ); const tool = files.find((file) => file.path.includes("cancel-order")); // Disabled notice first, the working call right below it, commented out - // — and it composes the generated region's raw caller like every other + // - and it composes the generated region's raw caller like every other // endpoint-backed tool. expect(tool?.contents).toContain('return toolDisabled("cancel-order.webmcp.ts");'); expect(tool?.contents).toContain("// const data = await fetchCancelOrder(input, signal);"); @@ -377,7 +377,7 @@ describe("js generator", () => { ); // The disabled tool's request is commented out, but its generated raw - // caller (fetchX) is live — so callApi is in the import line, and the + // caller (fetchX) is live - so callApi is in the import line, and the // enable instructions only name what's genuinely missing. const disabledWrite = files.find((file) => file.path.includes("cancel-order")); expect(disabledWrite?.contents).toContain( @@ -535,7 +535,7 @@ describe("js generator", () => { await writeAll( await output.generate( [ - reviewedTool(), // get-order-status → GET /orders/{id} + reviewedTool(), // get-order-status -> GET /orders/{id} reviewedTool({ id: "DELETE /admin/orders/{id}", name: "delete-order", diff --git a/packages/codegen/src/outputs/tools.ts b/packages/codegen/src/outputs/tools.ts index 7338462..76de72d 100644 --- a/packages/codegen/src/outputs/tools.ts +++ b/packages/codegen/src/outputs/tools.ts @@ -6,15 +6,15 @@ * Output layout for `tools({ outDir: "./src/webmcp" })`: * * src/webmcp/ - * ├── runtime.webmcp.ts ← fully generated, never edit - * ├── index.ts ← fully generated, registers everything - * ├── get-order-status.webmcp.ts ← generated contract + YOUR execute() - * └── ... + * |-- runtime.webmcp.ts <- fully generated, never edit + * |-- index.ts <- fully generated, registers everything + * |-- get-order-status.webmcp.ts <- generated contract + YOUR execute() + * `-- ... * * Each per-tool file has two regions, divided by marker comments: * * generated region schema, input type, tool definition, register() - * ── end generated ── everything below survives regeneration + * -- end generated -- everything below survives regeneration * your region execute(), scaffolded once, then owned by you * * A tool's identity is its source ref (the endpoint, or the declared schema @@ -45,7 +45,7 @@ export interface ToolsOutputOptions { outDir: string; /** * The spec's `exposedTo`: secure origins (embedded documents at these - * origins) the registered tools are shared with. Absent means the default — + * origins) the registered tools are shared with. Absent means the default - * tools are visible to the page itself, same-origin documents, and the * browser's built-in agent. Only list origins you trust to act for your user. */ @@ -58,9 +58,18 @@ export interface ToolsOutputOptions { * and we must never touch anything after it. tools-templates.ts imports * these so the marker text is defined in exactly one place. */ -export const GENERATED_START = "// ─── webmcp-codegen: generated. Do not edit this region. ───"; +export const GENERATED_START = "// --- webmcp-codegen: generated. Do not edit this region. ---"; export const GENERATED_END = - "// ─── webmcp-codegen: end generated. Your code below survives regeneration. ───"; + "// --- webmcp-codegen: end generated. Your code below survives regeneration. ---"; + +/** + * The pre-0.9 end marker, written with box-drawing characters. Reads still + * accept it so files generated by an older version migrate to the plain + * marker on the next run instead of looking like a hand edit. Written with + * escapes so this source file stays plain text. + */ +const GENERATED_END_LEGACY = + "// \u2500\u2500\u2500 webmcp-codegen: end generated. Your code below survives regeneration. \u2500\u2500\u2500"; /** Create the `tools` output for the config's `outputs` array. */ export function tools(options: ToolsOutputOptions): Output { @@ -98,10 +107,10 @@ export function tools(options: ToolsOutputOptions): Output { .filter((entry) => entry.endsWith(".webmcp.ts")) .sort(); } catch { - // No journeys directory yet — most repos, most of the time. + // No journeys directory yet - most repos, most of the time. } - // Endpoint ref → the file currently holding it. + // Endpoint ref -> the file currently holding it. const pathByRef = new Map(); for (const [path, contents] of existing) { const ref = sourceRefOf(contents); @@ -192,7 +201,7 @@ export function tools(options: ToolsOutputOptions): Output { // The skill file: the rules harness for the user's own coding agents, // at the cross-client skills location. Regenerated wholesale like the - // runtime — its header comment says why (project rules belong in the + // runtime - its header comment says why (project rules belong in the // user's own skill directory, which stacks on top). const skillFile = await plainFile( resolve(cwd, ".agents/skills/webmcp-tools/SKILL.md"), @@ -254,8 +263,12 @@ function toolFile( }; } - const markerIndex = existing.indexOf(GENERATED_END); - if (markerIndex === -1) { + const endMarker = existing.includes(GENERATED_END) + ? GENERATED_END + : existing.includes(GENERATED_END_LEGACY) + ? GENERATED_END_LEGACY + : undefined; + if (endMarker === undefined) { // Someone removed the markers or hand-wrote this path from scratch. // Never clobber their work: report a conflict and let the pipeline put // our version in a `.new` sibling for a human to merge. @@ -263,7 +276,7 @@ function toolFile( } // Keep everything the developer wrote below the marker, word for word. - const preservedTail = existing.slice(markerIndex + GENERATED_END.length); + const preservedTail = existing.slice(existing.indexOf(endMarker) + endMarker.length); const contents = head + preservedTail; return { path, diff --git a/packages/codegen/src/pipeline.ts b/packages/codegen/src/pipeline.ts index cb3dd0f..3f25533 100644 --- a/packages/codegen/src/pipeline.ts +++ b/packages/codegen/src/pipeline.ts @@ -1,5 +1,5 @@ /** - * The pipeline: sources → normalize → safety review → audit → write. + * The pipeline: sources -> normalize -> safety review -> audit -> write. * * This module is the only place the stages meet. It owns no opinions of its * own; naming, safety, and file formats all live in their own modules. It @@ -40,7 +40,7 @@ export interface GenerateOptions { */ overrides?: ToolOverrides; /** - * The names the last run produced (name → route ref), from the same file. + * The names the last run produced (name -> route ref), from the same file. * When a route's name changes between runs, the rename is reported and the * tool's overrides move with it: a rename is a report line, never a silent * break. @@ -58,9 +58,9 @@ export interface GenerateResult { files: GeneratedFile[]; /** Human-facing pipeline notes, e.g. "stripped the shared v1 prefix". */ notes: string[]; - /** Names that changed since the last run (old → new), overrides re-keyed. */ + /** Names that changed since the last run (old -> new), overrides re-keyed. */ crossRenames: { from: string; to: string }[]; - /** The names this run produced (name → route ref), for the caller to save. */ + /** The names this run produced (name -> route ref), for the caller to save. */ namesLedger: Record; /** Overrides with renamed tools re-keyed, when a rename moved any. */ migratedOverrides?: ToolOverrides; @@ -136,7 +136,7 @@ export async function runGenerate( // Cross-run renames. The route ref is the durable identity; the name is // derived. When they drift apart (a better algorithm, a spec edit), the // tool's dashboard overrides are keyed by the old name and would silently - // stop applying — so they are re-keyed here, before step 6 reads them. + // stop applying - so they are re-keyed here, before step 6 reads them. const refOf = (tool: (typeof named)[number]): string => tool.endpointRef ?? tool.source.ref; const crossRenames: { from: string; to: string }[] = []; if (options.previousNames) { @@ -235,7 +235,7 @@ export async function runGenerate( level: "warning" as const, tool: rename.to, message: - `Renamed "${rename.from}" → "${rename.to}" since the last run. ` + + `Renamed "${rename.from}" -> "${rename.to}" since the last run. ` + "Dashboard edits moved with it; update any code that imported the old name.", })), ...formFindings, diff --git a/packages/codegen/src/safety.ts b/packages/codegen/src/safety.ts index 077eb41..9b5a9a9 100644 --- a/packages/codegen/src/safety.ts +++ b/packages/codegen/src/safety.ts @@ -310,7 +310,7 @@ export function auditTools( findings.push({ level: "warning", tool: rename.to, - message: `Renamed "${rename.from}" → "${rename.to}" to keep tool names unique.`, + message: `Renamed "${rename.from}" -> "${rename.to}" to keep tool names unique.`, }); } diff --git a/packages/codegen/src/sources/openapi.test.ts b/packages/codegen/src/sources/openapi.test.ts index 8473f5b..5929bae 100644 --- a/packages/codegen/src/sources/openapi.test.ts +++ b/packages/codegen/src/sources/openapi.test.ts @@ -111,7 +111,7 @@ describe("openapi source", () => { expect(tools.map((tool) => tool.name)).toEqual([ "list-orders", "create-order", - "get-order", // no operationId → intent name from the route shape + "get-order", // no operationId -> intent name from the route shape "delete-order", ]); }); diff --git a/packages/codegen/src/sources/openapi.ts b/packages/codegen/src/sources/openapi.ts index c8f99e5..48c4eb6 100644 --- a/packages/codegen/src/sources/openapi.ts +++ b/packages/codegen/src/sources/openapi.ts @@ -7,10 +7,10 @@ * changing any application code. * * What we read from each operation: - * - name ← operationId, slugified (falls back to method + path) - * - description ← summary, else the first line of description, else a template - * - inputSchema ← path + query parameters merged with the JSON request body - * - outputSchema ← the first 2xx response's JSON schema, when present + * - name <- operationId, slugified (falls back to method + path) + * - description <- summary, else the first line of description, else a template + * - inputSchema <- path + query parameters merged with the JSON request body + * - outputSchema <- the first 2xx response's JSON schema, when present * * Header and cookie parameters are skipped on purpose: agents should not be * setting those by hand, and auth headers are the app's job, not the tool's. @@ -226,7 +226,7 @@ function buildInputSchema( for (const key of body.schema.required ?? []) required.add(key); } } else { - // A non-object body (array, raw string, …) goes under a "body" field. + // A non-object body (array, raw string, ...) goes under a "body" field. properties.body = body.schema; if (body.required) required.add("body"); } diff --git a/packages/codegen/src/sources/schema.ts b/packages/codegen/src/sources/schema.ts index 471e96e..ae51627 100644 --- a/packages/codegen/src/sources/schema.ts +++ b/packages/codegen/src/sources/schema.ts @@ -69,7 +69,7 @@ function isStandardSchema(value: unknown): value is StandardSchemaV1 { /** * TypeBox v1 schemas are JSON Schema documents, not wrapper objects: the value * has `type` and `properties` and no `~standard`. This is the only check that - * does not lie about what TypeBox is — there is no vendor marker to read. + * does not lie about what TypeBox is - there is no vendor marker to read. */ function isTypeBoxSchema(value: unknown): value is JsonSchema { if (typeof value !== "object" || value === null || Array.isArray(value)) return false; @@ -159,7 +159,7 @@ function toJsonSchema( anchorDir: string, ): JsonSchema { // TypeBox schemas are JSON Schema already: `Type.Object({...})` returns the - // draft-2020-12 shape with no wrapper. Detected by shape, not a marker — + // draft-2020-12 shape with no wrapper. Detected by shape, not a marker - // TypeBox v1 has no `~standard` and no `_def`; the object is the contract. if (isTypeBoxSchema(value)) { return value; diff --git a/packages/codegen/src/types.ts b/packages/codegen/src/types.ts index b0d686e..d24b749 100644 --- a/packages/codegen/src/types.ts +++ b/packages/codegen/src/types.ts @@ -3,7 +3,7 @@ * * Every stage of the pipeline speaks in these types: * - * Source → CandidateTool → (safety review) → ReviewedTool → Output → GeneratedFile + * Source -> CandidateTool -> (safety review) -> ReviewedTool -> Output -> GeneratedFile * * A source only has to produce CandidateTools. An output only has to turn * ReviewedTools into files. Everything in between lives here so the stages diff --git a/packages/codegen/src/verify.test.ts b/packages/codegen/src/verify.test.ts index 60137af..c5394f7 100644 --- a/packages/codegen/src/verify.test.ts +++ b/packages/codegen/src/verify.test.ts @@ -43,7 +43,7 @@ describe("verify description budgets", () => { ]); const budgets = checks.find((check) => check.area === "Budgets"); expect(budgets?.level).toBe("warning"); - expect(budgets?.findings[0]).toContain("get-order-status → notes"); + expect(budgets?.findings[0]).toContain("get-order-status -> notes"); expect(budgets?.findings[0]).toContain("150"); }); diff --git a/packages/codegen/src/verify.ts b/packages/codegen/src/verify.ts index d357b07..ebdab72 100644 --- a/packages/codegen/src/verify.ts +++ b/packages/codegen/src/verify.ts @@ -3,8 +3,8 @@ * * Why this exists: a generated surface's quality was only ever discovered * after deploy, by an external audit reading the page's registry. Verify runs - * the same rubric over the tools a run would register — name shape, what the - * description covers, field text, annotations, surface size — and prints a + * the same rubric over the tools a run would register - name shape, what the + * description covers, field text, annotations, surface size - and prints a * scorecard before anything ships. * * The checks are deterministic: same tools in, same verdict out, no key, no @@ -133,8 +133,8 @@ function check(area: string, offenders: string[], okText: string): VerifyCheck { } /** - * Walk every input field of a schema — nested objects and array items, two - * levels down, mirroring the describe layer's coverage — and run `visit` on + * Walk every input field of a schema - nested objects and array items, two + * levels down, mirroring the describe layer's coverage - and run `visit` on * each name and its text. */ function eachField( @@ -192,7 +192,7 @@ export function verifyTools( const longNames = registered .filter((tool) => tool.name.length > NAME_MAX) .map( - (tool) => `${tool.name} (${tool.name.length} chars) — rename it in the dashboard or config.`, + (tool) => `${tool.name} (${tool.name.length} chars) - rename it in the dashboard or config.`, ); const longParamNames: string[] = []; const longParamDescriptions: string[] = []; @@ -200,46 +200,46 @@ export function verifyTools( eachField(tool.inputSchema, 0, (name, description) => { if (name.length > PARAM_NAME_MAX) { longParamNames.push( - `${tool.name} → ${name} (${name.length} chars) — parameter names max ${PARAM_NAME_MAX}.`, + `${tool.name} -> ${name} (${name.length} chars) - parameter names max ${PARAM_NAME_MAX}.`, ); } if (description && description.length > FIELD_DESCRIPTION_MAX) { longParamDescriptions.push( - `${tool.name} → ${name} (${description.length} chars) — over the ${FIELD_DESCRIPTION_MAX}-character parameter budget; tighten it.`, + `${tool.name} -> ${name} (${description.length} chars) - over the ${FIELD_DESCRIPTION_MAX}-character parameter budget; tighten it.`, ); } }); } const nonVerb = registered .filter((tool) => !KNOWN_VERBS.has(tool.name.split("-")[0] ?? "")) - .map((tool) => `${tool.name} — agents pick tools by their first word; lead with the action.`); + .map((tool) => `${tool.name} - agents pick tools by their first word; lead with the action.`); const longDescriptions = registered .filter((tool) => tool.description && tool.description.length > TOOL_DESCRIPTION_MAX) .map( (tool) => - `${tool.name} (${tool.description.length} chars) — over the ${TOOL_DESCRIPTION_MAX}-character tool budget; tighten it.`, + `${tool.name} (${tool.description.length} chars) - over the ${TOOL_DESCRIPTION_MAX}-character tool budget; tighten it.`, ); const noDescription = registered .filter((tool) => !tool.description || tool.description.trim() === "") - .map((tool) => `${tool.name} — no description; the tool is invisible to agents.`); + .map((tool) => `${tool.name} - no description; the tool is invisible to agents.`); const templateDescription = registered .filter((tool) => tool.descriptionSource === "generated-template") - .map((tool) => `${tool.name} — description is a machine draft; write the real one.`); + .map((tool) => `${tool.name} - description is a machine draft; write the real one.`); const noReturnShape = registered .filter( (tool) => tool.outputSchema && tool.description && !RETURN_LANGUAGE.test(tool.description), ) - .map((tool) => `${tool.name} — the description never says what comes back.`); + .map((tool) => `${tool.name} - the description never says what comes back.`); const bareFields = registered .filter((tool) => hasBareField(tool.inputSchema, 0)) - .map((tool) => `${tool.name} — an input field has no description an agent can act on.`); + .map((tool) => `${tool.name} - an input field has no description an agent can act on.`); const readWithoutHint = registered .filter((tool) => tool.sideEffect === "read" && !tool.hints.readOnlyHint) - .map((tool) => `${tool.name} — a read without readOnlyHint looks unsafe to call.`); + .map((tool) => `${tool.name} - a read without readOnlyHint looks unsafe to call.`); const checks: VerifyCheck[] = [ check( @@ -296,7 +296,7 @@ export function verifyTools( area: "Surface", summary: `${surfaceTotal} registered (${breakdown})`, findings: [ - `${surfaceTotal} tools register on this surface (${breakdown}) — agents choose measurably worse past a handful. ` + + `${surfaceTotal} tools register on this surface (${breakdown}) - agents choose measurably worse past a handful. ` + "Withhold unreviewed tools, split journeys, or narrow with safety.exclude.", ], level: "warning", @@ -355,7 +355,7 @@ function journeyStepNames(contents: string): string[] { const keyMatch = keyPattern.exec(body); if (keyMatch) { names.push(keyMatch[1] ?? ""); - // The match consumed the step's opening brace — count it, since the + // The match consumed the step's opening brace - count it, since the // walk skips past the whole match including that brace. innerDepth++; i = cursor + keyMatch[0].length - 1; @@ -385,33 +385,33 @@ export function verifyJourneyFiles(files: JourneyFileInput[]): VerifyCheck[] { for (const { path, contents } of files) { if (!/createJourney\s*\(/.test(contents)) { structural.push( - `${path} — no createJourney() call; files in journeys/ must define a journey.`, + `${path} - no createJourney() call; files in journeys/ must define a journey.`, ); continue; } if (!/submit\s*:/.test(contents) || !/run\s*:/.test(contents)) { structural.push( - `${path} — no submit gate with a run; a journey without one is just loose tools.`, + `${path} - no submit gate with a run; a journey without one is just loose tools.`, ); } const steps = journeyStepNames(contents); if (steps.length > 5) { warnings.push( - `${path} — ${steps.length} steps; past five, agents lose the thread. Split it into two journeys.`, + `${path} - ${steps.length} steps; past five, agents lose the thread. Split it into two journeys.`, ); } for (const match of contents.matchAll(/description\s*:\s*"((?:[^"\\]|\\.)*)"/g)) { const text = match[1] ?? ""; if (text.length > TOOL_DESCRIPTION_MAX) { budget.push( - `${path} — a description runs ${text.length} characters (max ${TOOL_DESCRIPTION_MAX}); tighten it.`, + `${path} - a description runs ${text.length} characters (max ${TOOL_DESCRIPTION_MAX}); tighten it.`, ); } } const withoutImports = contents.replace(/^\s*import\s.*$/gm, ""); if (/\b(?:fetch|callApi)\s*\(/.test(withoutImports)) { warnings.push( - `${path} — calls fetch/callApi directly; use the generated raw callers (fetchX) or a tool's execute, so the contract lives in one place.`, + `${path} - calls fetch/callApi directly; use the generated raw callers (fetchX) or a tool's execute, so the contract lives in one place.`, ); } } diff --git a/packages/codegen/src/wire.ts b/packages/codegen/src/wire.ts index 00d56f3..100ebfc 100644 --- a/packages/codegen/src/wire.ts +++ b/packages/codegen/src/wire.ts @@ -3,7 +3,7 @@ * * Generated files do nothing until something calls registerAllTools() once at * startup. Rather than telling the developer to go do that, we do it for - * them — under strict rules, because this is the one place we edit *their* + * them - under strict rules, because this is the one place we edit *their* * files instead of ours: * * 1. Edits are additive only. We insert lines; we never change or remove @@ -67,16 +67,16 @@ export async function applyWiring(plan: WirePlan): Promise { } } -/* ── Next.js (app router) ──────────────────────────────────────────────── */ +/* -- Next.js (app router) ------------------------------------------------ */ /** * Next needs the registration to run on the client, so we generate a tiny * "use client" component next to the tools and mount it in the root layout: * - * import { WebMCPRegister } from "../webmcp/register"; ← added + * import { WebMCPRegister } from "../webmcp/register"; <- added * ... * - * ← added + * <- added * {children} */ async function planNextWiring(cwd: string, app: WebApp, outDir: string): Promise { @@ -92,7 +92,7 @@ async function planNextWiring(cwd: string, app: WebApp, outDir: string): Promise const registerPath = join(cwd, outDir, "register.tsx"); const layout = await readFile(layoutPath, "utf8"); if (layout.includes("WebMCPRegister")) { - // The layout mounts the component — but "wired" also means the file it + // The layout mounts the component - but "wired" also means the file it // points at exists. A deleted register.tsx (or a fresh clone where it // was never committed) must not leave the app broken. try { @@ -165,13 +165,13 @@ export function WebMCPRegister() { `; } -/* ── Vite + React (SPAs) ───────────────────────────────────────────────── */ +/* -- Vite + React (SPAs) ------------------------------------------------- */ /** * A Vite app boots in main.tsx, so wiring is two added lines there: * - * import { registerAllTools } from "./webmcp"; ← added - * void registerAllTools(); ← added + * import { registerAllTools } from "./webmcp"; <- added + * void registerAllTools(); <- added */ async function planViteWiring(cwd: string, app: WebApp, outDir: string): Promise { const entryCandidates = [ @@ -205,7 +205,7 @@ async function planViteWiring(cwd: string, app: WebApp, outDir: string): Promise }; } -/* ── Shared helpers ────────────────────────────────────────────────────── */ +/* -- Shared helpers ------------------------------------------------------ */ /** Insert a line after the file's last top-level import statement. */ function insertAfterLastImport(source: string, line: string): string | null { @@ -221,7 +221,7 @@ function insertAfterLastImport(source: string, line: string): string | null { /** * Turn a filesystem path into a JS import specifier: no extension, and an - * explicit "./" when the target is in the same directory or deeper — + * explicit "./" when the target is in the same directory or deeper - * `relative()` alone yields "webmcp/index", which JS would read as a * package name, not a file. */ diff --git a/site/content/docs/cli.mdx b/site/content/docs/cli.mdx index 86443ca..f6cd5d4 100644 --- a/site/content/docs/cli.mdx +++ b/site/content/docs/cli.mdx @@ -47,11 +47,11 @@ anything ships. Runs the same pipeline as `generate` (writing nothing) and repor scorecard: ``` -✓ Names: all 35 names within 30 characters, verb-first -! Descriptions: 2 problems -✓ Fields: every input field described, nested ones included -✓ Annotations: reads declare readOnlyHint; content declares its trust -! Surface: 35 registered +ok Names: all 35 names within 30 characters, verb-first +! Descriptions: 2 problems +ok Fields: every input field described, nested ones included +ok Annotations: reads declare readOnlyHint; content declares its trust +! Surface: 35 registered ``` Each finding names the tool and what to do about it. Exits 1 on error-level findings (a tool diff --git a/site/content/docs/devtools.mdx b/site/content/docs/devtools.mdx index 964c36c..c7ad637 100644 --- a/site/content/docs/devtools.mdx +++ b/site/content/docs/devtools.mdx @@ -7,7 +7,7 @@ Chrome DevTools has a dedicated WebMCP panel that shows every tool your page exp ## Open the panel -1. Enable WebMCP: `chrome://flags/#enable-webmcp-testing` (Chrome 149+, Edge 150+ — this flag is for local development; production pages use the origin trial) +1. Enable WebMCP: `chrome://flags/#enable-webmcp-testing` (Chrome 149+, Edge 150+. This flag is for local development; production pages use the origin trial.) 2. Open your app and open DevTools (F12) 3. Click the **Application** tab 4. In the sidebar, click **WebMCP** @@ -16,8 +16,8 @@ Chrome DevTools has a dedicated WebMCP panel that shows every tool your page exp The panel has two sections: -- **Available Tools** — every tool currently registered on the page, with name, description, and an invocation counter -- **Invoked Tools** — a log of every tool call, with status, input, and output +- **Available Tools**: every tool currently registered on the page, with name, description, and an invocation counter +- **Invoked Tools**: a log of every tool call, with status, input, and output ## Test a tool manually diff --git a/site/content/docs/journeys-faq.mdx b/site/content/docs/journeys-faq.mdx index ebc6b9d..574739c 100644 --- a/site/content/docs/journeys-faq.mdx +++ b/site/content/docs/journeys-faq.mdx @@ -113,7 +113,7 @@ registered tool, including journey steps and submits, lands in one flat list the browser exposes. The agent asks the page what it can do and gets the whole list, with no formal distinction between a normal tool and part of a journey. The only grouping signal is in the text: each step's name carries the journey prefix -(`book-ride-…`), and each description ends with something like *"Part of +(`book-ride-...`), and each description ends with something like *"Part of book-ride: Book a ride to a destination the user gives."* The agent reads that and infers the connection, the same way it figures out anything else. The factory stitches it mechanically, so it cannot be forgotten. diff --git a/site/content/docs/journeys.mdx b/site/content/docs/journeys.mdx index c114af3..a7913f4 100644 --- a/site/content/docs/journeys.mdx +++ b/site/content/docs/journeys.mdx @@ -116,7 +116,7 @@ agent book-ride-resolve-destination { "address": "SFO airport" } tool Stored. Still needed: book-ride-set-pickup-time, book-ride-submit. agent book-ride-set-pickup-time { "pickupAt": "2026-03-14T19:30:00Z" } -tool Stored. The journey is ready — call book-ride-submit. +tool Stored. The journey is ready. Call book-ride-submit. agent book-ride-submit (the page asks the human: "Allow the agent to: Request the ride and show the driver's details.") diff --git a/site/content/docs/quickstart.mdx b/site/content/docs/quickstart.mdx index 1afa764..bbe140c 100644 --- a/site/content/docs/quickstart.mdx +++ b/site/content/docs/quickstart.mdx @@ -59,7 +59,7 @@ guessing. 1. Start your app and open it in Chrome. 2. Turn on `chrome://flags/#enable-webmcp-testing` and reload. -3. Open DevTools → **Application** → **WebMCP** to see your tools listed. +3. Open DevTools, then **Application**, then **WebMCP**, to see your tools listed. 4. Click a tool to test it manually, or ask the agent to use one. The CLI suggests one, drawn from your spec ("list my trips"). diff --git a/site/content/docs/regeneration.mdx b/site/content/docs/regeneration.mdx index 58c7db9..c050af8 100644 --- a/site/content/docs/regeneration.mdx +++ b/site/content/docs/regeneration.mdx @@ -20,12 +20,12 @@ to be re-run every time your API changes. Each tool file has two regions, split by a marker comment: ```ts title="src/webmcp/delete-pet.webmcp.ts" -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- // name, description, input schema, input type, hints, register() // ... everything derived from the API contract -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- export async function executeDeletePet(input: DeletePetInput) { // This tool is withheld: nothing registers it, so agents cannot see or call it. From f780c302ba709b287cf835fb8df1679b47acb439 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 13:24:16 +0530 Subject: [PATCH 35/41] chore: write the release notes in terms of what users get The changeset for this release read like a feature inventory. It now leads with the user outcome, keeps the journey example, and folds the docs-domain note into the docs paragraph so the release is one entry. --- .changeset/intent-surface.md | 80 +++++++++++++++++++---------------- .changeset/new-docs-domain.md | 6 --- 2 files changed, 44 insertions(+), 42 deletions(-) delete mode 100644 .changeset/new-docs-domain.md diff --git a/.changeset/intent-surface.md b/.changeset/intent-surface.md index 1ecabdd..1a08cf6 100644 --- a/.changeset/intent-surface.md +++ b/.changeset/intent-surface.md @@ -2,39 +2,47 @@ "@webmcp-stack/codegen": minor --- -**The intent surface: journeys, handshake grouping, budgets, and an agent skill.** - -`generate` now scaffolds a skill file at `.agents/skills/webmcp-tools/SKILL.md` -so your own coding agent learns the naming rules, description budgets, the -execute contract, and the journey pattern. It also emits `title` and -`consequentialHint` from the current WebMCP spec, and accepts an `exposedTo` -config pass-through. - -Handshake endpoints that are one action split across two calls (a -request-upload plus a complete-upload) are detected and merged into one -withheld tool, thread-wiring the first response into the second by exact name. -Fuzzy pairs are skipped with a note. - -Journeys are the new multi-step primitive. `journey.webmcp.ts` is scaffolded -next to the runtime and regenerated every run; `journeys/*.webmcp.ts` files -import `createJourney` from it, the barrel registers them, and `verify` lints -them (submit gate present, step count, no direct `fetch`). `verify` also counts -journey tools in the surface total now. - -The built-in LLM layer is removed: `--llm`, `--suggest`, the provider flow, and -the config options. Rules reach models through the skill file; the CLI never -calls a model. - -**Budgets follow the tool's judgment better.** Chrome's 500/150 character -budgets are authoring guidance, not browser rules (the spec only rejects an -empty description or a name outside 1-128 chars). So generation no longer -silently shortens text a person or a spec wrote: machine-drafted text is -composed to fit, author text is kept in full, and `verify` reports an overrun -as a warning instead of a blocking error. The string trimmer also no longer -returns one character over budget on an unbroken token. - -**Fixes:** the generated `runtime.webmcp.ts` shipped with a string literal -broken across two lines, so it did not parse; it is valid again. Journey steps -no longer claim to be read-only when the tool they compose is a write, and -journeys resolve the WebMCP API at registration time so a late-installed -polyfill still registers them. +**Generate a small, intent-level surface, and teach your own coding agent the rules.** + +This release changes what `@webmcp-stack/codegen` thinks its job is. It still writes safe, typed tools from your OpenAPI spec. What is new is the shape of the surface: instead of a one-to-one mirror of your routes, you get a few intent-level tools, actions your API split across calls, and journeys for goals that take several steps. + +**Journeys: a real goal, not just an endpoint.** Some things a user asks for cannot be one tool call. Booking a ride is find the destination, set the pickup time, then confirm. A journey is a small file in `src/webmcp/journeys/` that composes the tools you already generated into one flow, with a shared draft and a single confirmed write: + +```ts +export const bookRide = createJourney({ + name: "book-ride", + goal: "Book a ride to a destination the user gives", + steps: { + "resolve-destination": { + tool: geocodeAddressTool, + call: (input, signal) => fetchGeocodeAddress(input as never, signal), + store: (place) => ({ destination: place }), + provides: ["destination"], + }, + "set-pickup-time": { + description: "Set when the user wants to be picked up.", + input: { type: "object", properties: { pickupAt: { type: "string" } }, required: ["pickupAt"] }, + provides: ["pickupAt"], + }, + }, + submit: { + description: "Request the ride and show the driver's details.", + build: (draft) => draft as RequestRideInput, + run: executeRequestRide, + }, +}); +``` + +`generate` scaffolds the `journey.webmcp.ts` factory next to the runtime, the barrel registers every journey file it finds, and `verify` lints them. The submit gate and the confirmation live in a file the generator owns and rewrites, so a journey file cannot edit them out. + +**One tool for an action your API split in two.** When a begin/end pair like request-upload plus complete-upload is really one action, the generator merges them into a single withheld tool and threads the first response into the second by exact name. A pair it cannot thread is skipped with a note, never guessed. + +**A skill file for your coding agent.** Every run writes `.agents/skills/webmcp-tools/SKILL.md`, where Claude Code, AGENTS.md, and the generic standard already look. It teaches the naming rules, the description budgets, the execute contract, and the journey pattern, so your own agent extends the surface in the shape this tool expects. The repo also ships an eval harness that runs prompts against a fixture and grades the result, so a wording change can be measured instead of guessed at. + +**Your descriptions are never silently shortened.** Chrome's 500 and 150 character budgets are authoring guidance, not rules the browser enforces. Generation composes its own text to fit, keeps your text and your spec's text in full, and `verify` warns on an overrun instead of blocking. A description that reads well no longer gets cut down to pass a counter. + +**Current with the WebMCP spec.** Generated tools carry `title`, the human label native UIs show, destructive tools carry `consequentialHint`, and the `tools` output accepts `exposedTo` to scope which origins your tools are shared with. + +**The CLI no longer calls a model.** `--llm` and `--suggest` are removed, along with the provider flow and its config options. If you used them, the skill file is the replacement: it teaches your own coding agent the rules, and a plain `generate` never touches the network. + +**Docs.** The journeys guide is rewritten around a relatable example, with new guides for why this matters, what to do after you generate, working with your coding agent, a prompt cookbook, choosing between a tool, a group, and a journey, and testing your tools. The docs site publishes `/llms.txt` and `/llms-full.txt` for agents, and the CLI's docs links point at the new home, https://webmcp.souravinsights.com. \ No newline at end of file diff --git a/.changeset/new-docs-domain.md b/.changeset/new-docs-domain.md deleted file mode 100644 index eae5c87..0000000 --- a/.changeset/new-docs-domain.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -"@webmcp-stack/codegen": patch ---- - -Point the CLI's docs links at the new production site, -https://webmcp.souravinsights.com. From 7f2655b5521193707140acf2c9dcaf0679fbd53d Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 13:35:32 +0530 Subject: [PATCH 36/41] docs: refresh the npm README for the current surface Drop the removed `llm` config, use the withheld wording the tool now uses, and cover the release's headline features: grouping, journeys, the agent skill file, the budget policy, and `verify`. Also replace the vague opening with what the tool actually turns into tools, and add a short "Where this fits" section: it generates from a contract you maintain, not from a codebase scan. Absolute docs links so they work on npm. --- packages/codegen/README.md | 154 +++++++++++++++++++++---------------- 1 file changed, 88 insertions(+), 66 deletions(-) diff --git a/packages/codegen/README.md b/packages/codegen/README.md index a300415..3b147b3 100644 --- a/packages/codegen/README.md +++ b/packages/codegen/README.md @@ -17,11 +17,15 @@ --- -Your spec already knows your tools. One command writes them, wires them into your app, and gets out of the way. Part of [webmcp-stack](https://github.com/SouravInsights/webmcp-stack), the open-source developer stack for [WebMCP](https://github.com/webmachinelearning/webmcp). +Hand-writing a WebMCP tool for every action is a chore, and the descriptions drift from the API they describe. If you already maintain an OpenAPI spec or validation schemas, you already have the source of truth. + +`@webmcp-stack/codegen` turns that source into WebMCP tools as real TypeScript files in your repo. It writes the safe defaults, withholds anything risky until you enable it, scaffolds a skill file so your own coding agent knows the rules, and ships a `verify` command you can gate in CI. + +Part of [webmcp-stack](https://github.com/SouravInsights/webmcp-stack), the open-source developer stack for [WebMCP](https://github.com/webmachinelearning/webmcp). ## Quick start -Zero install, zero config: +No install, no config for the first run: ```bash npx @webmcp-stack/codegen generate @@ -29,104 +33,106 @@ npx @webmcp-stack/codegen generate One run finds your OpenAPI spec (monorepos included), finds the package that is your web app, and: -- **generates working tools** in `src/webmcp/`: reads call your API out of the box, mutations are generated disabled (working code, one uncomment away) -- **filters what shouldn't be a tool**: webhooks skipped, auth and admin endpoints flagged and disabled -- **wires registration into your app** (two additive lines for Next.js and Vite, reported with undo instructions) +- writes one `.webmcp.ts` file per endpoint +- gives read tools working implementations that call your API with the signed-in user's session +- **withholds** writes and destructive tools: the working code is generated, but the tool is not registered until you enable it +- skips webhooks, flags auth and admin endpoints +- wires registration into your app with two additive lines +- scaffolds a skill file for your coding agent, and a journey factory if you want one -Preview first, write nothing: +Preview everything without writing: ```bash npx @webmcp-stack/codegen generate --dry-run ``` -Then start your app, open it in Chrome with `#enable-webmcp-testing`, and ask the agent to use one of your tools. - -## No OpenAPI spec? Use your schemas +Measure the surface before you ship, and gate it in CI: -Most React/Next.js apps don't have one. If your app validates with schemas (zod, valibot, arktype), you can declare agent-facing actions straight from them, and a schema entry can also *refine* an OpenAPI operation (your contract and words, the endpoint's mechanics): +```bash +npx @webmcp-stack/codegen verify +``` -```js -// codegen.config.mjs -import { defineConfig } from "@webmcp-stack/codegen"; -import { openapi, schema } from "@webmcp-stack/codegen/sources"; -import { tools } from "@webmcp-stack/codegen/outputs"; -import { CreateTripInput } from "./src/schemas"; +Browse and test the tools locally: -export default defineConfig({ - sources: [ - openapi({ spec: "./openapi.yaml" }), - schema({ - tools: [ - // operation fuses this with the spec's createTrip endpoint into one tool - { name: "create-trip", schema: CreateTripInput, operation: "createTrip" }, - ], - }), - ], - outputs: [tools({ outDir: "./src/webmcp" })], -}); +```bash +npx @webmcp-stack/codegen dev ``` -Every field gets text an agent can act on: your `.describe()` words stay verbatim, constraints are rendered as plain language ("A number from 30 to 600."), and anything still silent gets a marked draft the audit reports on. +Then open your app in Chrome with `chrome://flags/#enable-webmcp-testing` enabled, and let an agent use a tool. Other browsers work with the [WebMCP polyfill](https://github.com/webmachinelearning/webmcp-polyfill). -Working with a literal `` instead? The `form` output annotates it in place with WebMCP's declarative attributes (`{ name: "add-to-timesheet", schema: Entry, form: "./src/TimesheetForm.tsx" }` plus `form` in `outputs`), so the agent fills the visible form and a human keeps the final click on writes. +## What you get -## Safety is part of generation +**A small, intent-level surface.** Not every endpoint should be a tool. Reads register so you can see the surface; writes and destructive tools stay withheld until you turn them on one at a time. -This is not a dumb API -> WebMCP converter. Giving agents access to application actions is a new security surface, so the generator analyzes what every endpoint actually is: read-only, write, destructive, auth-boundary, or sensitive/PII-related. Every tool gets a safety classification and WebMCP hints, the audit pass runs inside `generate` (errors block, exit codes for CI), and higher-risk tools are generated disabled so you explicitly decide what agents can touch. The goal is that you stay in control of the agent-facing surface instead of blindly exposing every endpoint. +**Handshake grouping.** When a begin/end pair (request an upload, then complete it) is really one action split across two calls, the generator merges them into a single withheld tool and threads the first response into the second by exact name. A pair it cannot thread is skipped with a note, never guessed. -## The dashboard +**Journeys.** For a goal that takes several calls and ends in one confirmed write, you write a small file in `src/webmcp/journeys/` and the generator scaffolds the rest: a shared draft, one tool per step, and a submit gate a human confirms. See [Journeys](https://webmcp.souravinsights.com/docs/journeys). -```bash -npx @webmcp-stack/codegen dev -``` +**A skill file for your coding agent.** Every run writes `.agents/skills/webmcp-tools/SKILL.md`, where Claude Code, AGENTS.md, and the generic standard already look. Your agent follows the same naming, description, and safety rules the generator uses. + +**Safety as part of generation.** Every endpoint is classified read, write, or destructive, corrected when the name disagrees. The audit runs inside `generate`: it names PII in outputs, descriptions that try to instruct the agent, and auth or admin endpoints. Errors block generation and set an exit code for CI; warnings report and continue. -A local control panel for your WebMCP surface, the way Scalar is for APIs or Storybook is for components: browse your tools, inspect and edit metadata, toggle tools on and off, and run any tool directly to check it works. Edits save to `.webmcp-codegen.json` and survive regeneration. Nothing is added to your app. +**Descriptions you can trust.** Constraints become sentences ("A number from 30 to 600."), every description says what the tool returns, and free-text outputs are marked as untrusted. Your own text is kept in full. Chrome's 500 and 150 character budgets are treated as guidance, so `verify` warns on an overrun instead of shortening your words to pass a counter. -## What a generated tool looks like +**Regeneration that never clobbers your code.** The API contract lives above a marker line and regenerates freely; your `execute()` body lives below it and is never touched. Hand-edited generated regions produce a `.new` file to merge, never a silent overwrite. -One file per endpoint, like `delete-pet.webmcp.ts`: +## Where this fits + +This tool generates from a contract, not from a scan of your codebase. That is a deliberate choice, not a hidden limitation: + +- **It needs a source you maintain.** An OpenAPI spec, or the validation schemas your app already uses (zod, valibot, arktype, TypeBox). If neither exists yet, there is nothing good to generate from, and writing that source is the work to do first. +- **The source has to be good.** A tool's description is part of the prompt a model reasons over. A vague or stale contract produces vague or stale tools; what an agent can do is bounded by what your contract actually says. +- **One thing done well.** A generator that claims to work on any codebase, by guessing intent from source code, tends to work well nowhere. This one asks for a contract and rewards you for keeping it current. + +If you have an OpenAPI spec or maintained schemas, you are exactly who this is for. If you do not, the `schema` source lets you declare tools from schemas you write by hand in the meantime. + +## A generated tool + +One file per endpoint, like `create-trip.webmcp.ts`: ```ts // --- webmcp-codegen: generated. Do not edit this region. --- -export const deletePetInputSchema = { /* derived from your spec */ }; -export type DeletePetInput = { id: string }; -export async function registerDeletePet(signal?: AbortSignal) { - // Registers the tool; mutations ask the user to confirm, always. -} -// --- webmcp-codegen: end generated. Your code below survives regeneration. --- - -export async function executeDeletePet(input: DeletePetInput) { - // This tool starts disabled: it changes things. To enable it, delete the - // line below and uncomment the code. - return toolDisabled("delete-pet.webmcp.ts"); +export const createTripTool = { + name: "create-trip", + title: "Create Trip", + description: "Create a new trip. Returns the trip.", + inputSchema: createTripInputSchema, + annotations: { + readOnlyHint: false, + untrustedContentHint: true, + consequentialHint: false, + }, +}; - // const data = await callApi(`/pets/${input.id}`, { method: "DELETE" }); - // return toolResult(data); +// Journeys and your own code compose this raw caller; executeCreateTrip wraps it. +export async function fetchCreateTrip(input: CreateTripInput, signal?: AbortSignal) { + const data = await callApi("/v1/trips", { method: "POST", body: { ... }, signal }); + return data; } -``` -- **Real files in your repo**: readable, editable, no runtime dependency on this package -- **Working implementations**: path params, query strings, and JSON bodies built from the spec; session cookies included -- **Safety classification on every tool**: read/write/destructive, with WebMCP hints computed -- **An audit pass built into `generate`**: PII-in-response warnings, agent-instructing description linting, auth-boundary checks; errors block generation (exit codes for CI) -- **Regeneration never clobbers your code**: the contract regenerates above the marker, your code below it survives; hand-edited generated regions produce a `.new` file, never a silent overwrite +// Withheld, so the registration is generated but commented out, confirmation included: +// const confirmed = await requestUserConfirmation("Allow the agent to: Create a new trip..."); +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- +``` ## CLI | Command | What it does | |---|---| -| `webmcp-codegen generate` | Generate/update tools, wire registration (audit runs by default) | +| `webmcp-codegen generate` | Generate and update tools, wire registration (the audit runs by default) | | `generate --dry-run` | Preview everything, write nothing | | `generate --watch` | Re-generate when source files change | | `generate --force` | Write files even when the audit reports errors | | `generate --spec PATH` / `--out DIR` | Overrides without a config file | +| `webmcp-codegen verify` | Score the surface against the standard; exits 1 on errors | +| `verify --url URL` | Also check a deployed page serves an origin trial token | | `webmcp-codegen dev` | Open the tools dashboard (`--port N` to change the port) | | `webmcp-codegen init` | Write `codegen.config.mjs` for full control (needs the package installed) | ## Config -Structure lives in `codegen.config.mjs` (code); remembered choices and per-tool -overrides live in `.webmcp-codegen.json` (data, safe with npx, commit it). +Structure lives in `codegen.config.mjs` (code). Remembered choices and per-tool +overrides live in `.webmcp-codegen.json` (data, safe with npx, meant to be committed). ```js // codegen.config.mjs @@ -138,18 +144,34 @@ export default defineConfig({ sources: [openapi({ spec: "./openapi.yaml" })], outputs: [tools({ outDir: "./src/webmcp" })], safety: { - piiFields: ["internalId"], // extend the built-in PII heuristics - exclude: ["internal"], // skip tools by name or route substring + piiFields: ["internalId"], // extend the built-in PII heuristics + exclude: ["internal"], // skip tools by name or route substring }, - // llm: { apiKey: "..." }, // opt-in advisory layer: drafts and suggestions - // rendered as proposals, never applied, never blocking }); ``` +No OpenAPI spec? Declare tools from the schemas you already use, and a schema +entry can also refine an endpoint from your spec: + +```js +// import { schema } from "@webmcp-stack/codegen/sources"; +// import { CreateTripInput } from "./src/schemas"; + +sources: [ + openapi({ spec: "./openapi.yaml" }), + schema({ + tools: [{ name: "create-trip", schema: CreateTripInput, operation: "createTrip" }], + }), +], +``` + +Annotating a literal `` instead? The `form` output wires WebMCP's +declarative attributes onto it, so an agent fills the controls a person can see. + ## Requirements - Node.js 20 or newer -- To *use* the generated tools in a browser: enable `chrome://flags/#enable-webmcp-testing` for local development (Chrome 149+, Edge 150+); production pages join the WebMCP origin trial - or use the WebMCP polyfill +- To *use* the generated tools in a browser: enable `chrome://flags/#enable-webmcp-testing` for local development (Chrome 149+, Edge 150+). Production pages join the WebMCP origin trial, or use the polyfill. ## License From b4dcdc12ff7067d0c79443762a07b5fb8b6a749d Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 13:51:24 +0530 Subject: [PATCH 37/41] docs: say who the tool is for, and why it generates from contracts Add a short "Who this is for" on the why page, a matching bullet on the docs landing, and a line in the root README. The point, in one place: it generates from a contract you maintain, not from a scan of your code, so a vague contract makes vague tools. That is a deliberate trade, not a hidden gap. --- README.md | 2 ++ site/content/docs/index.mdx | 4 ++++ site/content/docs/why-agent-tools.mdx | 12 ++++++++++++ 3 files changed, 18 insertions(+) diff --git a/README.md b/README.md index d4d72d5..1fae9b9 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,8 @@ npx @webmcp-stack/codegen generate No install, no config for the first run. It detects your app, writes one `.webmcp.ts` file per tool, and adds the registration call to your entry file (additive edits, always reported; if it can't find the entry point, it prints the two lines for you to paste). +It generates from the contract you maintain, not by scanning your app and guessing at intent. A vague spec or schema makes vague tools, so the source is the part worth getting right. If you have an OpenAPI spec or maintained schemas, this is built for you; if not, writing that contract is the first step, and the `schema` source lets you declare tools by hand in the meantime. + ```bash npx @webmcp-stack/codegen dev # local dashboard: browse, edit, toggle, and test tools npx @webmcp-stack/codegen verify # check the tool set against the standard; exits 1 on errors diff --git a/site/content/docs/index.mdx b/site/content/docs/index.mdx index 49411b7..5a2be0e 100644 --- a/site/content/docs/index.mdx +++ b/site/content/docs/index.mdx @@ -63,6 +63,10 @@ silent overwrite. Being clear about the edges is part of using it well: +- It does not work without a contract you maintain. It generates from an + OpenAPI spec or the validation schemas your app already uses, not by scanning + your code, and a vague contract makes vague tools. This is on purpose. See + [Who this is for](/docs/why-agent-tools#who-this-is-for). - It does not decide your product's intents. That is your knowledge, or your agent's, guided by the skill file. - It does not make an unsafe API safe. Every tool calls your real endpoint, so diff --git a/site/content/docs/why-agent-tools.mdx b/site/content/docs/why-agent-tools.mdx index 9d10134..604eaaf 100644 --- a/site/content/docs/why-agent-tools.mdx +++ b/site/content/docs/why-agent-tools.mdx @@ -64,6 +64,18 @@ your validation schemas, into real TypeScript files in your repo: - It audits the result, in plain language, and `verify` fails CI when something is wrong. +## Who this is for + +This tool generates from a contract you maintain: an OpenAPI spec, or the +validation schemas your app already uses. If you have one, it turns that into a +safe agent surface. If you do not, writing it is the work to do first. + +That is deliberate. It does not read your code and guess at intent, because a +tool is only as good as the source behind it: a vague or stale contract makes +vague or stale tools. A generator that claims to work on any codebase, by +inferring meaning from source, tends to work well nowhere. We would rather do +this one thing well. + ## What it does not do Being clear about the edges is part of using it well: From a59271abb7635d043b48929d9646da6c2a12e5e7 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 14:14:06 +0530 Subject: [PATCH 38/41] docs(site): render Mermaid diagrams Wire up Fumadocs' Mermaid support: `remarkMdxMermaid` turns a ```mermaid block into ``, and the client component renders it with the official renderer. Mermaid is imported on demand, so pages without a diagram never load it. --- pnpm-lock.yaml | 863 ++++++++++++++++++++++++++++++++++++ site/components/mdx.tsx | 2 + site/components/mermaid.tsx | 65 +++ site/package.json | 2 + site/source.config.ts | 4 +- 5 files changed, 935 insertions(+), 1 deletion(-) create mode 100644 site/components/mermaid.tsx diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9cbb517..db306e7 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -96,9 +96,15 @@ importers: lucide-react: specifier: ^1.34.0 version: 1.34.0(react@19.2.8) + mermaid: + specifier: ^12.0.0 + version: 12.0.0 next: specifier: ^15.3.0 version: 15.5.24(@types/node@22.20.1)(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + next-themes: + specifier: ^0.4.6 + version: 0.4.6(react-dom@19.2.8(react@19.2.8))(react@19.2.8) react: specifier: ^19.0.0 version: 19.2.8 @@ -137,6 +143,9 @@ packages: resolution: {integrity: sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw==} engines: {node: '>=10'} + '@antfu/install-pkg@2.0.1': + resolution: {integrity: sha512-iCKVQcIC0e3oDxEfs3SHQGW+ovhBMZmS1TE+bTk50rVyMCBmCfClv7Qi3HQKlumYwvjb/iIMeWCW2i67q6kFfQ==} + '@biomejs/biome@2.5.10': resolution: {integrity: sha512-WRKXARA3kTuiV5sxqTpobJ/I0MVd4vk3pOL6wnp5az4LntFIhWTj1RWZq3DI9PCEN3lXcqy7p5aqUHzvq8AXyQ==} engines: {node: '>=14.21.3'} @@ -205,6 +214,9 @@ packages: commander: optional: true + '@braintree/sanitize-url@7.1.2': + resolution: {integrity: sha512-jigsZK+sMF/cuiB7sERuo9V7N9jx+dhmHHnQyDSVdpZwVutaBu7WvNYqMDLSgFgfB30n452TP3vjDAvFC973mA==} + '@changesets/apply-release-plan@8.0.0': resolution: {integrity: sha512-kUd2pbf1w5/AYmBMb0Tt+rkIPCjFJdT0SZMrkOjJT/WV/QbtmvkyB5jkV0oaNPheavprZk+SfUiozUti7TIL2w==} engines: {node: ^22.11 || ^24 || >=26} @@ -266,6 +278,21 @@ packages: resolution: {integrity: sha512-q/ThtP9gcnEP6xlv7LrY26C2mHqdGBti/MmOVngt/M/oIGYkssmQGxPK9WzBNt2juVcH/vml2WQ+ra8LXYOTaA==} engines: {node: ^22.11 || ^24 || >=26} + '@chevrotain/cst-dts-gen@11.1.2': + resolution: {integrity: sha512-XTsjvDVB5nDZBQB8o0o/0ozNelQtn2KrUVteIHSlPd2VAV2utEb6JzyCJaJ8tGxACR4RiBNWy5uYUHX2eji88Q==} + + '@chevrotain/gast@11.1.2': + resolution: {integrity: sha512-Z9zfXR5jNZb1Hlsd/p+4XWeUFugrHirq36bKzPWDSIacV+GPSVXdk+ahVWZTwjhNwofAWg/sZg58fyucKSQx5g==} + + '@chevrotain/regexp-to-ast@11.1.2': + resolution: {integrity: sha512-nMU3Uj8naWer7xpZTYJdxbAs6RIv/dxYzkYU8GSwgUtcAAlzjcPfX1w+RKRcYG8POlzMeayOQ/znfwxEGo5ulw==} + + '@chevrotain/types@11.1.2': + resolution: {integrity: sha512-U+HFai5+zmJCkK86QsaJtoITlboZHBqrVketcO2ROv865xfCMSFpELQoz1GkX5GzME8pTa+3kbKrZHQtI0gdbw==} + + '@chevrotain/utils@11.1.2': + resolution: {integrity: sha512-4mudFAQ6H+MqBTfqLmU7G1ZwRzCLfJEooL/fsF6rCX5eePMbGhoy5n4g+G4vlh2muDcsCTJtL+uKbOzWxs5LHA==} + '@clack/core@1.4.3': resolution: {integrity: sha512-/kr3UWNtdJfxZtPgDqUOmG2pvwlmcLGheex5yiZKdwbzZJxhV+HMNR9QNmyY5cGwTNV6LrR7Jtp+KjhUAP1qBQ==} engines: {node: '>= 20.12.0'} @@ -611,6 +638,12 @@ packages: '@formatjs/intl-localematcher@0.6.2': resolution: {integrity: sha512-XOMO2Hupl0wdd172Y06h6kLpBz6Dv+J4okPLl4LPtzbr8f66WbIoy4ev98EBuZ6ZK4h5ydTN6XneT4QVpD7cdA==} + '@iconify/types@2.0.0': + resolution: {integrity: sha512-+wluvCrRhXrhyOmRDJ3q8mux9JkKy5SJ/v8ol2tu4FVjyYvtEzkc/3pK15ET6RKg4b4w4BmTk1+gsCUhf21Ykg==} + + '@iconify/utils@3.1.7': + resolution: {integrity: sha512-JZHlwdID+dy+lTgbYC8NEC4zeugqeYsc6jewvzb4c58kHauJn+X7rNwQjxz5p2qSjqaEeQoLkCIQ9v/H4PK0/w==} + '@img/colour@1.1.0': resolution: {integrity: sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==} engines: {node: '>=18'} @@ -788,6 +821,10 @@ packages: '@mdx-js/mdx@3.1.1': resolution: {integrity: sha512-f6ZO2ifpwAQIpzGWaBQT2TXxPv6z3RBzQKpVftEWN78Vl/YweF1uwussDx8ECAXVtr3Rs89fKyG9YlzUs9DyGQ==} + '@mermaid-js/parser@2.0.0': + resolution: {integrity: sha512-K8BeapFUfrfxbRAUQAG5oBOCwo1+bNWzaVnPxTCutkfrTBn6T/j91FIhqxWJ32SUeQ9T3iy1zcmPZ5ROZEvDrg==} + engines: {node: '>=22.12.0'} + '@napi-rs/lzma-linux-x64-gnu@1.5.1': resolution: {integrity: sha512-oTXEIha4SsuXdTA4Iyskj0kpdx2yVXdhd75c2v3xGrHFfVMsbhTPZU/nMPL4sWKo4pBHm3aucLaqGlF696dTyQ==} engines: {node: ^22.20 || ^24.12 || >=25} @@ -1491,6 +1528,99 @@ packages: '@types/chai@5.2.3': resolution: {integrity: sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==} + '@types/d3-array@3.2.2': + resolution: {integrity: sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw==} + + '@types/d3-axis@3.0.6': + resolution: {integrity: sha512-pYeijfZuBd87T0hGn0FO1vQ/cgLk6E1ALJjfkC0oJ8cbwkZl3TpgS8bVBLZN+2jjGgg38epgxb2zmoGtSfvgMw==} + + '@types/d3-brush@3.0.6': + resolution: {integrity: sha512-nH60IZNNxEcrh6L1ZSMNA28rj27ut/2ZmI3r96Zd+1jrZD++zD3LsMIjWlvg4AYrHn/Pqz4CF3veCxGjtbqt7A==} + + '@types/d3-chord@3.0.6': + resolution: {integrity: sha512-LFYWWd8nwfwEmTZG9PfQxd17HbNPksHBiJHaKuY1XeqscXacsS2tyoo6OdRsjf+NQYeB6XrNL3a25E3gH69lcg==} + + '@types/d3-color@3.1.3': + resolution: {integrity: sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==} + + '@types/d3-contour@3.0.6': + resolution: {integrity: sha512-BjzLgXGnCWjUSYGfH1cpdo41/hgdWETu4YxpezoztawmqsvCeep+8QGfiY6YbDvfgHz/DkjeIkkZVJavB4a3rg==} + + '@types/d3-delaunay@6.0.4': + resolution: {integrity: sha512-ZMaSKu4THYCU6sV64Lhg6qjf1orxBthaC161plr5KuPHo3CNm8DTHiLw/5Eq2b6TsNP0W0iJrUOFscY6Q450Hw==} + + '@types/d3-dispatch@3.0.7': + resolution: {integrity: sha512-5o9OIAdKkhN1QItV2oqaE5KMIiXAvDWBDPrD85e58Qlz1c1kI/J0NcqbEG88CoTwJrYe7ntUCVfeUl2UJKbWgA==} + + '@types/d3-drag@3.0.7': + resolution: {integrity: sha512-HE3jVKlzU9AaMazNufooRJ5ZpWmLIoc90A37WU2JMmeq28w1FQqCZswHZ3xR+SuxYftzHq6WU6KJHvqxKzTxxQ==} + + '@types/d3-dsv@3.0.7': + resolution: {integrity: sha512-n6QBF9/+XASqcKK6waudgL0pf/S5XHPPI8APyMLLUHd8NqouBGLsU8MgtO7NINGtPBtk9Kko/W4ea0oAspwh9g==} + + '@types/d3-ease@3.0.2': + resolution: {integrity: sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==} + + '@types/d3-fetch@3.0.7': + resolution: {integrity: sha512-fTAfNmxSb9SOWNB9IoG5c8Hg6R+AzUHDRlsXsDZsNp6sxAEOP0tkP3gKkNSO/qmHPoBFTxNrjDprVHDQDvo5aA==} + + '@types/d3-force@3.0.10': + resolution: {integrity: sha512-ZYeSaCF3p73RdOKcjj+swRlZfnYpK1EbaDiYICEEp5Q6sUiqFaFQ9qgoshp5CzIyyb/yD09kD9o2zEltCexlgw==} + + '@types/d3-format@3.0.4': + resolution: {integrity: sha512-fALi2aI6shfg7vM5KiR1wNJnZ7r6UuggVqtDA+xiEdPZQwy/trcQaHnwShLuLdta2rTymCNpxYTiMZX/e09F4g==} + + '@types/d3-geo@3.1.1': + resolution: {integrity: sha512-65Emv9fQiQQqphLlRkuQ5ypPsOmWPhtBGCMv61JDPEPMvsx+gzhGf74yw1a78xFKPj6zw4AgQICJoQv0vK9M2w==} + + '@types/d3-hierarchy@3.1.7': + resolution: {integrity: sha512-tJFtNoYBtRtkNysX1Xq4sxtjK8YgoWUNpIiUee0/jHGRwqvzYxkq0hGVbbOGSz+JgFxxRu4K8nb3YpG3CMARtg==} + + '@types/d3-interpolate@3.0.4': + resolution: {integrity: sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==} + + '@types/d3-path@3.1.1': + resolution: {integrity: sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==} + + '@types/d3-polygon@3.0.2': + resolution: {integrity: sha512-ZuWOtMaHCkN9xoeEMr1ubW2nGWsp4nIql+OPQRstu4ypeZ+zk3YKqQT0CXVe/PYqrKpZAi+J9mTs05TKwjXSRA==} + + '@types/d3-quadtree@3.0.6': + resolution: {integrity: sha512-oUzyO1/Zm6rsxKRHA1vH0NEDG58HrT5icx/azi9MF1TWdtttWl0UIUsjEQBBh+SIkrpd21ZjEv7ptxWys1ncsg==} + + '@types/d3-random@3.0.4': + resolution: {integrity: sha512-UHYId5WTCx4L4YNel7NU00XUXXgvgpgZOvp10PuvsQENjMDXhh2RyFc0KBjO7B45ne4Ha1yVH7ii0vnzKkuzWA==} + + '@types/d3-scale-chromatic@3.1.0': + resolution: {integrity: sha512-iWMJgwkK7yTRmWqRB5plb1kadXyQ5Sj8V/zYlFGMUBbIPKQScw+Dku9cAAMgJG+z5GYDoMjWGLVOvjghDEFnKQ==} + + '@types/d3-scale@4.0.9': + resolution: {integrity: sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==} + + '@types/d3-selection@3.0.11': + resolution: {integrity: sha512-bhAXu23DJWsrI45xafYpkQ4NtcKMwWnAC/vKrd2l+nxMFuvOT3XMYTIj2opv8vq8AO5Yh7Qac/nSeP/3zjTK0w==} + + '@types/d3-shape@3.2.0': + resolution: {integrity: sha512-kVd74ta9eof3eJOvbNd1vGKS/XERRyQbT26Og63hIsvDO84cjD5gEOhsXf26w3FSoNlPVz84DOFcKv/oou+fMw==} + + '@types/d3-time-format@4.0.3': + resolution: {integrity: sha512-5xg9rC+wWL8kdDj153qZcsJ0FWiFt0J5RB6LYUNZjwSnesfblqrI/bJ1wBdJ8OQfncgbJG5+2F+qfqnqyzYxyg==} + + '@types/d3-time@3.0.4': + resolution: {integrity: sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==} + + '@types/d3-timer@3.0.2': + resolution: {integrity: sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==} + + '@types/d3-transition@3.0.9': + resolution: {integrity: sha512-uZS5shfxzO3rGlu0cC3bjmMFKsXv+SmZZcgp0KD22ts4uGXp5EVYGzu/0YdwZeKmddhcAccYtREJKkPfXkZuCg==} + + '@types/d3-zoom@3.0.8': + resolution: {integrity: sha512-iqMC4/YlFCSlO8+2Ii1GGGliCAY4XdeG748w5vQUbevlbDu0zSjH/+jojorQVBK/se0j6DUFNPBGSqD3YWYnDw==} + + '@types/d3@7.4.3': + resolution: {integrity: sha512-lZXZ9ckh5R8uiFVt8ogUNf+pIrK4EsWrx2Np75WvF/eTpJ0FMHNhjXk8CKEx/+gpHbNQyJWehbFaTvqmHWB3ww==} + '@types/debug@4.1.13': resolution: {integrity: sha512-KSVgmQmzMwPlmtljOomayoR89W4FynCAi3E8PPs7vmDVPe84hT+vGPKkJfThkmXs0x0jAaa9U8uW8bbfyS2fWw==} @@ -1503,6 +1633,9 @@ packages: '@types/estree@1.0.9': resolution: {integrity: sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg==} + '@types/geojson@7946.0.16': + resolution: {integrity: sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg==} + '@types/hast@3.0.5': resolution: {integrity: sha512-rp/ezSWaD1m44dPKICGhiskI13nVr7qTloFwDa/IYkhhf5nzwP+zIQcIJh3WIFSBOy/H1PzB40jPjMDksN4F+g==} @@ -1532,6 +1665,9 @@ packages: '@types/react@19.2.18': resolution: {integrity: sha512-AnzbBERsrLKtk2XSfTbYRLjQPdy116Sty4q+T+Bp3IC4l6jNBvreVPAHmpq9qhXQM7CXZPjLVmGMw9sy+hxQ3w==} + '@types/trusted-types@2.0.7': + resolution: {integrity: sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw==} + '@types/unist@2.0.11': resolution: {integrity: sha512-CmBKiL6NNo/OqgmMn95Fk9Whlp2mtvIv+KNpQKN2F4SjvrEesubTRWGYSg+BnWZOnlCaSTU1sMpsBOzgbYhnsA==} @@ -1547,6 +1683,9 @@ packages: '@ungap/structured-clone@1.3.4': resolution: {integrity: sha512-JL+CF0GeLHyPWI0rXu7UnxgiuOm9UQWzadi0OYOJNhNO2q6EZElpwlgXkNkfU1PzANDHq3YcwKVZprdvS+BrbQ==} + '@upsetjs/venn.js@2.0.0': + resolution: {integrity: sha512-WbBhLrooyePuQ1VZxrJjtLvTc4NVfpOyKx0sKqioq9bX1C1m7Jgykkn8gLrtwumBioXIqam8DLxp88Adbue6Hw==} + '@vitest/expect@4.1.11': resolution: {integrity: sha512-VX2x5vNJXET47KAFzwERI+KRMtTTCSWTfSMKsW7JsUsXV4psq++e3DvZpuTDOpHcxytiDs6p2nhVb2tVDiiUYw==} @@ -1660,6 +1799,9 @@ packages: character-reference-invalid@2.0.1: resolution: {integrity: sha512-iBZ4F4wRbyORVsu0jPV7gXkOsGYjGHPmAyv+HiHG8gi5PtC9KI2j1+v8/tlibRvjoWX027ypmG/n0HtO5t7unw==} + chevrotain@11.1.2: + resolution: {integrity: sha512-opLQzEVriiH1uUQ4Kctsd49bRoFDXGGSC4GUqj7pGyxM3RehRhvTlZJc1FL/Flew2p5uwxa1tUDWKzI4wNM8pg==} + chokidar@4.0.3: resolution: {integrity: sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==} engines: {node: '>= 14.16.0'} @@ -1695,6 +1837,14 @@ packages: resolution: {integrity: sha512-NOKm8xhkzAjzFx8B2v5OAHT+u5pRQc2UCa2Vq9jYL/31o2wi9mxBA7LIFs3sV5VSC49z6pEhfbMULvShKj26WA==} engines: {node: '>= 6'} + commander@7.2.0: + resolution: {integrity: sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw==} + engines: {node: '>= 10'} + + commander@8.3.0: + resolution: {integrity: sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww==} + engines: {node: '>= 12'} + compute-scroll-into-view@3.1.1: resolution: {integrity: sha512-VRhuHOLoKYOy4UbilLbUzbYg93XLjv2PncJC50EuTWPA3gaja1UjBsUP/D/9/juV3vQFr6XBEzn9KCAHdUvOHw==} @@ -1712,6 +1862,12 @@ packages: convert-source-map@2.0.0: resolution: {integrity: sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg==} + cose-base@1.0.3: + resolution: {integrity: sha512-s9whTXInMSgAp/NVXVNuVxVKzGH2qck3aQlVHxDCdAEPgtMKwc4Wq6/QKhgdEdgbLSi9rBTAcPoRa6JpiG4ksg==} + + cose-base@2.2.0: + resolution: {integrity: sha512-AzlgcsCbUMymkADOJtQm3wO9S3ltPfYOFD5033keQn9NJzIbtnZj+UdBJe7DYml/8TdbtHJW3j58SOnKhWY/5g==} + cssesc@3.0.0: resolution: {integrity: sha512-/Tb/JcjK111nNScGob5MNtsntNM1aCNUDipB/TkwZFhyDrrE47SOx/18wF2bbjgc3ZzCSKW1T5nt5EbFoAz/Vg==} engines: {node: '>=4'} @@ -1720,9 +1876,168 @@ packages: csstype@3.2.3: resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==} + cytoscape-cose-bilkent@4.1.0: + resolution: {integrity: sha512-wgQlVIUJF13Quxiv5e1gstZ08rnZj2XaLHGoFMYXz7SkNfCDOOteKBE6SYRfA9WxxI/iBc3ajfDoc6hb/MRAHQ==} + peerDependencies: + cytoscape: ^3.2.0 + + cytoscape-fcose@2.2.0: + resolution: {integrity: sha512-ki1/VuRIHFCzxWNrsshHYPs6L7TvLu3DL+TyIGEsRcvVERmxokbf5Gdk7mFxZnTdiGtnA4cfSmjZJMviqSuZrQ==} + peerDependencies: + cytoscape: ^3.2.0 + + cytoscape@3.34.3: + resolution: {integrity: sha512-yfYGhRcGAntq6YBD583j4n0Eg3jIxvWmZtz/5uz9UYkeIStSlMxuUja+ec5j3iBD8nv1rwaOAYMW09tBdkSeaQ==} + engines: {node: '>=0.10'} + + d3-array@2.12.1: + resolution: {integrity: sha512-B0ErZK/66mHtEsR1TkPEEkwdy+WDesimkM5gpZr5Dsg54BiTA5RXtYW5qTLIAcekaS9xfZrzBLF/OAkB3Qn1YQ==} + + d3-array@3.2.4: + resolution: {integrity: sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==} + engines: {node: '>=12'} + + d3-axis@3.0.0: + resolution: {integrity: sha512-IH5tgjV4jE/GhHkRV0HiVYPDtvfjHQlQfJHs0usq7M30XcSBvOotpmH1IgkcXsO/5gEQZD43B//fc7SRT5S+xw==} + engines: {node: '>=12'} + + d3-brush@3.0.0: + resolution: {integrity: sha512-ALnjWlVYkXsVIGlOsuWH1+3udkYFI48Ljihfnh8FZPF2QS9o+PzGLBslO0PjzVoHLZ2KCVgAM8NVkXPJB2aNnQ==} + engines: {node: '>=12'} + + d3-chord@3.0.1: + resolution: {integrity: sha512-VE5S6TNa+j8msksl7HwjxMHDM2yNK3XCkusIlpX5kwauBfXuyLAtNg9jCp/iHH61tgI4sb6R/EIMWCqEIdjT/g==} + engines: {node: '>=12'} + + d3-color@3.1.0: + resolution: {integrity: sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==} + engines: {node: '>=12'} + + d3-contour@4.0.2: + resolution: {integrity: sha512-4EzFTRIikzs47RGmdxbeUvLWtGedDUNkTcmzoeyg4sP/dvCexO47AaQL7VKy/gul85TOxw+IBgA8US2xwbToNA==} + engines: {node: '>=12'} + + d3-delaunay@6.0.4: + resolution: {integrity: sha512-mdjtIZ1XLAM8bm/hx3WwjfHt6Sggek7qH043O8KEjDXN40xi3vx/6pYSVTwLjEgiXQTbvaouWKynLBiUZ6SK6A==} + engines: {node: '>=12'} + + d3-dispatch@3.0.1: + resolution: {integrity: sha512-rzUyPU/S7rwUflMyLc1ETDeBj0NRuHKKAcvukozwhshr6g6c5d8zh4c2gQjY2bZ0dXeGLWc1PF174P2tVvKhfg==} + engines: {node: '>=12'} + + d3-drag@3.0.0: + resolution: {integrity: sha512-pWbUJLdETVA8lQNJecMxoXfH6x+mO2UQo8rSmZ+QqxcbyA3hfeprFgIT//HW2nlHChWeIIMwS2Fq+gEARkhTkg==} + engines: {node: '>=12'} + + d3-dsv@3.0.1: + resolution: {integrity: sha512-UG6OvdI5afDIFP9w4G0mNq50dSOsXHJaRE8arAS5o9ApWnIElp8GZw1Dun8vP8OyHOZ/QJUKUJwxiiCCnUwm+Q==} + engines: {node: '>=12'} + hasBin: true + + d3-ease@3.0.1: + resolution: {integrity: sha512-wR/XK3D3XcLIZwpbvQwQ5fK+8Ykds1ip7A2Txe0yxncXSdq1L9skcG7blcedkOX+ZcgxGAmLX1FrRGbADwzi0w==} + engines: {node: '>=12'} + + d3-fetch@3.0.1: + resolution: {integrity: sha512-kpkQIM20n3oLVBKGg6oHrUchHM3xODkTzjMoj7aWQFq5QEM+R6E4WkzT5+tojDY7yjez8KgCBRoj4aEr99Fdqw==} + engines: {node: '>=12'} + + d3-force@3.0.0: + resolution: {integrity: sha512-zxV/SsA+U4yte8051P4ECydjD/S+qeYtnaIyAs9tgHCqfguma/aAQDjo85A9Z6EKhBirHRJHXIgJUlffT4wdLg==} + engines: {node: '>=12'} + + d3-format@3.1.2: + resolution: {integrity: sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==} + engines: {node: '>=12'} + + d3-geo@3.1.1: + resolution: {integrity: sha512-637ln3gXKXOwhalDzinUgY83KzNWZRKbYubaG+fGVuc/dxO64RRljtCTnf5ecMyE1RIdtqpkVcq0IbtU2S8j2Q==} + engines: {node: '>=12'} + + d3-hierarchy@3.1.2: + resolution: {integrity: sha512-FX/9frcub54beBdugHjDCdikxThEqjnR93Qt7PvQTOHxyiNCAlvMrHhclk3cD5VeAaq9fxmfRp+CnWw9rEMBuA==} + engines: {node: '>=12'} + + d3-interpolate@3.0.1: + resolution: {integrity: sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==} + engines: {node: '>=12'} + + d3-path@1.0.9: + resolution: {integrity: sha512-VLaYcn81dtHVTjEHd8B+pbe9yHWpXKZUC87PzoFmsFrJqgFwDe/qxfp5MlfsfM1V5E/iVt0MmEbWQ7FVIXh/bg==} + + d3-path@3.1.0: + resolution: {integrity: sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==} + engines: {node: '>=12'} + + d3-polygon@3.0.1: + resolution: {integrity: sha512-3vbA7vXYwfe1SYhED++fPUQlWSYTTGmFmQiany/gdbiWgU/iEyQzyymwL9SkJjFFuCS4902BSzewVGsHHmHtXg==} + engines: {node: '>=12'} + + d3-quadtree@3.0.1: + resolution: {integrity: sha512-04xDrxQTDTCFwP5H6hRhsRcb9xxv2RzkcsygFzmkSIOJy3PeRJP7sNk3VRIbKXcog561P9oU0/rVH6vDROAgUw==} + engines: {node: '>=12'} + + d3-random@3.0.1: + resolution: {integrity: sha512-FXMe9GfxTxqd5D6jFsQ+DJ8BJS4E/fT5mqqdjovykEB2oFbTMDVdg1MGFxfQW+FBOGoB++k8swBrgwSHT1cUXQ==} + engines: {node: '>=12'} + + d3-sankey@0.12.3: + resolution: {integrity: sha512-nQhsBRmM19Ax5xEIPLMY9ZmJ/cDvd1BG3UVvt5h3WRxKg5zGRbvnteTyWAbzeSvlh3tW7ZEmq4VwR5mB3tutmQ==} + + d3-scale-chromatic@3.1.0: + resolution: {integrity: sha512-A3s5PWiZ9YCXFye1o246KoscMWqf8BsD9eRiJ3He7C9OBaxKhAd5TFCdEx/7VbKtxxTsu//1mMJFrEt572cEyQ==} + engines: {node: '>=12'} + + d3-scale@4.0.2: + resolution: {integrity: sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==} + engines: {node: '>=12'} + + d3-selection@3.0.0: + resolution: {integrity: sha512-fmTRWbNMmsmWq6xJV8D19U/gw/bwrHfNXxrIN+HfZgnzqTHp9jOmKMhsTUjXOJnZOdZY9Q28y4yebKzqDKlxlQ==} + engines: {node: '>=12'} + + d3-shape@1.3.7: + resolution: {integrity: sha512-EUkvKjqPFUAZyOlhY5gzCxCeI0Aep04LwIRpsZ/mLFelJiUfnK56jo5JMDSE7yyP2kLSb6LtF+S5chMk7uqPqw==} + + d3-shape@3.2.0: + resolution: {integrity: sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==} + engines: {node: '>=12'} + + d3-time-format@4.1.0: + resolution: {integrity: sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==} + engines: {node: '>=12'} + + d3-time@3.1.0: + resolution: {integrity: sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==} + engines: {node: '>=12'} + + d3-timer@3.0.1: + resolution: {integrity: sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==} + engines: {node: '>=12'} + + d3-transition@3.0.1: + resolution: {integrity: sha512-ApKvfjsSR6tg06xrL434C0WydLr7JewBB3V+/39RMHsaXTOG0zmt/OAXeng5M5LBm0ojmxJrpomQVZ1aPvBL4w==} + engines: {node: '>=12'} + peerDependencies: + d3-selection: 2 - 3 + + d3-zoom@3.0.0: + resolution: {integrity: sha512-b8AmV3kfQaqWAuacbPuNbL6vahnOJflOhexLzMMNLga62+/nh0JzvJ0aO/5a5MVgUFGS7Hu1P9P03o3fJkDCyw==} + engines: {node: '>=12'} + + d3@7.9.0: + resolution: {integrity: sha512-e1U46jVP+w7Iut8Jt8ri1YsPOvFpg46k+K8TpCb0P+zjCkjkPnV7WzfDJzMHy1LnA+wj5pLT1wjO901gLXeEhA==} + engines: {node: '>=12'} + + dagre-d3-es@7.0.14: + resolution: {integrity: sha512-P4rFMVq9ESWqmOgK+dlXvOtLwYg0i7u0HBGJER0LZDJT2VHIPAMZ/riPxqJceWMStH5+E61QxFra9kIS3AqdMg==} + dateformat@4.6.3: resolution: {integrity: sha512-2P0p0pFGzHS5EMnhdxQi7aJN+iMheud0UhG4dlE1DLAlvL8JHjJJTX/CSm4JXwV0Ka5nGk3zC5mcb5bUQUxxMA==} + dayjs@1.11.23: + resolution: {integrity: sha512-QDTCU0M0MxR3hQfnlDJfwekQiaanm1ubOD231u73WBckQ/fsamwRLiE2GBz6D3a/xF1NgfiDLJjXBa1hYOYTtQ==} + debug@4.4.3: resolution: {integrity: sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==} engines: {node: '>=6.0'} @@ -1739,6 +2054,9 @@ packages: resolution: {integrity: sha512-cZTYKFWspt9jZsMscWo8sc/5lbPC9Q0N5nBLgb+Yd915iL3udB1uFgS3B8YCx66UVHq018DAVFoee7x+gxggeA==} engines: {node: '>=0.10.0'} + delaunator@5.1.0: + resolution: {integrity: sha512-AGrQ4QSgssa1NGmWmLPqN5NY2KajF5MqxetNEO+o0n3ZwZZeTmt7bBnvzHWrmkZFxGgr4HdyFgelzgi06otLuQ==} + dequal@2.0.3: resolution: {integrity: sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA==} engines: {node: '>=6'} @@ -1753,6 +2071,12 @@ packages: devlop@1.1.0: resolution: {integrity: sha512-RWmIqhcFf1lRYBvNmr7qTNuyCt/7/ns2jbpp1+PalgE/rDQcBT0fioSMUpJ93irlUhC5hrg4cYqe6U+0ImW0rA==} + dompurify@3.4.15: + resolution: {integrity: sha512-EUBjM+B+lkDE41iE82DDSCfkoPGfXx8IxFxPMjNzm/Uk4xDet77rTN9wqlxlVg71kK7XGuUMv6wUxJUwwv+Xyw==} + + elkjs@0.9.3: + resolution: {integrity: sha512-f/ZeWvW/BCXbhGEf1Ujp29EASo/lk1FDnETgNKwJrsVvGZhUWCZyg3xLJjAsxfOmt8KjswHmI5EwCQcPMpOYhQ==} + emoji-regex@8.0.0: resolution: {integrity: sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==} @@ -1770,6 +2094,9 @@ packages: es-module-lexer@2.3.2: resolution: {integrity: sha512-poHGpORABojJJucnV9KbOavETW8lBVnphkW77ER5/BQ5Fz7oXSoCNek7IH3vR5nRjdsEz926ibFYX8KtLQmdyw==} + es-toolkit@1.52.0: + resolution: {integrity: sha512-XTNEJQh1tY1ZJVcf6ayP/2n4ZPyaHlW2FWs7xvw5ddPuhUVjLD3olQVQS7kf58JbAB48iL0uL/jerTrjtV3lDA==} + esast-util-from-estree@2.0.0: resolution: {integrity: sha512-4CyanoAudUSBAn5K13H4JhsMH6L9ZP7XbLVe/dKybkxMO7eDyLsT8UHl9TRNrU2Gr9nz+FovfSIjuXWJ81uVwQ==} @@ -1939,6 +2266,9 @@ packages: graceful-fs@4.2.11: resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==} + hachure-fill@0.5.2: + resolution: {integrity: sha512-3GKBOn+m2LX9iq+JC1064cSFprJY4jL1jCXTcpnfER5HYE2l/4EfWSGzkPa/ZDBmYI0ZOEj5VHV/eKnPGkHuOg==} + happy-dom@20.11.6: resolution: {integrity: sha512-Hldbg8AdAa5a5oDcZpjqnGitp7JB0hqWmfv/8qr+kft4vzSD8BHsbdRfzYvL/0QcbKcURC/yyoygbeDQarPvYg==} engines: {node: '>=20.0.0'} @@ -1976,6 +2306,10 @@ packages: resolution: {integrity: sha512-zPGsiS+dWoTZtZ4AtpA9Y+BdSFSNWvnouNlWNoUFyAM6xHOHmdCvqO3k8AIbdamCOv4gUFUVNPf6rJFfc4UiJw==} hasBin: true + iconv-lite@0.6.3: + resolution: {integrity: sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==} + engines: {node: '>=0.10.0'} + image-size@2.0.2: resolution: {integrity: sha512-IRqXKlaXwgSMAMtpNzZa1ZAe8m+Sa1770Dhk8VkSsP9LS+iHD62Zd8FQKs8fbPiagBE7BzoFX23cxFnwshpV6w==} engines: {node: '>=16.x'} @@ -1987,6 +2321,13 @@ packages: inline-style-parser@0.2.7: resolution: {integrity: sha512-Nb2ctOyNR8DqQoR0OwRG95uNWIC0C1lCgf5Naz5H6Ji72KZ8OcFZLz2P5sNgwlyoJ8Yif11oMuYs5pBQa86csA==} + internmap@1.0.1: + resolution: {integrity: sha512-lDB5YccMydFBtasVtxnZ3MRBHuaoE8GKsppq+EchKL2U4nK/DmEpPHNH8MZe5HkMtpSiTSOZwfN0tzYjO/lJEw==} + + internmap@2.0.3: + resolution: {integrity: sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==} + engines: {node: '>=12'} + is-accessor-descriptor@1.0.2: resolution: {integrity: sha512-AIbwAcazqP3R65dGvqk1V+a+vE5Fg1yu/ZKMOiBWSUIXXiwQkYmXQcVa2O0nh0tSDKDFKxG2mY7dB1Sr4hEP1g==} engines: {node: '>= 0.4'} @@ -2044,6 +2385,13 @@ packages: jsonc-parser@3.3.1: resolution: {integrity: sha512-HUgH65KyejrUFPvHFPbqOY0rsFip3Bo5wb4ngvdi1EpCYWUQDC5V+Y7mZws+DLkr4M//zQJoanu1SP+87Dv1oQ==} + katex@0.16.47: + resolution: {integrity: sha512-Eeo8Ys1doU1z+x8AZsPpQu+p/QcZBI5PeOo7QGQdy2x2m0MU/hYagBbGOmXwr5KVbEfVuWv9LpnQWeehogurjg==} + hasBin: true + + khroma@2.1.0: + resolution: {integrity: sha512-Ls993zuzfayK269Svk9hzpeGUKob/sIgZzyHYdjQoAdQetRKpOLj+k/QQQ/6Qi0Yz65mlROrfd+Ev+1+7dz9Kw==} + kind-of@3.2.2: resolution: {integrity: sha512-NOW9QQXMoZGg/oqnVNoNTTIFEIid1627WCffUBJEdMxYApq7mNE7CpzucIPc+ZQg25Phej7IJSmX3hO+oblOtQ==} engines: {node: '>=0.10.0'} @@ -2051,6 +2399,12 @@ packages: launch-editor@2.14.1: resolution: {integrity: sha512-QWBrQsMpH7gPr965dsKD/3cKWiNoTjpATQf++Xq63N6sKRGMwlVXz41O1IZTMfZQgBctD/K5Zt06+/I6pP6+HA==} + layout-base@1.0.2: + resolution: {integrity: sha512-8h2oVEZNktL4BH2JCOI90iD1yXwL6iNW7KcCKT2QZgQJR2vbqDsldCTPRU9NifTCqHZci57XvQQ15YTu+sTYPg==} + + layout-base@2.0.1: + resolution: {integrity: sha512-dp3s92+uNI1hWIpPGH3jK2kxE2lMjdXdr+DH8ynZHpd6PUlH6x6cbuXnoMmiNumznqaNO31xu9e79F0uuZ0JFg==} + lightningcss-android-arm64@1.32.0: resolution: {integrity: sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg==} engines: {node: '>= 12.0.0'} @@ -2132,6 +2486,12 @@ packages: resolution: {integrity: sha512-IXO6OCs9yg8tMKzfPZ1YmheJbZCiEsnBdcB03l0OcfK9prKnJb96siuHCr5Fl37/yo9DnKU+TLpxzTUspw9shg==} engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + lodash-es@4.17.23: + resolution: {integrity: sha512-kVI48u3PZr38HdYz98UmfPnXl2DXrpdctLrFLCd3kOx1xUkOmpFPx7gCWWM5MPkL/fD8zb+Ph0QzjGFs4+hHWg==} + + lodash-es@4.18.1: + resolution: {integrity: sha512-J8xewKD/Gk22OZbhpOVSwcs60zhd95ESDwezOFuA3/099925PdHJ7OFHNTGtajL3AlZkykD32HykiMo+BIBI8A==} + lodash.merge@4.6.2: resolution: {integrity: sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ==} @@ -2157,6 +2517,11 @@ packages: markdown-table@3.0.4: resolution: {integrity: sha512-wiYz4+JrLyb/DqW2hkFJxP7Vd7JuTDm77fvbM8VfEQdmSMqcImWeeRbHwZjBjIFki/VaMK2BhFi7oUUZeM5bqw==} + marked@16.4.2: + resolution: {integrity: sha512-TI3V8YYWvkVf3KJe1dRkpnjs68JUPyEa5vjKrp1XEEJUAOaQc+Qj+L1qWbPd0SJuAdQkFU0h73sXXqwDYxsiDA==} + engines: {node: '>= 20'} + hasBin: true + mdast-util-find-and-replace@3.0.2: resolution: {integrity: sha512-Tmd1Vg/m3Xz43afeNxDIhWRtFZgM2VLyaf4vSTYwudTyeuTneoL3qtWMA5jeLyz/O1vDJmmV4QuScFCA2tBPwg==} @@ -2205,6 +2570,10 @@ packages: mdast-util-to-string@4.0.0: resolution: {integrity: sha512-0H44vDimn51F0YwvxSJSm0eCDOJTRlmN0R1yBh4HLj9wiV1Dn0QoXGbvFAWj2hSItVTlCmBF1hqKlIyUBVFLPg==} + mermaid@12.0.0: + resolution: {integrity: sha512-/wQXC9iBxoGV8p3erbvaXs9h77VyLDBH6GdayVjj3hEcSQhFU4N1WUhUppotCEqlIxI2pRMwjwBSwTB1MfZBgQ==} + engines: {node: '>=22.12.0'} + micromark-core-commonmark@2.0.3: resolution: {integrity: sha512-RDBrHEMSxVFLg6xvnXmb1Ayr2WzLAWjeSATAoxwKYJV94TeNavgoIdA0a9ytzDSVzBy2YKFK+emCPOEibLeCrg==} @@ -2389,6 +2758,9 @@ packages: parse-entities@4.0.2: resolution: {integrity: sha512-GG2AQYWoLgL877gQIKeRPGO1xF9+eG1ujIb5soS5gPvLQ1y2o8FL90w2QWNdf9I361Mpp7726c+lj3U0qK1uGw==} + path-data-parser@0.1.0: + resolution: {integrity: sha512-NOnmBpt5Y2RWbuv0LMzsayp3lVylAHLPUTut412ZA3l+C4uw4ZVkQbjShYCQ8TCpUMdPapr4YjUqLYD6v68j+w==} + path-to-regexp@8.4.2: resolution: {integrity: sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA==} @@ -2427,6 +2799,12 @@ packages: resolution: {integrity: sha512-Nc3IT5yHzflTfbjgqWcCPpo7DaKy4FnpB0l/zCAW0Tc7jxAiuqSxHasntB3D7887LSrA93kDJ9IXovxJYxyLCA==} engines: {node: '>=4'} + points-on-curve@0.2.0: + resolution: {integrity: sha512-0mYKnYYe9ZcqMCWhUjItv/oHjvgEsfKvnUTg8sAtnHr3GVy7rGkXCb6d5cSyqrWqL4k81b9CPg3urd+T7aop3A==} + + points-on-path@0.2.1: + resolution: {integrity: sha512-25ClnWWuw7JbWZcgqY/gJ4FQWadKxGWk+3kR/7kD0tCaDtPPMj7oHu2ToLaVhfpnHrZzYby2w6tUA0eOIuUg8g==} + postcss-load-config@6.0.1: resolution: {integrity: sha512-oPtTM4oerL+UXmx+93ytZVN82RrlY/wPUV8IeDxFrzIjXOLF1pN+EmKPLbubvKHT2HC20xXsCAH2Z+CKV6Oz/g==} engines: {node: '>= 18'} @@ -2573,15 +2951,27 @@ packages: resolution: {integrity: sha512-qYg9KP24dD5qka9J47d0aVky0N+b4fTU89LN9iDnjB5waksiC49rvMB0PrUJQGoTmH50XPiqOvAjDfaijGxYZw==} engines: {node: '>=8'} + robust-predicates@3.0.3: + resolution: {integrity: sha512-NS3levdsRIUOmiJ8FZWCP7LG3QpJyrs/TE0Zpf1yvZu8cAJJ6QMW92H1c7kWpdIHo8RvmLxN/o2JXTKHp74lUA==} + rollup@4.63.0: resolution: {integrity: sha512-T5vnZ2y4QqC3/4P+w2+JO+Q/OVdnPsv4XcSYJYMEn0R9/jjl5AgLwO9LAZMzP2lN71O6pypn91rB7lDstUkfrQ==} engines: {node: '>=18.0.0', npm: '>=8.0.0'} hasBin: true + roughjs@4.6.6: + resolution: {integrity: sha512-ZUz/69+SYpFN/g/lUlo2FXcIjRkSu3nDarreVdGGndHEBJ6cXPdKguS8JGxwj5HA5xIbVKSmLgr5b3AWxtRfvQ==} + + rw@1.3.3: + resolution: {integrity: sha512-PdhdWy89SiZogBLaw42zdeqtRJ//zFd2PgQavcICDUgJT5oW10QCRKbJ6bg4r0/UY2M6BWd5tkxuGFRvCkgfHQ==} + safe-stable-stringify@2.5.0: resolution: {integrity: sha512-b3rppTKm9T+PsVCBEOUR46GWI7fdOs00VKZ1+9c1EWDaDMvjQc6tUwuFyIprgGgTcWoVHSKrU8H31ZHA2e0RHA==} engines: {node: '>=10'} + safer-buffer@2.1.2: + resolution: {integrity: sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==} + scheduler@0.27.0: resolution: {integrity: sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==} @@ -2676,6 +3066,9 @@ packages: babel-plugin-macros: optional: true + stylis@4.4.0: + resolution: {integrity: sha512-5Z9ZpRzfuH6l/UAvCPAPUo3665Nk2wLaZU3x+TLHKVzIz33+sbJqbtrYoC3KD4/uVOr2Zp+L0LySezP9OHV9yA==} + sucrase@3.35.1: resolution: {integrity: sha512-DhuTmvZWux4H1UOnWMB3sk0sbaCVOoQZjv8u1rDoTV0HTdGem9hkAZtl4JZy8P2z4Bg0nT+YMeOFyVr4zcG5Tw==} engines: {node: '>=16 || 14 >=14.17'} @@ -2734,6 +3127,10 @@ packages: trough@2.2.0: resolution: {integrity: sha512-tmMpK00BjZiUyVyvrBK7knerNgmgvcV/KLVyuma/SC+TQN167GrMRciANTz09+k3zW8L8t60jWO1GpfkZdjTaw==} + ts-dedent@2.3.0: + resolution: {integrity: sha512-JfJeIHke7y2egdGGgRAvpCwYFUsHlM2gPcrVOxFkznt/4uzQ7HFmvE63iFHVLBJNDuyDOQgijDK/tXH/f6Msjg==} + engines: {node: '>=6.10'} + ts-interface-checker@0.1.13: resolution: {integrity: sha512-Y/arvbn+rrz3JCKl9C4kVNfTfSm2/mEp5FSz5EsZSANGPSlQrpRI5M4PKF+mJnE52jOO90PnPSc3Ur3bTQw0gA==} @@ -2824,6 +3221,10 @@ packages: util-deprecate@1.0.2: resolution: {integrity: sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==} + uuid@14.0.2: + resolution: {integrity: sha512-xZe/16rV4aa+HGSOCiY2YeLT1OybRLrrkL/Rqaq7p7GMVXjFh+6wN4oMYgjFmnSnhY8t6Xpdl2l9qmnHYuMHwQ==} + hasBin: true + vfile-message@4.0.3: resolution: {integrity: sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw==} @@ -2958,6 +3359,11 @@ snapshots: '@alloc/quick-lru@5.2.0': {} + '@antfu/install-pkg@2.0.1': + dependencies: + package-manager-detector: 1.8.0 + tinyexec: 1.3.0 + '@biomejs/biome@2.5.10': optionalDependencies: '@biomejs/cli-darwin-arm64': 2.5.10 @@ -2998,6 +3404,8 @@ snapshots: cac: 7.0.0 commander: 14.0.3 + '@braintree/sanitize-url@7.1.2': {} + '@changesets/apply-release-plan@8.0.0': dependencies: '@changesets/config': 4.0.0 @@ -3102,6 +3510,23 @@ snapshots: '@changesets/types': 7.0.0 human-id: 4.2.1 + '@chevrotain/cst-dts-gen@11.1.2': + dependencies: + '@chevrotain/gast': 11.1.2 + '@chevrotain/types': 11.1.2 + lodash-es: 4.17.23 + + '@chevrotain/gast@11.1.2': + dependencies: + '@chevrotain/types': 11.1.2 + lodash-es: 4.17.23 + + '@chevrotain/regexp-to-ast@11.1.2': {} + + '@chevrotain/types@11.1.2': {} + + '@chevrotain/utils@11.1.2': {} + '@clack/core@1.4.3': dependencies: fast-wrap-ansi: 0.2.2 @@ -3299,6 +3724,14 @@ snapshots: dependencies: tslib: 2.8.1 + '@iconify/types@2.0.0': {} + + '@iconify/utils@3.1.7': + dependencies: + '@antfu/install-pkg': 2.0.1 + '@iconify/types': 2.0.0 + import-meta-resolve: 4.2.0 + '@img/colour@1.1.0': optional: true @@ -3470,6 +3903,10 @@ snapshots: transitivePeerDependencies: - supports-color + '@mermaid-js/parser@2.0.0': + dependencies: + '@chevrotain/types': 11.1.2 + '@napi-rs/lzma-linux-x64-gnu@1.5.1': optional: true @@ -4076,6 +4513,123 @@ snapshots: '@types/deep-eql': 4.0.2 assertion-error: 2.0.1 + '@types/d3-array@3.2.2': {} + + '@types/d3-axis@3.0.6': + dependencies: + '@types/d3-selection': 3.0.11 + + '@types/d3-brush@3.0.6': + dependencies: + '@types/d3-selection': 3.0.11 + + '@types/d3-chord@3.0.6': {} + + '@types/d3-color@3.1.3': {} + + '@types/d3-contour@3.0.6': + dependencies: + '@types/d3-array': 3.2.2 + '@types/geojson': 7946.0.16 + + '@types/d3-delaunay@6.0.4': {} + + '@types/d3-dispatch@3.0.7': {} + + '@types/d3-drag@3.0.7': + dependencies: + '@types/d3-selection': 3.0.11 + + '@types/d3-dsv@3.0.7': {} + + '@types/d3-ease@3.0.2': {} + + '@types/d3-fetch@3.0.7': + dependencies: + '@types/d3-dsv': 3.0.7 + + '@types/d3-force@3.0.10': {} + + '@types/d3-format@3.0.4': {} + + '@types/d3-geo@3.1.1': + dependencies: + '@types/geojson': 7946.0.16 + + '@types/d3-hierarchy@3.1.7': {} + + '@types/d3-interpolate@3.0.4': + dependencies: + '@types/d3-color': 3.1.3 + + '@types/d3-path@3.1.1': {} + + '@types/d3-polygon@3.0.2': {} + + '@types/d3-quadtree@3.0.6': {} + + '@types/d3-random@3.0.4': {} + + '@types/d3-scale-chromatic@3.1.0': {} + + '@types/d3-scale@4.0.9': + dependencies: + '@types/d3-time': 3.0.4 + + '@types/d3-selection@3.0.11': {} + + '@types/d3-shape@3.2.0': + dependencies: + '@types/d3-path': 3.1.1 + + '@types/d3-time-format@4.0.3': {} + + '@types/d3-time@3.0.4': {} + + '@types/d3-timer@3.0.2': {} + + '@types/d3-transition@3.0.9': + dependencies: + '@types/d3-selection': 3.0.11 + + '@types/d3-zoom@3.0.8': + dependencies: + '@types/d3-interpolate': 3.0.4 + '@types/d3-selection': 3.0.11 + + '@types/d3@7.4.3': + dependencies: + '@types/d3-array': 3.2.2 + '@types/d3-axis': 3.0.6 + '@types/d3-brush': 3.0.6 + '@types/d3-chord': 3.0.6 + '@types/d3-color': 3.1.3 + '@types/d3-contour': 3.0.6 + '@types/d3-delaunay': 6.0.4 + '@types/d3-dispatch': 3.0.7 + '@types/d3-drag': 3.0.7 + '@types/d3-dsv': 3.0.7 + '@types/d3-ease': 3.0.2 + '@types/d3-fetch': 3.0.7 + '@types/d3-force': 3.0.10 + '@types/d3-format': 3.0.4 + '@types/d3-geo': 3.1.1 + '@types/d3-hierarchy': 3.1.7 + '@types/d3-interpolate': 3.0.4 + '@types/d3-path': 3.1.1 + '@types/d3-polygon': 3.0.2 + '@types/d3-quadtree': 3.0.6 + '@types/d3-random': 3.0.4 + '@types/d3-scale': 4.0.9 + '@types/d3-scale-chromatic': 3.1.0 + '@types/d3-selection': 3.0.11 + '@types/d3-shape': 3.2.0 + '@types/d3-time': 3.0.4 + '@types/d3-time-format': 4.0.3 + '@types/d3-timer': 3.0.2 + '@types/d3-transition': 3.0.9 + '@types/d3-zoom': 3.0.8 + '@types/debug@4.1.13': dependencies: '@types/ms': 2.1.0 @@ -4088,6 +4642,8 @@ snapshots: '@types/estree@1.0.9': {} + '@types/geojson@7946.0.16': {} + '@types/hast@3.0.5': dependencies: '@types/unist': 3.0.3 @@ -4118,6 +4674,9 @@ snapshots: dependencies: csstype: 3.2.3 + '@types/trusted-types@2.0.7': + optional: true + '@types/unist@2.0.11': {} '@types/unist@3.0.3': {} @@ -4130,6 +4689,11 @@ snapshots: '@ungap/structured-clone@1.3.4': {} + '@upsetjs/venn.js@2.0.0': + optionalDependencies: + d3-selection: 3.0.0 + d3-transition: 3.0.1(d3-selection@3.0.0) + '@vitest/expect@4.1.11': dependencies: '@standard-schema/spec': 1.1.0 @@ -4227,6 +4791,15 @@ snapshots: character-reference-invalid@2.0.1: {} + chevrotain@11.1.2: + dependencies: + '@chevrotain/cst-dts-gen': 11.1.2 + '@chevrotain/gast': 11.1.2 + '@chevrotain/regexp-to-ast': 11.1.2 + '@chevrotain/types': 11.1.2 + '@chevrotain/utils': 11.1.2 + lodash-es: 4.17.23 + chokidar@4.0.3: dependencies: readdirp: 4.1.2 @@ -4256,6 +4829,10 @@ snapshots: commander@4.1.1: {} + commander@7.2.0: {} + + commander@8.3.0: {} + compute-scroll-into-view@3.1.1: {} confbox@0.1.8: {} @@ -4266,12 +4843,206 @@ snapshots: convert-source-map@2.0.0: {} + cose-base@1.0.3: + dependencies: + layout-base: 1.0.2 + + cose-base@2.2.0: + dependencies: + layout-base: 2.0.1 + cssesc@3.0.0: {} csstype@3.2.3: {} + cytoscape-cose-bilkent@4.1.0(cytoscape@3.34.3): + dependencies: + cose-base: 1.0.3 + cytoscape: 3.34.3 + + cytoscape-fcose@2.2.0(cytoscape@3.34.3): + dependencies: + cose-base: 2.2.0 + cytoscape: 3.34.3 + + cytoscape@3.34.3: {} + + d3-array@2.12.1: + dependencies: + internmap: 1.0.1 + + d3-array@3.2.4: + dependencies: + internmap: 2.0.3 + + d3-axis@3.0.0: {} + + d3-brush@3.0.0: + dependencies: + d3-dispatch: 3.0.1 + d3-drag: 3.0.0 + d3-interpolate: 3.0.1 + d3-selection: 3.0.0 + d3-transition: 3.0.1(d3-selection@3.0.0) + + d3-chord@3.0.1: + dependencies: + d3-path: 3.1.0 + + d3-color@3.1.0: {} + + d3-contour@4.0.2: + dependencies: + d3-array: 3.2.4 + + d3-delaunay@6.0.4: + dependencies: + delaunator: 5.1.0 + + d3-dispatch@3.0.1: {} + + d3-drag@3.0.0: + dependencies: + d3-dispatch: 3.0.1 + d3-selection: 3.0.0 + + d3-dsv@3.0.1: + dependencies: + commander: 7.2.0 + iconv-lite: 0.6.3 + rw: 1.3.3 + + d3-ease@3.0.1: {} + + d3-fetch@3.0.1: + dependencies: + d3-dsv: 3.0.1 + + d3-force@3.0.0: + dependencies: + d3-dispatch: 3.0.1 + d3-quadtree: 3.0.1 + d3-timer: 3.0.1 + + d3-format@3.1.2: {} + + d3-geo@3.1.1: + dependencies: + d3-array: 3.2.4 + + d3-hierarchy@3.1.2: {} + + d3-interpolate@3.0.1: + dependencies: + d3-color: 3.1.0 + + d3-path@1.0.9: {} + + d3-path@3.1.0: {} + + d3-polygon@3.0.1: {} + + d3-quadtree@3.0.1: {} + + d3-random@3.0.1: {} + + d3-sankey@0.12.3: + dependencies: + d3-array: 2.12.1 + d3-shape: 1.3.7 + + d3-scale-chromatic@3.1.0: + dependencies: + d3-color: 3.1.0 + d3-interpolate: 3.0.1 + + d3-scale@4.0.2: + dependencies: + d3-array: 3.2.4 + d3-format: 3.1.2 + d3-interpolate: 3.0.1 + d3-time: 3.1.0 + d3-time-format: 4.1.0 + + d3-selection@3.0.0: {} + + d3-shape@1.3.7: + dependencies: + d3-path: 1.0.9 + + d3-shape@3.2.0: + dependencies: + d3-path: 3.1.0 + + d3-time-format@4.1.0: + dependencies: + d3-time: 3.1.0 + + d3-time@3.1.0: + dependencies: + d3-array: 3.2.4 + + d3-timer@3.0.1: {} + + d3-transition@3.0.1(d3-selection@3.0.0): + dependencies: + d3-color: 3.1.0 + d3-dispatch: 3.0.1 + d3-ease: 3.0.1 + d3-interpolate: 3.0.1 + d3-selection: 3.0.0 + d3-timer: 3.0.1 + + d3-zoom@3.0.0: + dependencies: + d3-dispatch: 3.0.1 + d3-drag: 3.0.0 + d3-interpolate: 3.0.1 + d3-selection: 3.0.0 + d3-transition: 3.0.1(d3-selection@3.0.0) + + d3@7.9.0: + dependencies: + d3-array: 3.2.4 + d3-axis: 3.0.0 + d3-brush: 3.0.0 + d3-chord: 3.0.1 + d3-color: 3.1.0 + d3-contour: 4.0.2 + d3-delaunay: 6.0.4 + d3-dispatch: 3.0.1 + d3-drag: 3.0.0 + d3-dsv: 3.0.1 + d3-ease: 3.0.1 + d3-fetch: 3.0.1 + d3-force: 3.0.0 + d3-format: 3.1.2 + d3-geo: 3.1.1 + d3-hierarchy: 3.1.2 + d3-interpolate: 3.0.1 + d3-path: 3.1.0 + d3-polygon: 3.0.1 + d3-quadtree: 3.0.1 + d3-random: 3.0.1 + d3-scale: 4.0.2 + d3-scale-chromatic: 3.1.0 + d3-selection: 3.0.0 + d3-shape: 3.2.0 + d3-time: 3.1.0 + d3-time-format: 4.1.0 + d3-timer: 3.0.1 + d3-transition: 3.0.1(d3-selection@3.0.0) + d3-zoom: 3.0.0 + + dagre-d3-es@7.0.14: + dependencies: + d3: 7.9.0 + lodash-es: 4.18.1 + dateformat@4.6.3: {} + dayjs@1.11.23: {} + debug@4.4.3: dependencies: ms: 2.1.3 @@ -4284,6 +5055,10 @@ snapshots: dependencies: is-descriptor: 1.0.4 + delaunator@5.1.0: + dependencies: + robust-predicates: 3.0.3 + dequal@2.0.3: {} detect-libc@2.1.2: {} @@ -4294,6 +5069,12 @@ snapshots: dependencies: dequal: 2.0.3 + dompurify@3.4.15: + optionalDependencies: + '@types/trusted-types': 2.0.7 + + elkjs@0.9.3: {} + emoji-regex@8.0.0: {} end-of-stream@1.4.5: @@ -4309,6 +5090,8 @@ snapshots: es-module-lexer@2.3.2: {} + es-toolkit@1.52.0: {} + esast-util-from-estree@2.0.0: dependencies: '@types/estree-jsx': 1.0.5 @@ -4550,6 +5333,8 @@ snapshots: graceful-fs@4.2.11: {} + hachure-fill@0.5.2: {} + happy-dom@20.11.6: dependencies: '@types/node': 26.3.0 @@ -4638,12 +5423,20 @@ snapshots: human-id@4.2.1: {} + iconv-lite@0.6.3: + dependencies: + safer-buffer: 2.1.2 + image-size@2.0.2: {} import-meta-resolve@4.2.0: {} inline-style-parser@0.2.7: {} + internmap@1.0.1: {} + + internmap@2.0.3: {} + is-accessor-descriptor@1.0.2: dependencies: hasown: 2.0.4 @@ -4690,6 +5483,12 @@ snapshots: jsonc-parser@3.3.1: {} + katex@0.16.47: + dependencies: + commander: 8.3.0 + + khroma@2.1.0: {} + kind-of@3.2.2: dependencies: is-buffer: 1.1.6 @@ -4699,6 +5498,10 @@ snapshots: picocolors: 1.1.1 shell-quote: 1.10.0 + layout-base@1.0.2: {} + + layout-base@2.0.1: {} + lightningcss-android-arm64@1.32.0: optional: true @@ -4754,6 +5557,10 @@ snapshots: load-tsconfig@0.2.5: {} + lodash-es@4.17.23: {} + + lodash-es@4.18.1: {} + lodash.merge@4.6.2: {} longest-streak@3.1.0: {} @@ -4772,6 +5579,8 @@ snapshots: markdown-table@3.0.4: {} + marked@16.4.2: {} + mdast-util-find-and-replace@3.0.2: dependencies: '@types/mdast': 4.0.4 @@ -4935,6 +5744,32 @@ snapshots: dependencies: '@types/mdast': 4.0.4 + mermaid@12.0.0: + dependencies: + '@braintree/sanitize-url': 7.1.2 + '@iconify/utils': 3.1.7 + '@mermaid-js/parser': 2.0.0 + '@types/d3': 7.4.3 + '@upsetjs/venn.js': 2.0.0 + chevrotain: 11.1.2 + cytoscape: 3.34.3 + cytoscape-cose-bilkent: 4.1.0(cytoscape@3.34.3) + cytoscape-fcose: 2.2.0(cytoscape@3.34.3) + d3: 7.9.0 + d3-sankey: 0.12.3 + dagre-d3-es: 7.0.14 + dayjs: 1.11.23 + dompurify: 3.4.15 + elkjs: 0.9.3 + es-toolkit: 1.52.0 + katex: 0.16.47 + khroma: 2.1.0 + marked: 16.4.2 + roughjs: 4.6.6 + stylis: 4.4.0 + ts-dedent: 2.3.0 + uuid: 14.0.2 + micromark-core-commonmark@2.0.3: dependencies: decode-named-character-reference: 1.3.0 @@ -5283,6 +6118,8 @@ snapshots: is-decimal: 2.0.1 is-hexadecimal: 2.0.1 + path-data-parser@0.1.0: {} + path-to-regexp@8.4.2: {} pathe@2.0.3: {} @@ -5337,6 +6174,13 @@ snapshots: pluralize@8.0.0: {} + points-on-curve@0.2.0: {} + + points-on-path@0.2.1: + dependencies: + path-data-parser: 0.1.0 + points-on-curve: 0.2.0 + postcss-load-config@6.0.1(jiti@2.7.0)(postcss@8.5.26)(yaml@2.9.0): dependencies: lilconfig: 3.1.3 @@ -5517,6 +6361,8 @@ snapshots: resolve-from@5.0.0: {} + robust-predicates@3.0.3: {} + rollup@4.63.0: dependencies: '@types/estree': 1.0.9 @@ -5549,8 +6395,19 @@ snapshots: '@rollup/rollup-win32-x64-msvc': 4.63.0 fsevents: 2.3.3 + roughjs@4.6.6: + dependencies: + hachure-fill: 0.5.2 + path-data-parser: 0.1.0 + points-on-curve: 0.2.0 + points-on-path: 0.2.1 + + rw@1.3.3: {} + safe-stable-stringify@2.5.0: {} + safer-buffer@2.1.2: {} + scheduler@0.27.0: {} scroll-into-view-if-needed@3.1.0: @@ -5658,6 +6515,8 @@ snapshots: client-only: 0.0.1 react: 19.2.8 + stylis@4.4.0: {} + sucrase@3.35.1: dependencies: '@jridgewell/gen-mapping': 0.3.13 @@ -5709,6 +6568,8 @@ snapshots: trough@2.2.0: {} + ts-dedent@2.3.0: {} + ts-interface-checker@0.1.13: {} tslib@2.8.1: {} @@ -5814,6 +6675,8 @@ snapshots: util-deprecate@1.0.2: {} + uuid@14.0.2: {} + vfile-message@4.0.3: dependencies: '@types/unist': 3.0.3 diff --git a/site/components/mdx.tsx b/site/components/mdx.tsx index 0fd7716..4404138 100644 --- a/site/components/mdx.tsx +++ b/site/components/mdx.tsx @@ -4,6 +4,7 @@ import { Step, Steps } from "fumadocs-ui/components/steps"; import { Tab, Tabs } from "fumadocs-ui/components/tabs"; import defaultMdxComponents from "fumadocs-ui/mdx"; import type { MDXComponents } from "mdx/types"; +import { Mermaid } from "@/components/mermaid"; export function getMDXComponents(components?: MDXComponents) { return { @@ -16,6 +17,7 @@ export function getMDXComponents(components?: MDXComponents) { File, Files, Folder, + Mermaid, Step, Steps, Tab, diff --git a/site/components/mermaid.tsx b/site/components/mermaid.tsx new file mode 100644 index 0000000..0565c55 --- /dev/null +++ b/site/components/mermaid.tsx @@ -0,0 +1,65 @@ +// biome-ignore-all lint/security/noDangerouslySetInnerHtml: this component renders Mermaid's SVG from this repo's own diagram source, not from user input. +"use client"; + +import { useTheme } from "next-themes"; +import { use, useEffect, useId, useState } from "react"; + +/** + * Renders a Mermaid diagram on the client. `remarkMdxMermaid` turns a + * ```mermaid code block in MDX into ``, so a diagram is + * authored as text and stays reviewable in the raw markdown (which also flows + * into llms-full.txt). + * + * This follows the Fumadocs recommended renderer. Mermaid is imported on + * demand, so pages without a diagram never load it. + */ +export function Mermaid({ chart }: { chart: string }) { + const [mounted, setMounted] = useState(false); + + useEffect(() => { + setMounted(true); + }, []); + + if (!mounted) return; + return ; +} + +const cache = new Map>(); + +function cachePromise(key: string, setPromise: () => Promise): Promise { + const cached = cache.get(key); + if (cached) return cached as Promise; + + const promise = setPromise(); + cache.set(key, promise); + return promise; +} + +function MermaidContent({ chart }: { chart: string }) { + const id = useId(); + const { resolvedTheme } = useTheme(); + const { default: mermaid } = use(cachePromise("mermaid", () => import("mermaid"))); + + mermaid.initialize({ + startOnLoad: false, + securityLevel: "loose", + fontFamily: "inherit", + themeCSS: "margin: 1.5rem auto 0;", + theme: resolvedTheme === "dark" ? "dark" : "default", + }); + + const { svg, bindFunctions } = use( + cachePromise(`${chart}-${resolvedTheme}`, () => { + return mermaid.render(id, chart.replaceAll("\\n", "\n")); + }), + ); + + return ( +
{ + if (container) bindFunctions?.(container); + }} + dangerouslySetInnerHTML={{ __html: svg }} + /> + ); +} diff --git a/site/package.json b/site/package.json index 84eb6e4..1018f52 100644 --- a/site/package.json +++ b/site/package.json @@ -17,7 +17,9 @@ "fumadocs-mdx": "^11.5.0", "fumadocs-ui": "^15.0.0", "lucide-react": "^1.34.0", + "mermaid": "^12.0.0", "next": "^15.3.0", + "next-themes": "^0.4.6", "react": "^19.0.0", "react-dom": "^19.0.0" }, diff --git a/site/source.config.ts b/site/source.config.ts index 2f022f4..32a478c 100644 --- a/site/source.config.ts +++ b/site/source.config.ts @@ -1,3 +1,4 @@ +import { remarkMdxMermaid } from "fumadocs-core/mdx-plugins"; import { defineConfig, defineDocs } from "fumadocs-mdx/config"; export const docs = defineDocs({ @@ -6,7 +7,8 @@ export const docs = defineDocs({ export default defineConfig({ mdxOptions: { - remarkPlugins: [], + // ```mermaid blocks become ; see components/mermaid.tsx. + remarkPlugins: [remarkMdxMermaid], rehypePlugins: [], }, }); From 297472463ecb0303644bb6c83c9bf5f598bdade0 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 14:14:06 +0530 Subject: [PATCH 39/41] docs: add a journey sequence and a tool-call lifecycle diagram The journeys page trades its text transcript for a sequence diagram of the agent filling the draft, the human confirming, and the real write running. The visible-effects page gains the same view of one call, including the UI update that is easy to forget. Both stay as text in the raw markdown, so they also read well in llms-full.txt. --- site/content/docs/journeys.mdx | 42 ++++++++++++++++++--------- site/content/docs/visible-effects.mdx | 18 +++++++++++- 2 files changed, 46 insertions(+), 14 deletions(-) diff --git a/site/content/docs/journeys.mdx b/site/content/docs/journeys.mdx index a7913f4..a701345 100644 --- a/site/content/docs/journeys.mdx +++ b/site/content/docs/journeys.mdx @@ -109,19 +109,35 @@ book-ride-set-pickup-time Set when the user wants to be picked up. Part of book-ride-submit Request the ride and show the driver's details. ``` -The agent fills the form by calling them in order, reading each reply: - -```text -agent book-ride-resolve-destination { "address": "SFO airport" } -tool Stored. Still needed: book-ride-set-pickup-time, book-ride-submit. - -agent book-ride-set-pickup-time { "pickupAt": "2026-03-14T19:30:00Z" } -tool Stored. The journey is ready. Call book-ride-submit. - -agent book-ride-submit - (the page asks the human: "Allow the agent to: Request the ride and show the driver's details.") -human approves -tool ride requested +The agent fills the form by calling them in order, reading each reply. The +whole exchange, as a diagram: + +```mermaid +sequenceDiagram + actor User + actor Agent as Your coding agent + participant Page as Page (WebMCP) + participant API as Your backend + + Note over Page: createJourney().register() ran on load + Page->>Agent: tools registered: resolve-destination, set-pickup-time, submit + + Agent->>Page: book-ride-resolve-destination({ address: "SFO airport" }) + Page->>API: geocode-address + API-->>Page: coordinates + Note right of Page: stored on the draft + Page-->>Agent: Stored. Still needed: set-pickup-time, submit + + Agent->>Page: book-ride-set-pickup-time({ pickupAt: "2026-03-14T19:30:00Z" }) + Page-->>Agent: Stored. Ready. Call book-ride-submit. + + Agent->>Page: book-ride-submit + Page->>User: confirm "Request the ride?" + User-->>Page: approves + Page->>API: request-ride(build(draft)) + API-->>Page: ride requested + Page-->>Agent: done + Note over Page: draft cleared ``` Two details are doing a lot of work here. The "still needed" replies are how the diff --git a/site/content/docs/visible-effects.mdx b/site/content/docs/visible-effects.mdx index e4e3e9b..36b282e 100644 --- a/site/content/docs/visible-effects.mdx +++ b/site/content/docs/visible-effects.mdx @@ -10,7 +10,23 @@ changes something and the screen does not move, the person sees nothing happen, experiences of your app drift apart: the agent's and the human's. So a tool's job is not finished when the request succeeds. It is finished when the page -reflects it. +reflects it. One step in the call is easy to forget, because nothing fails when you skip it: + +```mermaid +sequenceDiagram + actor User as Human + actor Agent as Agent + participant Page as Page (WebMCP) + participant API as Your API + + Agent->>Page: call a tool + Note over Page: the browser validates the input against the schema + Page->>API: execute() calls your real endpoint + API-->>Page: the response + Note over Page: update the UI: navigate, invalidate a query, dispatch an event + Page-->>Agent: toolResult + Note over User: the human sees the change +``` ## The pattern From 8eefb43fe2cfd010b9708388daee13ebb38a66bc Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 14:27:32 +0530 Subject: [PATCH 40/41] docs: remove duplicate headings and repeated content across the docs The docs render the title from frontmatter above the body, so a leading `#` heading repeated it on the journeys FAQ and the visible-effects page. Reading the rest of the pages turned up the same content written twice across pages: - working-with-your-agent repeated three prompts that already live, verbatim, in the prompt cookbook. - the cookbook's warning restated the agent-exposure section on that page, now a link instead. - the introduction and the why page shared the same browser-support sentence. - the guides page carried a full dashboard section that duplicates the CLI reference. Title headings, duplicate answers, and cross-page copies are all in one commit so the cleanup is easy to review or revert together. --- site/content/docs/guides.mdx | 14 ------- site/content/docs/journeys-faq.mdx | 6 +-- site/content/docs/prompt-cookbook.mdx | 6 +-- site/content/docs/visible-effects.mdx | 2 - site/content/docs/why-agent-tools.mdx | 9 ++--- site/content/docs/working-with-your-agent.mdx | 37 ------------------- 6 files changed, 8 insertions(+), 66 deletions(-) diff --git a/site/content/docs/guides.mdx b/site/content/docs/guides.mdx index 7a71178..ee05be0 100644 --- a/site/content/docs/guides.mdx +++ b/site/content/docs/guides.mdx @@ -52,20 +52,6 @@ The generated tools register on page load. To watch that happen: dashboard (server-side, so without your browser session; auth'd endpoints will answer accordingly). -## The dashboard - -```bash -npx @webmcp-stack/codegen dev -``` - -![The webmcp-codegen dashboard](/images/dev-server-dashboard.png) - -A local control panel for the generated tools: browse and search them grouped by risk -level, see each one's route and audit findings, edit descriptions, toggle tools on and -off, and run them directly. Edits are written to `.webmcp-codegen.json` as overrides, so -they survive regeneration and can be committed. The dashboard never touches your app: it -is a localhost page served by the CLI, gone when you Ctrl+C. - ## CI The audit exits non-zero on errors, which makes it a CI check: diff --git a/site/content/docs/journeys-faq.mdx b/site/content/docs/journeys-faq.mdx index 574739c..96f9560 100644 --- a/site/content/docs/journeys-faq.mdx +++ b/site/content/docs/journeys-faq.mdx @@ -3,8 +3,6 @@ title: Journey questions, answered description: The draft, the lifecycle, what the agent does and what you do, and the edge cases, answered in plain terms. --- -# Journey questions, answered - The [journeys page](/docs/journeys) explains the feature. This page answers the questions people ask once they read the code. None of them are dumb questions; they are the ones almost everyone has. @@ -57,9 +55,7 @@ Nothing durable, ever. The simplest possible: a flat bag of labeled values, for example `{ destination: {...}, pickupAt: "2026-03-14T19:30:00Z" }`. Nothing enforces -nesting, and any step can write any field. One caveat: if two steps use the same -field name, the second silently overwrites the first. Avoiding that is on -whoever writes the journey. +nesting, and any step can write any field. ## Will it hold data reliably for a session? diff --git a/site/content/docs/prompt-cookbook.mdx b/site/content/docs/prompt-cookbook.mdx index 28ecebd..eb5d2af 100644 --- a/site/content/docs/prompt-cookbook.mdx +++ b/site/content/docs/prompt-cookbook.mdx @@ -13,9 +13,9 @@ coding agent to use them; each one also works as a checklist for doing the task by hand. - Read the diff before you accept it, and do not let an agent enable a write on - its own. Enabling a write is a human decision about what a model may do while - acting as your user. + Read the diff before you accept it. The + [Working with your coding agent](/docs/working-with-your-agent) page covers + what to check, and why a write is always your decision, not the agent's. ## Improve one description diff --git a/site/content/docs/visible-effects.mdx b/site/content/docs/visible-effects.mdx index 36b282e..ffb8795 100644 --- a/site/content/docs/visible-effects.mdx +++ b/site/content/docs/visible-effects.mdx @@ -3,8 +3,6 @@ title: Make the effect visible description: An agent acts while a human watches the page. A call that changes something should change it on screen, not just in the response. --- -# Make the effect visible - When an agent calls one of your tools, a person is usually watching the page. If the call changes something and the screen does not move, the person sees nothing happen, and the two experiences of your app drift apart: the agent's and the human's. diff --git a/site/content/docs/why-agent-tools.mdx b/site/content/docs/why-agent-tools.mdx index 604eaaf..b42e87f 100644 --- a/site/content/docs/why-agent-tools.mdx +++ b/site/content/docs/why-agent-tools.mdx @@ -95,11 +95,10 @@ Being clear about the edges is part of using it well: ## Honesty about where WebMCP is -WebMCP is early. The tools work today in Chrome behind a flag and in the origin -trial, and elsewhere with a small polyfill. The specification is still moving, -and it may change in ways that affect what you generated. This project pins the -draft it targets and reports spec drift, but you should treat the generated -surface as something you review, not something you set and forget. +WebMCP is early, and the specification is still moving. It may change in ways +that affect what you generated: this project pins the draft it targets and +reports spec drift, but treat the generated surface as something you review, +not something you set and forget. That is also why the output is plain files in your repo with no runtime dependency. If this tool disappears tomorrow, your tools keep working. diff --git a/site/content/docs/working-with-your-agent.mdx b/site/content/docs/working-with-your-agent.mdx index 884606d..c4b4395 100644 --- a/site/content/docs/working-with-your-agent.mdx +++ b/site/content/docs/working-with-your-agent.mdx @@ -50,43 +50,6 @@ safety rules instead of inventing its own. 3. **The existing generated tools**, so the agent composes them instead of re-implementing the API. -## Prompts that work - -**Improve a tool's description.** Descriptions are the highest-leverage edit, -and a small prompt goes a long way: - -> Read `.agents/skills/webmcp-tools/SKILL.md`. Then look at -> `src/webmcp/list-trips.webmcp.ts`. The description reads mechanical. Rewrite -> it so an agent knows what it does, when to use it, and what it returns, within -> the skill's budget. Put the new text in `.webmcp-codegen.json`, not in the -> generated region. - -**Write a journey.** This is the one worth getting right. A good prompt names -the goal, the data the agent cannot invent, and the moment that needs a human: - -> Read `.agents/skills/webmcp-tools/SKILL.md` and the generated tools in -> `src/webmcp`. -> -> Write a journey called `book-ride` in `src/webmcp/journeys`. Goal: let a user -> book a ride to a destination they give. -> -> - The ride API needs coordinates, which only the geocode endpoint can -> provide. Compose the generated `geocode-address` tool for that step and -> store the result on the draft as `destination`. -> - The user sets a pickup time in a second step. -> - Requesting the ride is a write, so it goes through the submit gate, not a -> step. Use the real `request-ride` execute as the submit's `run`. -> -> Run `npx @webmcp-stack/codegen verify` when you are done and fix what it -> reports. - -**Review the whole surface.** Useful before a release: - -> Read `.agents/skills/webmcp-tools/SKILL.md`. Review every tool in -> `src/webmcp`. For each one, tell me: is the name intent-shaped, does the -> description say what it returns, and should an agent be able to call it at -> all? Do not enable any withheld tool. List what you would change and why. - ## What to check when the agent is done An agent can be confidently wrong, especially about intent. Before you accept From 1a49cad57ab023e537f83a4304e46797bfbbf7a4 Mon Sep 17 00:00:00 2001 From: Sourav Kumar Nanda Date: Sat, 12 Sep 2026 14:35:03 +0530 Subject: [PATCH 41/41] docs: retitle the why page to say what it is about "Why give your site tools for agents" made the site the owner of the tools and read like a teaser. "Why make your site agent-ready" says the same thing in plain words. The intro card matches, and the URL is unchanged. --- site/content/docs/index.mdx | 2 +- site/content/docs/why-agent-tools.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/site/content/docs/index.mdx b/site/content/docs/index.mdx index 5a2be0e..8b85d9e 100644 --- a/site/content/docs/index.mdx +++ b/site/content/docs/index.mdx @@ -76,7 +76,7 @@ Being clear about the edges is part of using it well: ## Where to start - + Why a small, well-described surface matters, and what a model can and cannot do with your site. diff --git a/site/content/docs/why-agent-tools.mdx b/site/content/docs/why-agent-tools.mdx index b42e87f..c830061 100644 --- a/site/content/docs/why-agent-tools.mdx +++ b/site/content/docs/why-agent-tools.mdx @@ -1,5 +1,5 @@ --- -title: Why give your site tools for agents +title: Why make your site agent-ready description: Agents already read your pages and guess. A small set of well-described tools is how your site takes part on purpose, instead of by accident. ---