From 89aa4fa8dbfa627f1fb5660260f5218d14ee4c24 Mon Sep 17 00:00:00 2001 From: Janghoon Lee <44862514+its-janghoon@users.noreply.github.com> Date: Tue, 22 Sep 2026 13:26:26 +0900 Subject: [PATCH 01/33] docs: design of record for the local engine API (#44) Three products are written against a local engine endpoint that has never existed. redrob-browser POSTs to /experimental/external/chat/completions and streams the reply; no such route is served, and the string "external" does not appear under packages/redrob/src/server. redrob-reblend ranks a LOCAL transport its own sidecar cannot use. redrob-office compiles this engine at release time, ships it inside its installer behind a packaging guard, and never calls it. The products that drive the engine successfully drive its session API. The ones that tried to give it an OpenAI face are the broken ones. So this serves that face for real, at one stable path. Decisions recorded here: - POST /v1/chat/completions is the core path, with no alias. It sits on packages/llm and the provider registry, not on Session: a completion is stateless and a session is not, and local inference then falls out as just another resolved provider. - Tool ownership is decided by the request, not configured. `tools` present means the CALLER owns them and the engine returns tool_calls; absent means the engine runs its own. Office edits a document held in its renderer and browser drives a live tab, so for them caller ownership is the only correct answer. - Inbound auth is the server's existing Basic. Outbound is the engine's OWN Console credential, which is what makes one login serve every product. - 401 carries code "engine_not_authenticated", and that code alone may trigger a sign-in prompt. Branching on message text instead is how a network timeout shipped as "please log in". - Every /experimental route is promoted in one breaking rename. No product calls any of them; the only callers are this repo's generated SDKs, which httpapi-codegen regenerates. No compatibility window is owed to callers that do not exist. - Products stop bundling the binary. One shared daemon, recorded at ~/.redrob/run/server.json (0600), resolved via REDROB_CODE_BIN then $HOME/.redrob/bin then PATH. Per-product processes were rejected: with local inference that loads the same weights N times. - Nothing pins an engine version. /capabilities gains a chatCompletions entry and products declare a MINIMUM capability, so auto-update cannot break them silently. Compatibility is kept by rules plus a contract test that fails CI when a v1 field is removed, not by a promise. No code changes. The route, the promotion, the contract test, and the per-product migration follow as their own pull requests. Co-authored-by: Janghoon Lee <44862514+savagemanage@users.noreply.github.com> --- docs/LOCAL-ENGINE-API.md | 209 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100644 docs/LOCAL-ENGINE-API.md diff --git a/docs/LOCAL-ENGINE-API.md b/docs/LOCAL-ENGINE-API.md new file mode 100644 index 0000000000..3e07f5b8d6 --- /dev/null +++ b/docs/LOCAL-ENGINE-API.md @@ -0,0 +1,209 @@ +# The local engine API + +Every Redrob product needs the same three things from this engine: a credential the +user only supplies once, model and provider routing, and — the reason this document +exists — the option of inference that does not leave the machine. None of those is +the agent. So the shared surface is a plain inference endpoint, and the agent stays +where it already is. + +This document is the design of record for that endpoint, for promoting the +`/experimental` surface, and for shipping the engine as one shared install instead +of a copy inside every product. + +## Why this is being written now + +Three products were already written against a local engine endpoint that has never +existed. + +- `redrob-browser` POSTs to `/experimental/external/chat/completions` and + streams the SSE reply (`sources/chrome/browser/redrob/sidecar_client.cc:592`). No + such route is served: the experimental group registers `capabilities`, `console`, + `tool`, `worktree`, `session` and `resource` and nothing else + (`packages/redrob/src/server/routes/instance/httpapi/groups/experimental.ts:91-101`). + The string `external` does not appear anywhere under `packages/redrob/src/server`. +- `redrob-reblend` ranks a LOCAL transport at `http://127.0.0.1:4096` and health-probes + it, but its sidecar rejects any non-HTTPS base URL, so that transport cannot carry a + request even when the probe succeeds. +- `redrob-office` compiles this engine from source at release time and ships it inside + its installer, with a packaging guard that refuses to build without it — and never + calls it. + +The products that DO drive the engine successfully (`redrob-cowork`, `redrob-cad`) +drive its own session API. The ones that tried to give the engine an OpenAI face are +the ones that are broken. That is the signal this design follows: serve the OpenAI +face for real, at one stable path, and stop pretending the session API is one. + +## 1. `POST /v1/chat/completions` + +The core path. There is no alias: `/experimental/external/chat/completions` is not +served, and `redrob-browser`'s call site moves to `/v1/chat/completions` in the same +wave (see Migration). + +### Where it sits + +On `packages/llm` and the provider registry — NOT on `Session`. A chat completion is +stateless and a session is not; mapping one onto the other is what forces an +ephemeral-session-per-request design and a fake OpenAI face over an event bus. The +route resolves the provider and credential exactly as the agent does, then makes the +model call directly. + +This is also what makes local inference work: a locally served model is just another +provider the registry resolves, and callers see no difference. + +### Request + +An OpenAI chat-completions subset: + +| field | required | notes | +| --- | --- | --- | +| `model` | yes | `redrob/auto` routes through the Console; a local provider id targets a local model | +| `messages` | yes | `system` / `user` / `assistant` / `tool` roles | +| `tools` | no | **presence changes who executes them — see below** | +| `tool_choice` | no | `auto` \| `none` \| a named function | +| `stream` | no | default `false`; `true` returns SSE | +| `max_tokens` | no | | +| `temperature` | no | ignored for models that fix their own sampling | +| `reasoning_effort` | no | passed through where the provider accepts it | + +Unknown fields are ignored rather than rejected, so a newer client can talk to an +older engine without a version check. + +### Tool ownership is decided by the request + +This is the whole design. One rule, no modes to configure: + +- **`tools` present → the CALLER owns them.** The engine does not execute anything. + It emits `tool_calls` and ends the turn with `finish_reason: "tool_calls"`. The + caller executes them and sends a follow-up request carrying `role: "tool"` + messages. Standard OpenAI semantics, and stateless. +- **`tools` absent → the ENGINE owns them.** Present behaviour: the engine selects + and runs its own tools and returns final text. + +In v1 the two do not mix. Declaring `tools` disables engine-side tools. One +predictable rule beats three configurable ones, and it is the rule every caller +already assumes. + +Why it has to be this way: `redrob-office`'s tools edit an open document held in the +renderer. `redrob-browser`'s tools drive a live tab. Neither can be executed by the +engine — handed over, the engine would edit the disk instead of the document. Caller +ownership is not a preference, it is the only correct answer for those hosts. + +### Streaming + +`stream: true` returns `text/event-stream`: OpenAI delta chunks, then +`data: [DONE]`. Tool calls stream as `delta.tool_calls` fragments keyed by index, the +way callers already parse them. + +### Auth + +Inbound uses the server's existing HTTP Basic (`packages/redrob/src/server/auth.ts:18`). +Callers do NOT send a Console key. + +Outbound uses the engine's OWN Console credential — `REDROB_API_KEY`, then the +credential store, then `auth.json` (`packages/core/src/console-key.ts:34-72`). This is +the single-login property: one `redrob providers login` and every product is +authenticated, because no product holds a key of its own on this path. + +### Errors + +The OpenAI error envelope, so existing client error handling works unchanged: + +```json +{ "error": { "message": "...", "type": "...", "code": "..." } } +``` + +One code is load-bearing: when the engine itself has no Console credential, the reply +is `401` with `code: "engine_not_authenticated"`. That is the only condition under +which a client should offer a sign-in action. Clients must branch on this code and not +on the message text, which is localized. A client that offers sign-in on any failure +reports a network timeout as "please log in" — that shipped, and it is what this code +exists to prevent. + +## 2. Promoting `/experimental` + +Every `/experimental/*` route is promoted out of the experimental namespace and +becomes part of the stable surface. + +This is safe to do as one breaking rename, because **no product calls any of them.** +The only callers are this repo's own generated SDKs +(`packages/sdk/js/src/gen`, `packages/sdk/js/src/v2/gen`), which `httpapi-codegen` +regenerates from the route definitions. The one external reference to an experimental +path is `redrob-browser`'s dead completions call, which this design replaces anyway. + +So: rename the paths, regenerate the SDKs, and delete the experimental namespace. No +compatibility window is owed to callers that do not exist. + +## 3. One shared install, no bundling, no pin + +Products stop shipping a copy of this binary. `redrob-office` drops ~145 MB from its +installer and drops its packaging guard; `redrob-cowork` drops its download-and-stage +step. + +### Resolution order + +1. `REDROB_CODE_BIN` — explicit override, development +2. `$HOME/.redrob/bin/redrob` — where `install.sh` already installs +3. `PATH` +4. absent → the product offers a one-click install that runs the official installer + (GitHub Releases, verified against `SHA256SUMS`) + +### One daemon, not one per product + +A shared record at `$HOME/.redrob/run/server.json`, mode `0600`: + +```json +{ "port": 51234, "pid": 8412, "secret": "…", "version": "0.1.7" } +``` + +The first product that needs the engine starts it; the rest attach. Required +behaviour: liveness check before attaching, recovery from a stale record whose pid is +gone, and reconnection by attached products when an upgrade restarts the daemon. + +Per-product processes were considered and rejected: with local inference, N processes +means the same weights loaded N times, which defeats the reason for doing this. + +That file is the auth boundary — it carries the Basic secret. It is per-user by +location and must stay `0600`; a multi-user machine gets one daemon per user, never a +shared one. + +### Capability negotiation replaces pinning + +No product pins an engine version. Auto-update would otherwise break clients +silently. Instead `/capabilities` (promoted, formerly `/experimental/capabilities`, +handler at `packages/redrob/src/server/routes/instance/httpapi/handlers/experimental.ts:39-41`) +grows an entry: + +```json +{ "chatCompletions": { "version": 1, "callerTools": true, "stream": true } } +``` + +A product declares the MINIMUM capability it needs, never a version number. If the +entry is missing or its `version` is below that minimum, the product tells the user to +run `redrob upgrade` instead of failing with a transport error. + +### Backward compatibility, as rules rather than a promise + +- The contract is versioned by PATH (`/v1/...`), never by engine version. +- Adding a request or response field is allowed. Removing one, or changing what one + means, requires `/v2`. +- When `/v2` appears, `/v1` keeps being served for two minor versions. +- A contract test in this repo pins the v1 request and response schema and fails CI + when a field is removed or its type changes. Without that test "we keep + compatibility" is a sentence, not a guarantee. + +## 4. Migration + +| product | change | +| --- | --- | +| redrob-browser | point `sidecar_client.cc:592` at `/v1/chat/completions`; its host-owned tool loop already matches the contract | +| redrob-office | drop the bundled binary and the packaging guard; point the engine route at the local daemon; keep its editor-owned tool loop | +| redrob-cowork | drop the download-and-stage step and the duplicate alias copy; keep driving the session API for agent work | +| redrob-reblend | either pass the local base URL and allow loopback HTTP in its sidecar, or delete the LOCAL transport that cannot work | +| redrob-canvas, redrob-query, redrob-recall | optional: switching from the Console to the local daemon buys them one shared login | +| redrob-extension, redrob-cad | unchanged; they want engine-owned tools and the session API already serves that | + +## Open items + +- Whether `tools` and engine-owned tools may ever be combined in one turn. Deferred + out of v1 deliberately. +- Which local inference providers the registry should resolve first. From e152b21e1483940dbafc945c692a724781190139 Mon Sep 17 00:00:00 2001 From: Janghoon Lee <44862514+its-janghoon@users.noreply.github.com> Date: Tue, 22 Sep 2026 18:45:40 +0900 Subject: [PATCH 02/33] feat(server): OpenAI-compatible /v1/chat/completions on the engine (#58) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(api): the /v1/chat/completions contract and its wire conversion First half of the route from docs/LOCAL-ENGINE-API.md: the declared contract and the translation layer. No handler yet, so nothing is served — the endpoint is not reachable until the following commit wires it. The group goes on the v2 protocol surface rather than the instance httpapi: Authorization is declared once at that HttpApi level so a new group gets it instead of having to remember .middleware(Authorization), it has the typed SSE helper (HttpApiSchema.StreamSse), and it is the surface with a contract changelog. Paths there are literal, so /v1/... sits alongside /api/... without a prefix mechanism. Tool ownership is decided by the request and nothing else: `tools` present means the caller executes them and the turn ends with finish_reason tool_calls; `tools` absent means a plain completion. One rule rather than three configurable ones, and it is what Office and browser already assume — their tools mutate an open document and a live tab, which the engine cannot do on their behalf. Errors are one class per status sharing the OpenAI envelope. Collapsing them would report a rejected key as a bad request, and the status is what a client's transport sees first. 401 carries code `engine_not_authenticated`, which is the only condition a client may turn into a sign-in prompt. The conversion layer is pure — no Effect, no services — because it is where the silent failures live: a mistranslation does not fail the request, it just gives the model something other than what the caller wrote. What it refuses is the interesting part. Malformed tool_call arguments and an unsupported content part are rejected rather than forwarded as {} or dropped, since an image quietly discarded looks to a user like the model ignored it. A tool message with no tool_call_id is refused because the provider pairs results to calls by that id and there is nothing sensible to guess. Two asymmetries the tests pin down: the wire carries tool arguments as a JSON string while the engine wants them parsed, so both directions convert; and "error"/"unknown" finish reasons have no OpenAI equivalent and must not be reported as "stop", which would tell the caller a failed turn ended cleanly. 22 tests pass. protocol and server typecheck clean. * feat(api): serve POST /v1/chat/completions The handler for the contract in the previous commit. The route is now reachable. It touches no Session, Agent or tool runtime: it resolves a provider the way the agent does (catalog entry plus the active integration credential), makes one model call, and translates. Everything stateful stays in the session API. handleRaw rather than handle, because the reply is JSON or text/event-stream depending on `stream`, and only the raw form chooses its own response. The cost is that the payload is not decoded for you, so the declared schema is applied in the handler rather than the body being trusted. Streaming is where the care went. Tool calls are emitted from the completed `tool-call` event, not from `tool-input-delta`: the engine parses arguments and the wire wants a JSON string, so re-emitting fragments would mean re-serializing a half-parsed value. And `provider-error` arrives as a stream ELEMENT rather than a failure — left unhandled, an upstream 500 would end the stream with a clean [DONE] and the caller would read a truncated answer as a complete one, so it is matched explicitly and sent as an error frame. Once headers are out, a frame is the only way to say the turn broke. Error mapping keeps one value load-bearing: 401 with code engine_not_authenticated, covering both a missing credential and a rejected one — from the caller's side those are one problem and neither is fixed by retrying. What is NOT remapped is a content-policy classification: the executor checks the body before the status, so an upstream 401 mentioning safety arrives as ContentPolicy, and reclassifying it here would report a policy refusal as a login problem. A caller-supplied base URL is deliberately not accepted. The config layer only admits a new openai-compatible provider at a local address (isLocalEndpoint); honouring an arbitrary URL here would bypass that and forward the engine's own credential to whatever host the caller named. A local model is selected by its `provider/model` id instead. Four APIs were guessed wrong first and corrected against the source, not worked around: this Effect version has no Effect.either, no Effect.catchAll and no Stream.mapConcat, LLMResponse carries finishReason rather than a finish object, and catalog.model.get returns an Effect on the Service (only the Draft form is synchronous). tool_choice now travels as the engine's own shape instead of a bare string. A bare string is ambiguous — the normalizer reads auto/none/required as modes and anything else as a tool name — so a tool actually named `auto` would silently become a mode. A test pins that case. 23 tests pass; protocol and server typecheck clean. Still to come: the capabilities flag, the httpapi-exercise scenario (test:httpapi runs --fail-on-missing, so CI will fail until it exists), the SSE OpenAPI patch in public.ts, and a route test. * docs: provider authentication, and what each vendor actually permits The design of record for how a user connects their own model access, so one setup serves code, cowork, office and whatever comes after. The product ask was 'let people use the Claude or ChatGPT they already pay for'. Half of that is not available, and the split is contractual rather than technical, so the document leads with the survey and links every claim. Not available: a sign-in button in our own apps that spends a consumer subscription. Anthropic's compliance page forbids third parties offering Claude.ai login or routing through Free/Pro/Max credentials, and bans accounts for spoofing the Claude Code harness - our own upstream removed Claude support on 2026-02-19 under legal request. Google's Gemini CLI FAQ says piggybacking its OAuth may terminate the account, and there is a documented mass-ban thread. OpenAI allows it in practice, but only by embedding OpenAI's own Codex runtime, and no clause grants it, so it is a revocable product decision. Available, and it covers most of the intent: BYOK everywhere (Anthropic and Google both name a user API key as THE supported third-party path), plus three real sign-in buttons - GitHub Copilot through the Copilot SDK, which spends the user's own Copilot subscription with GitHub's blessing and a documented desktop device flow; OpenRouter OAuth PKCE, which reaches Claude and GPT models behind an account login and is the closest legitimate thing to what was asked for; and Azure OpenAI via Entra device code. The design itself adds little and removes a lot. Credentials stay in this engine's existing auth.json, and products stop having stores of their own - today Office, Design, the extension, Query and Recall keep five stores that never read each other, which is the whole reason a user logs in again in every app. Products never hold a key: /v1/chat/completions already carries the engine's credential outward and takes none from the caller, so adding a provider is one change here rather than a release in every app, and an app that holds no key cannot leak one. Four routes are proposed to expose the login flow that already exists in , with three rules: the key never comes back out, the OAuth code stays in the engine, and no caller-supplied base URL - that last one because honouring it would forward this engine's credential to whatever host the caller named. Also records a compliance check to keep: this repository contains no Anthropic OAuth client id and no claude.ai endpoint, and the only hardcoded vendor OAuth is xAI's. * docs: local harness backends for every product, from one place Codex and Claude Code as an option in all ten products from one implementation in this engine, instead of ten adapters in ten repositories. Corrects where the first adapter was put. It was written inside redrob-cowork behind that app's engine-spawn hook; it works and it found real bugs, but it gives Cowork a Codex option and the other nine nothing. It belongs here, because every product already reaches a model through this engine and the ones that still call the Console directly are what /v1/chat/completions was built to bring in. The mechanism turned out to already exist. config/plugin/local-provider.ts admits a provider when the package is exactly the trusted @ai-sdk/openai-compatible and the URL is on this machine - a door opened for Ollama, LM Studio, llama.cpp and vLLM whose only shared trait is speaking OpenAI-compatible chat-completions locally. A small local shim in front of codex exec or claude -p meets the same bar, so this needs no new provider-loading machinery and does not widen the arbitrary-npm-package refusal by a millimetre. This recommends a shim where docs/CODEX-RUNTIME.md argues against one, and the doc says why rather than pretending to be consistent: a Cowork shim must emulate the OpenCode session API, which is large, ours and still changing, so falling subtly behind produces bugs that look like model bugs. A shim here emulates chat-completions, which is small, published, frozen and not ours. Different surface, different answer. What cannot be papered over: these are agent harnesses, not completion endpoints, so support has two tiers. The chat tier - prompt in, text out - covers every product's ordinary use and needs no per-product code. The agent tier, where the runtime owns a real loop against a workspace, cannot go through chat-completions at all, because that contract says tools present means the caller owns them while here the runtime owns the loop. Cowork, code and cad need it; it must not block the chat tier. One safety point that is easy to miss: a harness with a writable sandbox will read and edit the user's filesystem, and a user asking Office to reword a sentence has not consented to an agent walking their disk. The shim pins read-only, approvals never, network off, unless an agent-tier caller asked for more. Sequencing puts /v1/chat/completions first because nothing here ships before it, and records two things to carry rather than rediscover: claude -p's stream-json schema is unverified against the real binary, and Anthropic has announced then paused a change capping third-party subscription usage, so the Claude backend should surface usage state rather than assume it is free. * feat(server): close the CI gates on /v1/chat/completions The route had a handler and a conversion layer but nothing that made CI accept it. This adds the coverage scenario, the capability flag, and route tests - and the tests immediately found a real bug in the handler. The bug: a body that does not decode against ChatCompletionRequest came back as 502 api_error. schemaBodyJson fails with HttpServerError | Schema.SchemaError, and neither was recognised, so both fell through to the catch-all that reports an upstream failure. A caller who sent a malformed payload was told our upstream had failed, which sends them debugging the wrong system. Now classified 400 invalid_request_error. Matched on _tag rather than instanceof, because Schema.SchemaError and the HTTP request errors are separate hierarchies and an instanceof chain over both stops matching silently when either is re-exported through a different module instance. The coverage gate was mandatory and I nearly talked myself out of it. Reading routing.ts, coverage derives from routeKeys(OpenApi.fromApi(modules.PublicApi)) -- the instance httpapi -- while this route is registered on the v2 protocol surface, so I concluded it was out of scope. Running it said otherwise: PublicApi merges the v2 groups, the exerciser reported MISS POST /v1/chat/completions, and --fail-on-missing failed the run. The scenario is real and the reading was wrong. That scenario deliberately exercises the 400 path. A real completion needs a provider credential this harness has none of, so asserting 200 would make the gate depend on the machine it runs on. An unknown model id is refused before any provider resolves, which is deterministic and still covers what most likely breaks: that the body decodes, and that the failure arrives in the OpenAI error envelope rather than Effect's default shape. Verified in effect mode against a real server, not just in coverage mode. The capability flag is chatCompletions: { version: 1, callerTools: true, stream: true } on /experimental/capabilities, with a matching struct in the group schema. Declared as a struct rather than a version number so a caller negotiates a capability instead of pinning an engine build - an app needing caller-owned tools checks callerTools, and an engine without the route omits the key entirely. version is the contract's revision, not the engine's. Five route tests drive the real app in-process through httpapi-layer. The envelope assertions are the point: a caller written against OpenAI reads error.message and error.type, so the tests also assert Effect's { name, data } shape does NOT leak. One test pins the v1 contract by sending every promised field including the caller-owned tools path, and checks the failure is about the model rather than a decode, so dropping a field from the schema fails it. Two things deliberately NOT in this change. The SDK codegen is not regenerated. bun script/generate.ts also runs a formatter across the repo (61 files, mostly markdown table alignment) and its openapi.json output contains routes that are not mine - /api/variant/paraphrase among them - which means the generated SDK was already stale before this branch. Regenerating here would sweep another session's unregenerated surface into this PR. It needs its own commit. This repo has no CI check for stale codegen, which is why it drifted. The SSE response is still undocumented in OpenAPI. The group declares only ChatCompletionResponse, so the streaming path has the same gap /event has and the same fix site in public.ts. Doing it properly means defining a ChatCompletionChunk schema in the protocol, which is more than closing a gate, so it is recorded rather than half-done. Coverage 211 pass / 0 fail / 0 missing. Route tests 5/5. Wire tests 23/23. Typecheck clean on both packages. * fix(protocol): stop declaring endpoint errors codegen cannot discriminate CI failed six jobs with: Promise error must have a literal discriminator: server.chat.chat.completions httpapi-codegen requires every declared endpoint error to carry a _tag or name STRING LITERAL to branch on (declaredErrorFields). The four error classes shared one OpenAI envelope and had nothing to discriminate by, so generation threw and every job that generates the client died with it. Removed the error declarations rather than adding a discriminator, for two independent reasons: The handler never used them. It is handleRaw - one request answers with JSON and another with text/event-stream - so it writes every response itself, failures included, through errorResponse(). The declared classes were decoration that had never been on the path. And adding _tag to satisfy the generator would make the generated spec and SDK claim a field the wire does not carry, because OpenAI's envelope has no such key. A spec that lies about the shape is worse than one that is silent about it. So the statuses are documented in the endpoint description instead - 400, 401 with code engine_not_authenticated as the only sign-in trigger, 429, 502 - and the envelope survives as an exported ChatCompletionError type so the handler and any caller build against one shape. docs/LOCAL-ENGINE-API.md already specifies the wire contract. Note this was caught by a real CI gate I had earlier described as absent. The generated client IS checked: packages/client check:generated regenerates and fails on any diff. That is a different check from the SDK staleness I noted in the PR body, and the generated client output is committed here for it. Coverage 211/0/0. Route + wire tests 28/28. protocol, server and client typecheck clean. * test: give the fake LLM server its own router, not the engine's CI failed with: Method 'POST' already declared for route '/v1/chat/completions' taking down every test that stands up both the engine's HttpApi and the fake provider (httpapi-sdk prompt streaming, project-skills prompt context). TestLLMServer piped Layer.provide(HttpRouter.layer), which layer memoization resolves to the SAME HttpRouter instance the engine's HttpApi is built on, so the fake and the app registered into one router. That was invisible while no engine route shared a path with the fake. Adding POST /v1/chat/completions to the engine collided with llm-server.ts:702, which serves that exact path. Fixed with Layer.fresh so the fake gets its own router instance. The fake's path is deliberately NOT renamed to dodge the clash. It exists to look like a real OpenAI-compatible endpoint, and /v1/chat/completions is where a real one lives; moving it to a made-up path would stop the fake exercising what the tests are actually about. The collision was a wiring accident, not a naming one, so the wiring is what changed. Worth noting the failure mode: the fake had been sharing the app's router all along, which means any future engine route matching a test double's path would have broken the same way. This makes that class of collision impossible rather than resolving one instance of it. All 130 tests across the 12 files that use TestLLMServer pass (1 pre-existing skip). httpapi-sdk 18/18. --------- Co-authored-by: Janghoon Lee <44862514+savagemanage@users.noreply.github.com> --- docs/LOCAL-HARNESS-BACKENDS.md | 146 ++++ docs/PROVIDER-AUTH.md | 189 +++++ .../client/src/generated-effect/client.ts | 42 ++ packages/client/src/generated/client.ts | 32 + packages/client/src/generated/types.ts | 674 ++++++++++++++++++ packages/protocol/src/api.ts | 5 + .../protocol/src/groups/chat-completion.ts | 200 ++++++ .../instance/httpapi/groups/experimental.ts | 16 + .../instance/httpapi/handlers/experimental.ts | 9 +- packages/redrob/test/lib/llm-server.ts | 18 +- .../test/server/chat-completion-wire.test.ts | 242 +++++++ .../server/httpapi-chat-completion.test.ts | 147 ++++ .../test/server/httpapi-exercise/index.ts | 29 +- packages/server/src/handlers.ts | 2 + .../src/handlers/chat-completion-wire.ts | 231 ++++++ .../server/src/handlers/chat-completion.ts | 317 ++++++++ 16 files changed, 2294 insertions(+), 5 deletions(-) create mode 100644 docs/LOCAL-HARNESS-BACKENDS.md create mode 100644 docs/PROVIDER-AUTH.md create mode 100644 packages/protocol/src/groups/chat-completion.ts create mode 100644 packages/redrob/test/server/chat-completion-wire.test.ts create mode 100644 packages/redrob/test/server/httpapi-chat-completion.test.ts create mode 100644 packages/server/src/handlers/chat-completion-wire.ts create mode 100644 packages/server/src/handlers/chat-completion.ts diff --git a/docs/LOCAL-HARNESS-BACKENDS.md b/docs/LOCAL-HARNESS-BACKENDS.md new file mode 100644 index 0000000000..0ffdaec883 --- /dev/null +++ b/docs/LOCAL-HARNESS-BACKENDS.md @@ -0,0 +1,146 @@ +# Local harness backends, for every product at once + +How Codex and Claude Code become an option in *all* Redrob products — office, +cowork, browser, design, extension, canvas, query, recall, cad, reblend — from one +implementation in this engine, rather than ten adapters in ten repositories. + +Companion documents: `docs/PROVIDER-AUTH.md` (what each vendor permits, and why we +never hold a token) and `docs/LOCAL-ENGINE-API.md` (the `/v1/chat/completions` +contract this rides on). redrob-cowork `docs/CODEX-RUNTIME.md` holds the working +prototype that proved the subprocess mechanics. + +## The mistake this corrects + +The first adapter was written inside redrob-cowork, behind that app's engine-spawn +hook. It works, and it found real bugs, but its home is wrong: it gives Cowork a +Codex option and gives the other nine products nothing. Ten products would mean +ten adapters, ten settings screens, ten credential stories, and ten places for the +subprocess environment bug described below to be re-introduced. + +The adapter belongs here instead, because every product already reaches a model +through this engine — and the ones that still call the Console directly are exactly +the ones `/v1/chat/completions` was built to bring in. + +## The mechanism already exists + +`packages/core/src/config/plugin/local-provider.ts` admits a provider when two +conditions hold: the package is exactly `@ai-sdk/openai-compatible` (the trusted, +pinned one the Console provider itself uses), and the URL is on this machine or +network. That door was opened for Ollama, LM Studio, llama.cpp and vLLM, whose only +shared trait is that they **speak OpenAI-compatible chat-completions over a local +address**. + +A harness can meet the same bar. Put a small local server in front of `codex exec` +or `claude -p` that speaks chat-completions, and it is admissible through a path +this engine already trusts — no new provider-loading machinery, no widening of the +arbitrary-npm-package refusal, no second credential store. + +``` +codex exec / claude -p subprocess; holds its OWN credential + ▲ + harness shim local, OpenAI-compatible, 127.0.0.1 + ▲ + redrob-code engine registered via the existing local-provider path + ▲ + POST /v1/chat/completions one receiving route + ▲ + office · cowork · browser · design · extension · canvas · query · recall · cad · reblend +``` + +Models then select a backend by id — `codex/`, `claude-code/` — which +is the same way a local model is already selected, and needs no new request field. + +### Why a shim here, having argued against one in Cowork + +`docs/CODEX-RUNTIME.md` recommends *against* a protocol shim in Cowork and this +document recommends one. That is not an inconsistency, it is the surface being +different, and the difference is the whole argument. + +A Cowork shim would have to emulate the **OpenCode session API** — sessions, the SSE +event shape, permissions — which is large, ours, and still changing. A shim that +falls subtly behind produces bugs that look like model bugs. A shim here emulates +**chat-completions**, which is small, published, frozen, and not ours to change. The +first is a maintenance liability; the second is an adapter against a stable +contract. + +## Two tiers, and most products only need the first + +This is the part that cannot be papered over: **Codex and Claude Code are agent +harnesses, not completion endpoints.** They run their own loop with their own +tools. So there are two levels of support, not one. + +**Chat tier — all ten products.** Prompt in, text out. The harness's loop runs but +its tools are constrained to nothing the caller did not ask for. This covers every +product's ordinary AI use: rewrite this paragraph, summarise this sheet, answer +this question. It maps cleanly onto `/v1/chat/completions` and needs no per-product +code. + +**Agent tier — cowork, code, cad.** The harness runs a real agent loop against a +workspace. chat-completions cannot express this, because that contract's rule is +"tools present → the caller owns them", and here the *runtime* owns the loop. This +needs ACP or a session surface, and it is separate work. Do not let it block the +chat tier. + +A safety point that belongs in the chat tier and is easy to miss: a harness given a +writable sandbox will happily read and edit the user's filesystem. A user asking +Office to reword a sentence has not consented to an agent walking their disk. So +the shim pins `--sandbox read-only` with approvals set to never, and network access +off, unless a caller is on the agent tier and asked for more. The prototype already +defaults this way; the shim must not relax it for convenience. + +The tool-ownership split surveyed earlier decides which products need more than the +chat tier: + +- **No tool protocol** — query, recall, reblend. Chat tier is the whole story. +- **Host-owned tools** — office, browser, canvas. Chat tier works today. Giving the + harness their tools needs MCP servers (`~/.codex/config.toml`, or per-invocation + `--config mcp_servers.…`) or Codex app-server's `dynamicTools`, which lets the + tool stay in the host process. `dynamicTools` is the better fit and is labelled + experimental by OpenAI, so it is not a foundation to build on yet. +- **Engine-owned tools** — cowork, cad, extension, design. Under a harness backend + the engine's own tools are not in the loop at all. That is the agent tier's + problem to solve. + +## What the products have to do + +Almost nothing, which is the point. + +Nothing at all to *work*: a product that names a model and calls the engine gets +the new backends when the engine gets them. + +One thing to be *usable*: somewhere to turn it on, and three states rather than +one — runtime not installed, installed but not signed in, ready. Collapsing those +into "unavailable" strands the user, because the remedy differs and we are not +allowed to offer the vendor's login ourselves. The remedy we may show is "run +`codex login`" or "run `claude`". + +Credentials need no work anywhere. The harness holds its own; the engine holds none +for it. Cowork already demonstrates the pattern for the BYOK case — it stores no +provider key and treats the engine's `auth.json` as the single source of truth. + +## Sequencing + +1. **Land `/v1/chat/completions`.** It is the receiving route for everything above + and it is not merged: the handler exists on `feature/v1-chat-completions`, and + `test:httpapi` fails without an `httpapi-exercise` scenario. Also needs the + `chatCompletions` capability flag, the SSE OpenAPI patch, and a route test. + Nothing here can ship before it. +2. **Build the shim with both backends.** One local chat-completions server; two + normalizers behind it. The prototype's split — event folding separated from the + subprocess — is what makes the second runtime a second normalizer rather than a + second architecture. Reuse it rather than re-deriving it. +3. **Register through the local-provider path**, and surface the backends in + `/v1/providers` with their three states so a product can render settings + without hardcoding a list. +4. **Verify against real binaries.** Neither runtime is installed on the build + host, so the live path — a real subscription actually paying for a turn — is + unproven until someone runs it on a machine with `codex` and `claude` signed in. +5. **Then the agent tier**, for cowork, code and cad, over ACP. + +Two things to carry forward rather than discover later. `claude -p`'s +`stream-json` event schema has not been checked against the real binary — only the +flags are confirmed — so step 2 starts by reading it, not by assuming it mirrors +Codex's JSONL. And Anthropic has announced, then paused, a change that moves +third-party subscription usage onto a capped monthly credit; it currently still +draws from the subscription, but the trajectory is known, so the Claude backend +should surface usage state rather than assume it is free. diff --git a/docs/PROVIDER-AUTH.md b/docs/PROVIDER-AUTH.md new file mode 100644 index 0000000000..e478ac6413 --- /dev/null +++ b/docs/PROVIDER-AUTH.md @@ -0,0 +1,189 @@ +# Provider authentication + +How a user connects their own model access to this engine, and therefore to every +product built on it. One credential store, read by all of them, so a user sets up +once rather than once per app. + +The goal this answers is "let people use the Claude or ChatGPT access they already +pay for". Part of that is available and part of it is not, and the split is not +technical — it is what each vendor's terms permit a third-party application to do. +So the shapes are enumerated first, then the design. + +Not legal advice. Every claim below links the document it came from, checked +2026-09-22; re-check before shipping, because three of these pages changed in the +first half of this year. + +## What each vendor actually allows a third-party app + +| vendor | "sign in" for a third-party app | user's own API key | user's consumer subscription | +| --- | --- | --- | --- | +| Anthropic | **No** — expressly forbidden | Yes, expressly | **No** — prohibited, enforced with account bans | +| OpenAI | No self-serve registration; granted case by case | Yes | Only by embedding OpenAI's own Codex runtime | +| Google Gemini | OAuth exists but bills *your* Cloud project | Yes — the named supported path | **No** — prohibited, enforced with bans | +| GitHub Copilot | **Yes** — the Copilot SDK, officially | Yes | Yes, billed to the user's own subscription | +| OpenRouter | **Yes** — OAuth PKCE | Yes | n/a (it is BYOK by design) | +| Azure OpenAI | **Yes** — Microsoft Entra ID | Yes | n/a (user's own Azure resource) | +| Amazon Bedrock | No (SigV4 / Bedrock API keys) | Yes — the user's own AWS account | n/a | +| AWS Kiro | **No** | Kiro API key, subscriber-only | Only by driving the user's own installed CLI | + +Sources, and the sentences that decide it: + +- Anthropic, [Claude Code legal and compliance](https://code.claude.com/docs/en/legal-and-compliance): + "Anthropic does not permit third-party developers to offer Claude.ai login into + their own applications, or to route requests through Free, Pro, or Max plan + credentials on behalf of their users." And developers "should use API key + authentication through Claude Console or a supported cloud provider." Enforced: + accounts were banned for spoofing the Claude Code harness, and our own upstream + (OpenCode) removed Claude subscription AND Claude API key support on 2026-02-19 + citing Anthropic legal requests ([The Register, + 2026-02-20](https://www.theregister.com/software/2026/02/20/anthropic-clarifies-ban-on-third-party-tool-access-to-claude/5014546)). + The only sanctioned subscription route is shipping Claude Code itself, + unmodified, with its auth methods intact. +- OpenAI, [Codex authentication](https://developers.openai.com/codex/auth/): + "Sign in with ChatGPT" is documented for OpenAI's own surfaces only. Third + parties that have it — Zed, OpenClaw — get there by wrapping OpenAI's own Codex + runtime ([Zed](https://zed.dev/blog/chatgpt-subscription-in-zed)). Treat that as + a revocable product decision, not an entitlement: no terms clause grants it. +- Google, [Gemini CLI FAQ](https://github.com/google-gemini/gemini-cli/blob/main/docs/resources/faq.md): + "the supported and secure method is to use a Vertex AI or Google AI Studio API + key", and piggybacking Gemini CLI's OAuth "may be grounds for immediate + suspension or termination". Enforced in + [this thread](https://github.com/google-gemini/gemini-cli/discussions/20632). + Note the Gemini API terms also say the API is "not for consumer use" and + require paid services for EEA/UK/CH users. +- GitHub, [Copilot SDK OAuth setup](https://docs.github.com/en/copilot/how-tos/copilot-sdk/setup/github-oauth): + "Copilot requests are made on behalf of each authenticated user, using their + Copilot subscription… Your app never handles model API keys." A device flow is + documented for exactly our case, "Desktop applications where users interact + directly". +- OpenRouter, [OAuth PKCE](https://openrouter.ai/docs/guides/overview/auth/oauth): + send the user to `/auth` with a `code_challenge`, exchange the code for a + **user-controlled API key**. Loopback callback on any port, plus a headless + paste mode. +- Azure, [Entra ID auth](https://learn.microsoft.com/azure/ai-services/openai/how-to/managed-identity) + with the [device authorization grant](https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-device-code). +- Kiro, [Authentication](https://kiro.dev/docs/getting-started/authentication/): + subscriber `ksk_` API keys exist; the only documented embed path is driving the + user's own CLI over [ACP](https://kiro.dev/docs/cli/acp/). + +### What this means for the product ask + +"Sign in with Claude" and "Sign in with ChatGPT", as buttons in our own apps +spending the user's consumer subscription, are not available. Building them means +impersonating a first-party client, and the vendors ban accounts for it — the cost +lands on our users, not on us. + +What IS available, and covers most of the intent: + +1. **BYOK for every provider.** Anthropic and Google both name this as the + supported path for third-party tools. A user with a Claude API key or a Gemini + key connects in one step. +2. **Three real sign-in buttons**: GitHub Copilot, OpenRouter, Azure OpenAI. The + first two are the interesting ones — Copilot spends the user's own Copilot + subscription with GitHub's blessing, and OpenRouter fronts Claude and GPT + models behind an account login, which is the closest legitimate thing to what + was asked for. +3. **Optionally, unmodified first-party binaries.** Shipping Claude Code or the + Codex CLI as-is and letting it authenticate itself is sanctioned by both + vendors. It is a different product shape — their harness, not ours — so it is + recorded here as available rather than recommended. + +## Design + +### One store, in the engine + +Credentials live where they already live: `Global.Path.data/auth.json`, through +`packages/redrob/src/auth`, whose `Info` union is already `Oauth | Api | +WellKnown`. `packages/core/src/console-key.ts` reads env → credential store → +that file, and `Integration` resolves a connection into a `Credential.Value` for +the provider layer. + +Nothing new is invented, because the point is that products stop having stores of +their own. Today Office keeps keys in `userData/ai-settings.json`, Design in the +macOS keychain, the extension in `chrome.storage.local`, Query and Recall in +separate keyring services — five stores that never read each other, which is why +a user logs in again in every app. + +A product reads the engine's store instead. It keeps its own as a cache if it +wants, but the engine's file is the source of truth, and the engine is what makes +the call. + +### Products never hold a key + +`POST /v1/chat/completions` (see `docs/LOCAL-ENGINE-API.md`) already carries the +engine's credential outward and takes none from the caller. That is the whole +mechanism: a product names a `model`, the engine resolves the provider and its +credential. Adding a provider is then a change in one place, and every downstream +app gets it without shipping a release. + +This also removes a class of bug rather than moving it: a product that holds no +key cannot leak one, log one, or sync one. + +### Connecting a provider + +`redrob providers login` already implements both shapes — a generic OAuth flow +with `authorize()` plus `auto` and `code` callbacks, and an API-key path +(`packages/redrob/src/cli/cmd/providers.ts`). What is missing is not the flow but +its exposure: a product cannot drive it today. + +Add, on the v2 surface beside the completions route: + +- `GET /v1/providers` — what this engine can use, each entry declaring which auth + methods it accepts (`oauth`, `api-key`) and whether a credential is present. + This is what lets an app render a settings page without hardcoding a vendor + list that goes stale. +- `POST /v1/providers/:id/login` — starts a flow. For OAuth it returns the + authorization URL and an opaque attempt id; for an API key it accepts the key. +- `POST /v1/providers/:id/login/:attempt` — completes an OAuth attempt with the + authorization code, or reports that the loopback callback already completed it. +- `DELETE /v1/providers/:id/credential` — disconnect. + +Three rules on those routes: + +1. **The key never comes back out.** A response says a credential is present and + names it; it never returns the secret. A product that cannot read the key + cannot mishandle it, and this is also what keeps the loopback API from becoming + a credential-exfiltration endpoint if something else on the machine reaches it. +2. **The OAuth code stays in the engine.** The product gets the URL to open and an + opaque id, nothing else — the same split `apps/shell/src/main/redrob-connect.ts` + already uses between Electron's main process and its renderer. +3. **No caller-supplied base URL.** Keep this ban. It is not only policy: the + config layer admits a new openai-compatible provider only at a local address + (`isLocalEndpoint`), and honouring an arbitrary URL from a request would + forward the engine's own credential to whatever host the caller named. A local + model is selected by its `provider/model` id. + +### Which providers get a login button + +Ship BYOK for all of them. Add OAuth only where the vendor documents it for third +parties: GitHub Copilot, OpenRouter, Azure OpenAI. Anthropic, Google, OpenAI and +Bedrock get an API-key field and no button. + +The UI should say which is which. A user who expects "sign in with Claude" and +finds a key field deserves the reason in one line — that Anthropic requires an API +key for third-party tools — rather than being left to assume the feature is +missing. + +### Order of work + +1. `GET /v1/providers` and the API-key path. This alone gives Office, Cowork and + Design one shared BYOK setup, and needs no vendor negotiation. +2. OpenRouter OAuth. Smallest real sign-in button and the one that reaches Claude + and GPT models legitimately. +3. GitHub Copilot via the Copilot SDK. A user's existing Copilot subscription, + sanctioned, with a documented device flow for desktop. +4. Azure OpenAI via Entra device code. + +## What must not be built + +- Reusing Claude Code's or the Codex CLI's OAuth client id to spend a consumer + subscription from our own harness. Anthropic and Google both ban accounts for + it; our upstream removed the code under legal pressure. +- Collecting, storing or proxying Claude.ai session tokens. Named explicitly in + Anthropic's compliance page. +- Paying for or intermediating another vendor's usage on a user's behalf. Also + named there, and it is what "just put our key in it" would amount to. + +A compliance check worth keeping: this repository currently contains no Anthropic +OAuth client id and no `claude.ai` endpoint. The only hardcoded vendor OAuth is +xAI's (`packages/redrob/src/plugin/xai.ts`). Keep it that way. diff --git a/packages/client/src/generated-effect/client.ts b/packages/client/src/generated-effect/client.ts index a0e209872b..77b6b52a9d 100644 --- a/packages/client/src/generated-effect/client.ts +++ b/packages/client/src/generated-effect/client.ts @@ -705,6 +705,47 @@ const adaptGroup18 = (raw: RawClient["server.projectCopy"]) => ({ refresh: Endpoint18_2(raw), }) +type Endpoint19_0Request = Parameters[0] +type Endpoint19_0Input = { + readonly model: Endpoint19_0Request["payload"]["model"] + readonly messages: Endpoint19_0Request["payload"]["messages"] + readonly tools?: Endpoint19_0Request["payload"]["tools"] + readonly tool_choice?: Endpoint19_0Request["payload"]["tool_choice"] + readonly stream?: Endpoint19_0Request["payload"]["stream"] + readonly max_tokens?: Endpoint19_0Request["payload"]["max_tokens"] + readonly max_completion_tokens?: Endpoint19_0Request["payload"]["max_completion_tokens"] + readonly temperature?: Endpoint19_0Request["payload"]["temperature"] + readonly top_p?: Endpoint19_0Request["payload"]["top_p"] + readonly stop?: Endpoint19_0Request["payload"]["stop"] + readonly seed?: Endpoint19_0Request["payload"]["seed"] + readonly frequency_penalty?: Endpoint19_0Request["payload"]["frequency_penalty"] + readonly presence_penalty?: Endpoint19_0Request["payload"]["presence_penalty"] + readonly reasoning_effort?: Endpoint19_0Request["payload"]["reasoning_effort"] + readonly user?: Endpoint19_0Request["payload"]["user"] +} +const Endpoint19_0 = (raw: RawClient["server.chat"]) => (input: Endpoint19_0Input) => + raw["chat.completions"]({ + payload: { + model: input["model"], + messages: input["messages"], + tools: input["tools"], + tool_choice: input["tool_choice"], + stream: input["stream"], + max_tokens: input["max_tokens"], + max_completion_tokens: input["max_completion_tokens"], + temperature: input["temperature"], + top_p: input["top_p"], + stop: input["stop"], + seed: input["seed"], + frequency_penalty: input["frequency_penalty"], + presence_penalty: input["presence_penalty"], + reasoning_effort: input["reasoning_effort"], + user: input["user"], + }, + }).pipe(Effect.mapError(mapClientError)) + +const adaptGroup19 = (raw: RawClient["server.chat"]) => ({ completions: Endpoint19_0(raw) }) + const adaptClient = (raw: RawClient) => ({ health: adaptGroup0(raw["server.health"]), location: adaptGroup1(raw["server.location"]), @@ -725,6 +766,7 @@ const adaptClient = (raw: RawClient) => ({ questions: adaptGroup16(raw["server.question"]), references: adaptGroup17(raw["server.reference"]), projectCopies: adaptGroup18(raw["server.projectCopy"]), + "server.chat": adaptGroup19(raw["server.chat"]), }) export const make = (options?: { readonly baseUrl?: URL | string }) => diff --git a/packages/client/src/generated/client.ts b/packages/client/src/generated/client.ts index 521b25e0b4..211fad5430 100644 --- a/packages/client/src/generated/client.ts +++ b/packages/client/src/generated/client.ts @@ -116,6 +116,8 @@ import type { ProjectCopiesRemoveOutput, ProjectCopiesRefreshInput, ProjectCopiesRefreshOutput, + ServerChatCompletionsInput, + ServerChatCompletionsOutput, } from "./types" import { ClientError } from "./client-error" @@ -1017,6 +1019,36 @@ export function make(options: ClientOptions) { requestOptions, ), }, + "server.chat": { + completions: (input: ServerChatCompletionsInput, requestOptions?: RequestOptions) => + request( + { + method: "POST", + path: `/v1/chat/completions`, + body: { + model: input["model"], + messages: input["messages"], + tools: input["tools"], + tool_choice: input["tool_choice"], + stream: input["stream"], + max_tokens: input["max_tokens"], + max_completion_tokens: input["max_completion_tokens"], + temperature: input["temperature"], + top_p: input["top_p"], + stop: input["stop"], + seed: input["seed"], + frequency_penalty: input["frequency_penalty"], + presence_penalty: input["presence_penalty"], + reasoning_effort: input["reasoning_effort"], + user: input["user"], + }, + successStatus: 200, + declaredStatuses: [401, 400], + empty: false, + }, + requestOptions, + ), + }, } } diff --git a/packages/client/src/generated/types.ts b/packages/client/src/generated/types.ts index 948d394efd..5db37e3579 100644 --- a/packages/client/src/generated/types.ts +++ b/packages/client/src/generated/types.ts @@ -2881,3 +2881,677 @@ export type ProjectCopiesRefreshInput = { } export type ProjectCopiesRefreshOutput = void + +export type ServerChatCompletionsInput = { + readonly model: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["model"] + readonly messages: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["messages"] + readonly tools?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["tools"] + readonly tool_choice?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["tool_choice"] + readonly stream?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["stream"] + readonly max_tokens?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["max_tokens"] + readonly max_completion_tokens?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["max_completion_tokens"] + readonly temperature?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["temperature"] + readonly top_p?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["top_p"] + readonly stop?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["stop"] + readonly seed?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["seed"] + readonly frequency_penalty?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["frequency_penalty"] + readonly presence_penalty?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["presence_penalty"] + readonly reasoning_effort?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["reasoning_effort"] + readonly user?: { + readonly model: string + readonly messages: ReadonlyArray<{ + readonly role: "system" | "developer" | "user" | "assistant" | "tool" + readonly content?: string | null | ReadonlyArray<{ readonly [x: string]: unknown }> | undefined + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly tool_call_id?: string | undefined + readonly name?: string | undefined + }> + readonly tools?: + | ReadonlyArray<{ + readonly type: "function" + readonly function: { + readonly name: string + readonly description?: string | undefined + readonly parameters?: { readonly [x: string]: unknown } | undefined + } + }> + | undefined + readonly tool_choice?: + | "auto" + | "none" + | "required" + | { readonly type: "function"; readonly function: { readonly name: string } } + | undefined + readonly stream?: boolean | undefined + readonly max_tokens?: number | undefined + readonly max_completion_tokens?: number | undefined + readonly temperature?: number | undefined + readonly top_p?: number | undefined + readonly stop?: string | ReadonlyArray | undefined + readonly seed?: number | undefined + readonly frequency_penalty?: number | undefined + readonly presence_penalty?: number | undefined + readonly reasoning_effort?: string | undefined + readonly user?: string | undefined + }["user"] +} + +export type ServerChatCompletionsOutput = { + readonly id: string + readonly object: "chat.completion" + readonly created: number + readonly model: string + readonly choices: ReadonlyArray<{ + readonly index: number + readonly message: { + readonly role: "assistant" + readonly content: string | null + readonly tool_calls?: + | ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } + }> + | undefined + readonly reasoning_content?: string | undefined + } + readonly finish_reason: string | null + }> + readonly usage?: + | { readonly prompt_tokens: number; readonly completion_tokens: number; readonly total_tokens: number } + | undefined +} diff --git a/packages/protocol/src/api.ts b/packages/protocol/src/api.ts index e7256237ca..9280a31083 100644 --- a/packages/protocol/src/api.ts +++ b/packages/protocol/src/api.ts @@ -22,6 +22,7 @@ import { LocationGroup } from "./groups/location" import { IntegrationGroup } from "./groups/integration" import { CredentialGroup } from "./groups/credential" import { ProjectCopyGroup } from "./groups/project-copy" +import { ChatCompletionGroup } from "./groups/chat-completion" // Protocol owns middleware placement, while Server injects concrete keys so Core service identities stay downstream. const makeApiFromGroup = < @@ -55,6 +56,10 @@ const makeApiFromGroup = < .add(makeQuestionGroup(locationMiddleware, sessionLocationMiddleware)) .add(ReferenceGroup.middleware(locationMiddleware)) .add(ProjectCopyGroup.middleware(locationMiddleware)) + // The OpenAI-compatible inference route. It takes locationMiddleware like the + // rest: the provider registry it resolves against is per-location, so a + // project-local provider must be visible to it. + .add(ChatCompletionGroup.middleware(locationMiddleware)) .annotateMerge( OpenApi.annotations({ title: "redrob HttpApi", diff --git a/packages/protocol/src/groups/chat-completion.ts b/packages/protocol/src/groups/chat-completion.ts new file mode 100644 index 0000000000..dc1ebdde43 --- /dev/null +++ b/packages/protocol/src/groups/chat-completion.ts @@ -0,0 +1,200 @@ +/** + * `POST /v1/chat/completions` — the OpenAI-compatible inference route. + * + * This is the one surface every Redrob product is meant to reach the engine + * through: it carries the engine's own Console credential, so a product never + * holds a key of its own, and it resolves the same provider registry the agent + * does, so a locally-served model is reachable through the same call. + * + * It is NOT the agent. A chat completion is stateless and a session is not, so + * this route makes one model call and returns; mapping it onto the session API + * is what produced the broken half-implementations this replaces (see + * docs/LOCAL-ENGINE-API.md). + * + * WHO EXECUTES TOOLS IS DECIDED BY THE REQUEST, and that is the whole contract: + * + * - `tools` present -> the CALLER owns them. Nothing is executed here; the + * turn ends with `finish_reason: "tool_calls"` and the caller sends the + * results back as `role: "tool"` messages. Standard OpenAI semantics. + * - `tools` absent -> a plain completion. + * + * Hosts whose tools mutate something they hold — an open document, a live + * browser tab — cannot hand execution to the engine, so caller ownership is a + * requirement for them rather than a preference. + * + * The path is versioned, not the engine: fields may be ADDED here, but removing + * one or changing its meaning requires /v2. Products negotiate with + * /api/capabilities instead of pinning an engine version. + */ +import { Schema } from "effect" +import { HttpApiEndpoint, HttpApiGroup, OpenApi } from "effect/unstable/httpapi" + +/** OpenAI puts the function under a `function` key and tags the wrapper. */ +const FunctionTool = Schema.Struct({ + type: Schema.Literal("function"), + function: Schema.Struct({ + name: Schema.String, + description: Schema.optional(Schema.String), + /** Raw JSON Schema, passed to the provider untouched. */ + parameters: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)), + }), +}) + +const ToolCall = Schema.Struct({ + id: Schema.String, + type: Schema.Literal("function"), + function: Schema.Struct({ + name: Schema.String, + /** A JSON *string*, which is what OpenAI clients parse. */ + arguments: Schema.String, + }), +}) + +/** + * Message content is a string in the common case and a part array when a client + * sends images. `Schema.Unknown` for the array case keeps this route from + * becoming a second, divergent definition of multimodal content — the parts are + * handed to the provider layer, which owns that shape. + */ +const MessageContent = Schema.Union([ + Schema.String, + Schema.Null, + Schema.Array(Schema.Record(Schema.String, Schema.Unknown)), +]) + +const Message = Schema.Struct({ + role: Schema.Literals(["system", "developer", "user", "assistant", "tool"]), + content: Schema.optional(MessageContent), + /** Present on an assistant message that is replaying a previous tool turn. */ + tool_calls: Schema.optional(Schema.Array(ToolCall)), + /** Required on a `tool` message: which call this is the result of. */ + tool_call_id: Schema.optional(Schema.String), + name: Schema.optional(Schema.String), +}) + +const ToolChoice = Schema.Union([ + Schema.Literals(["auto", "none", "required"]), + Schema.Struct({ + type: Schema.Literal("function"), + function: Schema.Struct({ name: Schema.String }), + }), +]) + +export const ChatCompletionRequest = Schema.Struct({ + /** `redrob/auto` routes through the Console; a local provider id stays local. */ + model: Schema.String, + messages: Schema.Array(Message), + tools: Schema.optional(Schema.Array(FunctionTool)), + tool_choice: Schema.optional(ToolChoice), + /** Default false. True returns text/event-stream. */ + stream: Schema.optional(Schema.Boolean), + max_tokens: Schema.optional(Schema.Int), + /** OpenAI's newer name for the same cap; `max_tokens` wins if both appear. */ + max_completion_tokens: Schema.optional(Schema.Int), + temperature: Schema.optional(Schema.Number), + top_p: Schema.optional(Schema.Number), + stop: Schema.optional(Schema.Union([Schema.String, Schema.Array(Schema.String)])), + seed: Schema.optional(Schema.Int), + frequency_penalty: Schema.optional(Schema.Number), + presence_penalty: Schema.optional(Schema.Number), + /** Passed through where the provider accepts it. */ + reasoning_effort: Schema.optional(Schema.String), + /** Accepted and ignored: a caller may send it, and refusing would be rude. */ + user: Schema.optional(Schema.String), +}).annotate({ identifier: "ChatCompletionRequest" }) +export type ChatCompletionRequest = typeof ChatCompletionRequest.Type + +const Usage = Schema.Struct({ + prompt_tokens: Schema.Int, + completion_tokens: Schema.Int, + total_tokens: Schema.Int, +}) + +const Choice = Schema.Struct({ + index: Schema.Int, + message: Schema.Struct({ + role: Schema.Literal("assistant"), + content: Schema.NullOr(Schema.String), + tool_calls: Schema.optional(Schema.Array(ToolCall)), + /** Present when the model emitted reasoning; not an OpenAI field. */ + reasoning_content: Schema.optional(Schema.String), + }), + finish_reason: Schema.NullOr(Schema.String), +}) + +export const ChatCompletionResponse = Schema.Struct({ + id: Schema.String, + object: Schema.Literal("chat.completion"), + created: Schema.Int, + model: Schema.String, + choices: Schema.Array(Choice), + usage: Schema.optional(Usage), +}).annotate({ identifier: "ChatCompletionResponse" }) +export type ChatCompletionResponse = typeof ChatCompletionResponse.Type + +/** + * The OpenAI error envelope. + * + * NOT declared on the endpoint, and that is deliberate rather than an omission. + * Two independent reasons: + * + * 1. The handler cannot use declared error schemas. It is `handleRaw`, because + * one request answers with JSON and another with `text/event-stream`, so it + * writes every response itself — including failures, through `errorResponse` + * in `chat-completion.ts`. Declared error classes were never on the path. + * 2. `httpapi-codegen` requires each declared endpoint error to carry a `_tag` + * or `name` STRING LITERAL to discriminate on (`declaredErrorFields`), and + * four statuses sharing one envelope have nothing to discriminate by. + * Adding `_tag` to satisfy it would make the generated spec and SDK claim a + * field the wire does not carry, since OpenAI's envelope has no such key — + * a spec that lies is worse than a spec that is silent. + * + * So the statuses are documented in the endpoint description and specified in + * `docs/LOCAL-ENGINE-API.md`, and this type exists to keep the shape in one place + * for the handler to build against. + * + * `code` is the part clients must branch on. `engine_not_authenticated` means the + * ENGINE has no credential, and it is the only condition under which a client + * should offer a sign-in action — the message text is localized downstream and is + * not a contract. Branching on text instead is how a network timeout reached a + * user as "please log in". + */ +export const ChatCompletionError = Schema.Struct({ + error: Schema.Struct({ + message: Schema.String, + type: Schema.String, + code: Schema.NullOr(Schema.String), + param: Schema.optional(Schema.NullOr(Schema.String)), + }), +}).annotate({ identifier: "ChatCompletionError" }) +export type ChatCompletionError = typeof ChatCompletionError.Type + +export const ChatCompletionGroup = HttpApiGroup.make("server.chat") + .add( + HttpApiEndpoint.post("chat.completions", "/v1/chat/completions", { + payload: ChatCompletionRequest, + success: ChatCompletionResponse, + }).annotateMerge( + OpenApi.annotations({ + identifier: "v1.chat.completions", + summary: "Create a chat completion", + description: + "OpenAI-compatible inference against the engine's own credential and provider registry. " + + "Declaring `tools` makes the caller responsible for executing them: the turn ends with " + + "finish_reason tool_calls and the results come back as role:tool messages. " + + "`stream: true` returns text/event-stream instead of this JSON body. " + + "Failures use OpenAI's error envelope -- {error:{message,type,code,param}} -- with " + + "400 invalid_request_error, 401 authentication_error (code engine_not_authenticated, the " + + "only status a client should offer a sign-in action for), 429 rate_limit_error or " + + "insufficient_quota, and 502 api_error. They are written by the handler rather than " + + "declared as endpoint errors, because this is a raw handler and the envelope carries no " + + "discriminator field for codegen to branch on.", + }), + ), + ) + .annotateMerge( + OpenApi.annotations({ + title: "chat completions", + description: "OpenAI-compatible inference route. The shared engine surface for Redrob products.", + }), + ) diff --git a/packages/redrob/src/server/routes/instance/httpapi/groups/experimental.ts b/packages/redrob/src/server/routes/instance/httpapi/groups/experimental.ts index 6101ac16cc..368755fada 100644 --- a/packages/redrob/src/server/routes/instance/httpapi/groups/experimental.ts +++ b/packages/redrob/src/server/routes/instance/httpapi/groups/experimental.ts @@ -25,8 +25,24 @@ const ConsoleStateResponse = Schema.Struct({ switchableOrgCount: NonNegativeInt, }).annotate({ identifier: "ConsoleState" }) +/** + * What this engine's `/v1/chat/completions` supports. + * + * Declared as a struct rather than a version number so a caller negotiates a + * capability instead of pinning an engine build: a downstream app that needs + * caller-owned tools checks `callerTools`, and an older engine that lacks the whole + * route simply omits `chatCompletions`. `version` is the contract's revision, not + * the engine's. + */ +const ChatCompletionsCapability = Schema.Struct({ + version: NonNegativeInt, + callerTools: Schema.Boolean, + stream: Schema.Boolean, +}).annotate({ identifier: "ChatCompletionsCapability" }) + const CapabilitiesResponse = Schema.Struct({ backgroundSubagents: Schema.Boolean, + chatCompletions: ChatCompletionsCapability, }).annotate({ identifier: "ExperimentalCapabilities" }) const ConsoleOrgOption = Schema.Struct({ diff --git a/packages/redrob/src/server/routes/instance/httpapi/handlers/experimental.ts b/packages/redrob/src/server/routes/instance/httpapi/handlers/experimental.ts index b218c6040d..94df91239d 100644 --- a/packages/redrob/src/server/routes/instance/httpapi/handlers/experimental.ts +++ b/packages/redrob/src/server/routes/instance/httpapi/handlers/experimental.ts @@ -37,7 +37,14 @@ export const experimentalHandlers = HttpApiBuilder.group(InstanceHttpApi, "exper const flags = yield* RuntimeFlags.Service const capabilities = Effect.fn("ExperimentalHttpApi.capabilities")(function* () { - return { backgroundSubagents: flags.experimentalBackgroundSubagents } + return { + backgroundSubagents: flags.experimentalBackgroundSubagents, + // Declared so a caller can negotiate instead of pinning an engine version. + // A downstream app reads this, sees what this engine's `/v1/chat/completions` + // supports, and degrades explicitly rather than discovering a gap by getting + // a 400 mid-feature. `version` is the contract revision, not the engine's. + chatCompletions: { version: 1, callerTools: true, stream: true }, + } }) const getConsole = Effect.fn("ExperimentalHttpApi.console")(function* () { diff --git a/packages/redrob/test/lib/llm-server.ts b/packages/redrob/test/lib/llm-server.ts index 245acc7280..6c7f7fbd20 100644 --- a/packages/redrob/test/lib/llm-server.ts +++ b/packages/redrob/test/lib/llm-server.ts @@ -775,5 +775,21 @@ export class TestLLMServer extends Context.Service [...misses]), }) }), - ).pipe(Layer.provide(HttpRouter.layer), Layer.provide(NodeHttpServer.layer(() => Http.createServer(), { port: 0 }))) + ).pipe( + // Layer.fresh, not a bare HttpRouter.layer. Layer memoization otherwise hands + // this fake the SAME HttpRouter instance the engine's own HttpApi is built on, + // so both register into one router. That was harmless only while no engine + // route shared a path with the fake: once the engine gained + // POST /v1/chat/completions -- the real OpenAI path, which this fake must also + // serve to be a credible provider double -- the second registration threw + // "Method 'POST' already declared for route '/v1/chat/completions'" and took + // down every test that stands up both. + // + // The fake's path is deliberately NOT renamed to dodge the clash: it exists to + // look like a real OpenAI-compatible endpoint, and a fake at a made-up path + // would stop exercising the thing under test. Isolating the router is the fix; + // the collision was a wiring accident, not a naming one. + Layer.provide(Layer.fresh(HttpRouter.layer)), + Layer.provide(NodeHttpServer.layer(() => Http.createServer(), { port: 0 })), + ) } diff --git a/packages/redrob/test/server/chat-completion-wire.test.ts b/packages/redrob/test/server/chat-completion-wire.test.ts new file mode 100644 index 0000000000..b2dde1ffd6 --- /dev/null +++ b/packages/redrob/test/server/chat-completion-wire.test.ts @@ -0,0 +1,242 @@ +/** + * Contract: the OpenAI wire format converts to this engine's LLM schema without + * inventing, dropping, or silently reinterpreting anything a caller sent. + * + * These are the conversions that carry the real risk on POST /v1/chat/completions, + * because the format is fixed by clients that already exist and a mistranslation + * is invisible: the request succeeds and the model is simply given something else + * than the caller wrote. + */ +import { describe, expect, it } from "bun:test" +import type { ChatCompletionRequest } from "@redrob-code/protocol/groups/chat-completion" +import { + ConversionError, + toFinishReason, + toGeneration, + toLLMMessages, + toLLMTools, + toWireToolCalls, +} from "@redrob-code/server/handlers/chat-completion-wire" + +const base = { model: "redrob/auto" } as const + +function request(fields: Partial): ChatCompletionRequest { + return { ...base, messages: [], ...fields } as ChatCompletionRequest +} + +describe("messages", () => { + it("carries a plain exchange through in order", () => { + const messages = toLLMMessages( + request({ + messages: [ + { role: "system", content: "be brief" }, + { role: "user", content: "hello" }, + { role: "assistant", content: "hi" }, + ], + }), + ) + expect(messages.map((m) => m.role)).toEqual(["system", "user", "assistant"]) + expect(messages[1]!.content).toEqual([{ type: "text", text: "hello" }]) + }) + + it("treats developer as system, because OpenAI renamed the role and kept both", () => { + const messages = toLLMMessages(request({ messages: [{ role: "developer", content: "rules" }] })) + expect(messages).toHaveLength(1) + expect(messages[0]!.role).toBe("system") + }) + + it("drops an empty message instead of sending a blank turn", () => { + expect(toLLMMessages(request({ messages: [{ role: "system", content: "" }] }))).toHaveLength(0) + expect(toLLMMessages(request({ messages: [{ role: "assistant", content: null }] }))).toHaveLength(0) + }) + + it("parses tool_call arguments, which arrive as a JSON string and must not stay one", () => { + const messages = toLLMMessages( + request({ + messages: [ + { + role: "assistant", + tool_calls: [ + { id: "call_1", type: "function", function: { name: "read", arguments: '{"path":"a.txt"}' } }, + ], + }, + ], + }), + ) + const part = messages[0]!.content[0] as { type: string; id: string; name: string; input: unknown } + expect(part.type).toBe("tool-call") + expect(part.id).toBe("call_1") + expect(part.input).toEqual({ path: "a.txt" }) + }) + + it("keeps text and tool calls together on one assistant turn", () => { + const messages = toLLMMessages( + request({ + messages: [ + { + role: "assistant", + content: "let me look", + tool_calls: [{ id: "c1", type: "function", function: { name: "read", arguments: "{}" } }], + }, + ], + }), + ) + expect(messages[0]!.content.map((part) => part.type)).toEqual(["text", "tool-call"]) + }) + + it("refuses malformed tool_call arguments rather than forwarding an empty object", () => { + // Silently sending {} would make the model answer a question nobody asked. + expect(() => + toLLMMessages( + request({ + messages: [ + { + role: "assistant", + tool_calls: [{ id: "c1", type: "function", function: { name: "read", arguments: "{not json" } }], + }, + ], + }), + ), + ).toThrow(ConversionError) + }) + + it("refuses a tool message with no tool_call_id", () => { + // The provider pairs the result to the call by this id; without it the model + // sees an orphan result and there is nothing sensible to guess. + expect(() => toLLMMessages(request({ messages: [{ role: "tool", content: "done" }] }))).toThrow( + /requires tool_call_id/, + ) + }) + + it("carries a tool result under the id it answers", () => { + const messages = toLLMMessages( + request({ messages: [{ role: "tool", tool_call_id: "c1", name: "read", content: "file body" }] }), + ) + const part = messages[0]!.content[0] as { type: string; id: string } + expect(messages[0]!.role).toBe("tool") + expect(part.type).toBe("tool-result") + expect(part.id).toBe("c1") + }) + + it("accepts an array content part of type text", () => { + const messages = toLLMMessages( + request({ messages: [{ role: "user", content: [{ type: "text", text: "from a part" }] }] }), + ) + expect(messages[0]!.content).toEqual([{ type: "text", text: "from a part" }]) + }) + + it("refuses a content part it cannot carry instead of dropping it", () => { + // An image silently discarded looks to the user like the model ignored it. + expect(() => + toLLMMessages( + request({ + messages: [{ role: "user", content: [{ type: "image_url", image_url: { url: "data:..." } }] }], + }), + ), + ).toThrow(/not supported/) + }) +}) + +describe("tools", () => { + it("is undefined when the caller declared none, which is what selects engine-owned tools", () => { + expect(toLLMTools(request({}))).toBeUndefined() + expect(toLLMTools(request({ tools: [] }))).toBeUndefined() + }) + + it("converts a declaration and passes parameters through untouched", () => { + const parameters = { type: "object", properties: { path: { type: "string" } }, required: ["path"] } + const tools = toLLMTools( + request({ tools: [{ type: "function", function: { name: "read", description: "read a file", parameters } }] }), + ) + expect(tools!.definitions).toHaveLength(1) + expect(tools!.definitions[0]!.name).toBe("read") + expect(tools!.definitions[0]!.inputSchema).toEqual(parameters) + }) + + it("gives a parameterless tool an empty object schema rather than dropping it", () => { + const tools = toLLMTools(request({ tools: [{ type: "function", function: { name: "now" } }] })) + expect(tools!.definitions[0]!.inputSchema).toEqual({ type: "object", properties: {} }) + // An invented description would put words in the prompt the caller never wrote. + expect(tools!.definitions[0]!.description).toBe("") + }) + + it("carries tool_choice as the engine's shape, never as a bare string", () => { + // A bare string is ambiguous: the engine's normalizer reads + // "auto"/"none"/"required" as modes and anything else as a tool name, so a + // tool actually named `auto` would silently become a mode. + expect( + toLLMTools(request({ tools: [{ type: "function", function: { name: "a" } }], tool_choice: "none" }))!.choice, + ).toEqual({ type: "none" }) + expect( + toLLMTools( + request({ + tools: [{ type: "function", function: { name: "a" } }], + tool_choice: { type: "function", function: { name: "a" } }, + }), + )!.choice, + ).toEqual({ type: "tool", name: "a" }) + }) + + it("names a tool called auto as a tool, not as the auto mode", () => { + expect( + toLLMTools( + request({ + tools: [{ type: "function", function: { name: "auto" } }], + tool_choice: { type: "function", function: { name: "auto" } }, + }), + )!.choice, + ).toEqual({ type: "tool", name: "auto" }) + }) +}) + +describe("finish reason", () => { + it("maps the engine's vocabulary onto OpenAI's", () => { + expect(toFinishReason("stop")).toBe("stop") + expect(toFinishReason("length")).toBe("length") + expect(toFinishReason("tool-calls")).toBe("tool_calls") + expect(toFinishReason("content-filter")).toBe("content_filter") + expect(toFinishReason(undefined)).toBeNull() + }) + + it("does not report a failed turn as a clean stop", () => { + // "error" and "unknown" have no OpenAI equivalent. Reporting "stop" would + // tell the caller the turn ended normally when it did not. + expect(toFinishReason("error")).not.toBe("stop") + expect(toFinishReason("unknown")).not.toBe("stop") + }) +}) + +describe("tool calls out", () => { + it("re-stringifies input, because the engine parses it and clients expect a string", () => { + const wire = toWireToolCalls([{ id: "c1", name: "read", input: { path: "a.txt" } }]) + expect(wire[0]!.function.arguments).toBe('{"path":"a.txt"}') + expect(JSON.parse(wire[0]!.function.arguments)).toEqual({ path: "a.txt" }) + expect(wire[0]!.type).toBe("function") + }) + + it("emits {} for a call with no input, never undefined", () => { + // `arguments: undefined` breaks a client that calls JSON.parse on it. + expect(toWireToolCalls([{ id: "c1", name: "now", input: undefined }])[0]!.function.arguments).toBe("{}") + }) +}) + +describe("generation options", () => { + it("is undefined when the caller set none", () => { + expect(toGeneration(request({}))).toBeUndefined() + }) + + it("renames the wire fields to the engine's", () => { + const generation = toGeneration(request({ max_tokens: 256, temperature: 0.2, top_p: 0.9 })) + expect(generation).toEqual({ maxTokens: 256, temperature: 0.2, topP: 0.9 }) + }) + + it("lets max_tokens win over max_completion_tokens when both are sent", () => { + expect(toGeneration(request({ max_tokens: 100, max_completion_tokens: 900 }))!["maxTokens"]).toBe(100) + expect(toGeneration(request({ max_completion_tokens: 900 }))!["maxTokens"]).toBe(900) + }) + + it("normalizes a single stop string to a list", () => { + expect(toGeneration(request({ stop: "END" }))!["stop"]).toEqual(["END"]) + expect(toGeneration(request({ stop: ["A", "B"] }))!["stop"]).toEqual(["A", "B"]) + }) +}) diff --git a/packages/redrob/test/server/httpapi-chat-completion.test.ts b/packages/redrob/test/server/httpapi-chat-completion.test.ts new file mode 100644 index 0000000000..25dac31b9f --- /dev/null +++ b/packages/redrob/test/server/httpapi-chat-completion.test.ts @@ -0,0 +1,147 @@ +/** + * Route tests for `POST /v1/chat/completions`, driven in-process against the real + * app through `httpapi-layer`. + * + * Scope note. These tests cover what can be asserted without a provider + * credential: that the route is served on the v2 surface at all, that a body + * decodes against `ChatCompletionRequest`, that every failure comes back in the + * OpenAI error envelope rather than Effect's default shape, and that the v1 + * contract's field names are actually present. A completion that reaches a model + * needs a credential this suite has none of, so the streaming happy path is + * covered by `chat-completion-wire.test.ts` at the conversion layer instead of + * being faked here. + * + * The envelope assertions are the point. A caller written against OpenAI's API + * reads `error.message` and `error.type`; if this route ever returns Effect's + * `{ name, data }` shape instead, every such caller silently sees an + * error-less response and reports a truncated or empty answer rather than a + * failure. + */ +import { describe, expect } from "bun:test" +import { LayerNode } from "@redrob-code/core/effect/layer-node" +import { Effect, Layer } from "effect" +import { HttpClientResponse } from "effect/unstable/http" +import { Session } from "@/session/session" +import { Database } from "@redrob-code/core/database/database" +import { TestInstance } from "../fixture/fixture" +import { testEffect } from "../lib/effect" +import { httpApiLayer, requestInDirectory } from "./httpapi-layer" + +const it = testEffect(Layer.mergeAll(LayerNode.compile(LayerNode.group([Session.node, Database.node])), httpApiLayer)) + +const PATH = "/v1/chat/completions" + +function post(directory: string, body: unknown) { + return requestInDirectory(PATH, directory, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(body), + }) +} + +function json(response: HttpClientResponse.HttpClientResponse) { + return response.json.pipe(Effect.map((value) => value as T)) +} + +type ErrorEnvelope = { + error?: { message?: unknown; type?: unknown; code?: unknown; param?: unknown } +} + +describe("POST /v1/chat/completions", () => { + it.instance("is served on the v2 surface", () => + Effect.gen(function* () { + const tmp = yield* TestInstance + const response = yield* post(tmp.directory, { + model: "no-such-provider/no-such-model", + messages: [{ role: "user", content: "ping" }], + }) + + // Any status other than 404 proves the route exists and is reached. The + // specific failure is asserted below; this case exists so a registration + // regression (the group dropped from protocol/src/api.ts) fails loudly and + // separately from a behaviour change. + expect(response.status).not.toBe(404) + }), + ) + + it.instance("reports an unknown model in the OpenAI error envelope", () => + Effect.gen(function* () { + const tmp = yield* TestInstance + const response = yield* post(tmp.directory, { + model: "no-such-provider/no-such-model", + messages: [{ role: "user", content: "ping" }], + }) + + expect(response.status).toBe(400) + const body = yield* json(response) + expect(typeof body.error).toBe("object") + expect(typeof body.error?.message).toBe("string") + expect(typeof body.error?.type).toBe("string") + // Effect's default failure shape must NOT leak through. + expect(body).not.toHaveProperty("name") + expect(body).not.toHaveProperty("data") + }), + ) + + it.instance("rejects a body that is not a chat-completions request", () => + Effect.gen(function* () { + const tmp = yield* TestInstance + const response = yield* post(tmp.directory, { model: "x" }) + + expect(response.status).toBe(400) + const body = yield* json(response) + expect(typeof body.error?.message).toBe("string") + }), + ) + + it.instance("rejects an empty message list rather than calling a provider", () => + Effect.gen(function* () { + const tmp = yield* TestInstance + const response = yield* post(tmp.directory, { model: "x/y", messages: [] }) + + expect(response.status).toBe(400) + }), + ) + + // v1 CONTRACT. This is the test that should fail when someone removes or renames + // a field downstream callers read. It asserts the request shape is still + // ACCEPTED (decode succeeds, so the failure is about the model rather than the + // body) for every field the v1 contract promises, including the tools path that + // `docs/LOCAL-ENGINE-API.md` describes as caller-owned. + it.instance("still accepts every field the v1 contract promises", () => + Effect.gen(function* () { + const tmp = yield* TestInstance + const response = yield* post(tmp.directory, { + model: "no-such-provider/no-such-model", + messages: [ + { role: "system", content: "be brief" }, + { role: "user", content: "ping" }, + { role: "assistant", content: "pong" }, + ], + stream: true, + temperature: 0.2, + max_tokens: 64, + tools: [ + { + type: "function", + function: { + name: "get_weather", + description: "Look up the weather", + parameters: { type: "object", properties: { city: { type: "string" } } }, + }, + }, + ], + tool_choice: "auto", + }) + + // 400 for the unknown model, NOT a decode failure: if a promised field had + // been dropped from the schema this would still be 400 but with a decode + // message, so the body is checked for the model rather than the shape. + expect(response.status).toBe(400) + const body = yield* json(response) + const message = String(body.error?.message ?? "") + expect(message.toLowerCase()).not.toContain("expected") + expect(message.toLowerCase()).not.toContain("is missing") + }), + ) +}) diff --git a/packages/redrob/test/server/httpapi-exercise/index.ts b/packages/redrob/test/server/httpapi-exercise/index.ts index 551550e90c..e8cc795e47 100644 --- a/packages/redrob/test/server/httpapi-exercise/index.ts +++ b/packages/redrob/test/server/httpapi-exercise/index.ts @@ -97,9 +97,7 @@ const scenarios: Scenario[] = [ Effect.gen(function* () { object(body) check(body.username === "httpapi-global", "global config update should return patched config") - const text = yield* Effect.promise(() => - Bun.file(path.join(exerciseConfigDirectory, "redrob.jsonc")).text(), - ) + const text = yield* Effect.promise(() => Bun.file(path.join(exerciseConfigDirectory, "redrob.jsonc")).text()) check(text.includes('"username": "httpapi-global"'), "global config update should write isolated config file") }), "status", @@ -581,7 +579,32 @@ const scenarios: Scenario[] = [ http.protected.get("/experimental/capabilities", "experimental.capabilities.get").json(200, (body) => { check(typeof body === "object" && body !== null, "capabilities should be an object") check("backgroundSubagents" in body, "capabilities should report background subagents") + check("chatCompletions" in body, "capabilities should report the chat-completions contract") }), + // The OpenAI-compatible route. Exercised on its 400 path deliberately: a real + // completion needs a provider credential this harness has none of, so asserting + // 200 would make the gate depend on the machine it runs on. An unknown model id + // is refused before any provider is resolved, which is what makes this + // deterministic -- and it still covers the parts most likely to break, namely + // that the body decodes against ChatCompletionRequest and that the failure comes + // back in the OpenAI error envelope rather than Effect's default shape. + http.protected + .post("/v1/chat/completions", "chat.completions") + .at((ctx) => ({ + path: "/v1/chat/completions", + headers: ctx.headers(), + body: { + model: "redrob-httpapi-exercise/no-such-model", + messages: [{ role: "user", content: "ping" }], + }, + })) + .json(400, (body) => { + check(typeof body === "object" && body !== null, "chat completions error should be an object") + const error = (body as { error?: { message?: unknown; type?: unknown } }).error + check(typeof error === "object" && error !== null, "error should be nested under `error`, as OpenAI does") + check(typeof error?.message === "string", "error should carry a message") + check(typeof error?.type === "string", "error should carry a type") + }), http.protected .post("/experimental/session/{sessionID}/background", "experimental.session.background") .mutating() diff --git a/packages/server/src/handlers.ts b/packages/server/src/handlers.ts index 2f7bb9e92d..ab62df08e9 100644 --- a/packages/server/src/handlers.ts +++ b/packages/server/src/handlers.ts @@ -18,6 +18,7 @@ import { LocationHandler } from "./handlers/location" import { IntegrationHandler } from "./handlers/integration" import { CredentialHandler } from "./handlers/credential" import { ProjectCopyHandler } from "./handlers/project-copy" +import { ChatCompletionHandler } from "./handlers/chat-completion" export const handlers = Layer.mergeAll( HealthHandler, @@ -39,4 +40,5 @@ export const handlers = Layer.mergeAll( QuestionHandler, ReferenceHandler, ProjectCopyHandler, + ChatCompletionHandler, ) diff --git a/packages/server/src/handlers/chat-completion-wire.ts b/packages/server/src/handlers/chat-completion-wire.ts new file mode 100644 index 0000000000..40dd82fdb6 --- /dev/null +++ b/packages/server/src/handlers/chat-completion-wire.ts @@ -0,0 +1,231 @@ +/** + * Translation between the OpenAI chat-completions wire format and this engine's + * LLM schema. Pure: no Effect, no services, no I/O — so the parts most likely to + * be wrong are the parts that can be tested directly. + * + * Direction matters here. Requests come in from a caller we do not control, so + * every field is validated or ignored rather than trusted. Responses go out to + * clients that already exist (redrob-browser builds and parses this format + * today), so the output shape is fixed by them, not by convenience. + */ +import { Message, ToolCallPart, ToolDefinition, ToolResultPart } from "@redrob-code/llm" +import type { ChatCompletionRequest } from "@redrob-code/protocol/groups/chat-completion" + +/** What the caller declared, after validation. */ +export interface ToolsIn { + readonly definitions: ReadonlyArray + /** + * The engine's own shape, not the wire's. A bare string would be ambiguous: + * its normalizer reads "auto"/"none"/"required" as modes and anything else as + * a tool name, so a tool actually named `auto` would silently become a mode. + */ + readonly choice: { readonly type: "auto" | "none" | "required" | "tool"; readonly name?: string } | undefined +} + +export class ConversionError extends Error { + constructor( + message: string, + readonly param: string | undefined, + ) { + super(message) + this.name = "ConversionError" + } +} + +/** + * OpenAI allows string content or an array of parts. The array form is passed + * through as text parts only: media parts need a mediaType this route has no + * validated source for, and silently dropping an image would be worse than + * saying so. + */ +function contentParts( + content: ChatCompletionRequest["messages"][number]["content"], + where: string, +): ReadonlyArray> { + if (content === undefined || content === null) return [] + if (typeof content === "string") return content.length > 0 ? [Message.text(content)] : [] + const parts: Array> = [] + for (const part of content) { + const type = part["type"] + if (type === "text" && typeof part["text"] === "string") { + parts.push(Message.text(part["text"] as string)) + continue + } + throw new ConversionError( + `${where}: content part of type ${String(type)} is not supported on this route`, + "messages", + ) + } + return parts +} + +/** + * `system` and `developer` both become a system message: OpenAI renamed the role + * and kept accepting the old one, so a caller may send either. + * + * A `tool` message must name the call it answers. Without `tool_call_id` the + * provider cannot pair it with the assistant turn that asked, and the model sees + * an orphan result — so this is rejected rather than guessed at. + */ +export function toLLMMessages(request: ChatCompletionRequest): ReadonlyArray { + const out: Array = [] + request.messages.forEach((message, index) => { + const where = `messages[${index}]` + switch (message.role) { + case "system": + case "developer": { + const text = typeof message.content === "string" ? message.content : "" + if (text.length > 0) out.push(Message.system(text)) + return + } + case "user": { + out.push(Message.make({ role: "user", content: [...contentParts(message.content, where)] })) + return + } + case "assistant": { + // An assistant turn being replayed can carry text, tool calls, or both. + const parts: Array | ToolCallPart> = [ + ...contentParts(message.content, where), + ] + for (const call of message.tool_calls ?? []) { + parts.push( + ToolCallPart.make({ + id: call.id, + name: call.function.name, + // The wire carries a JSON string; the engine wants the parsed value. + // A malformed string is the caller's bug and must not be forwarded + // as a silently empty argument object. + input: parseArguments(call.function.arguments, `${where}.tool_calls`), + }), + ) + } + if (parts.length > 0) out.push(Message.make({ role: "assistant", content: parts })) + return + } + case "tool": { + if (message.tool_call_id === undefined || message.tool_call_id.length === 0) { + throw new ConversionError(`${where}: a tool message requires tool_call_id`, "messages") + } + out.push( + Message.tool( + ToolResultPart.make({ + id: message.tool_call_id, + name: message.name ?? "", + result: typeof message.content === "string" ? message.content : "", + }), + ), + ) + return + } + } + }) + return out +} + +function parseArguments(raw: string, where: string): unknown { + if (raw.length === 0) return {} + try { + return JSON.parse(raw) + } catch { + throw new ConversionError(`${where}: arguments is not valid JSON`, "messages") + } +} + +/** + * A declared tool with no `parameters` still needs a schema the provider will + * accept, so it becomes an empty object schema rather than being dropped. + * + * `description` is required by ToolDefinition and optional on the wire; an empty + * string is the honest default — inventing one would put words in the model's + * prompt that the caller never wrote. + */ +export function toLLMTools(request: ChatCompletionRequest): ToolsIn | undefined { + if (request.tools === undefined || request.tools.length === 0) return undefined + const definitions = request.tools.map((tool, index) => { + if (tool.function.name.length === 0) { + throw new ConversionError(`tools[${index}]: function.name is required`, "tools") + } + return new ToolDefinition({ + name: tool.function.name, + description: tool.function.description ?? "", + inputSchema: tool.function.parameters ?? { type: "object", properties: {} }, + }) + }) + return { definitions, choice: toToolChoice(request.tool_choice) } +} + +function toToolChoice(choice: ChatCompletionRequest["tool_choice"]): ToolsIn["choice"] { + if (choice === undefined) return undefined + if (typeof choice === "string") return { type: choice } + return { type: "tool", name: choice.function.name } +} + +/** OpenAI's finish_reason vocabulary. The engine's is close but not identical. */ +export function toFinishReason(reason: string | undefined): string | null { + switch (reason) { + case "stop": + return "stop" + case "length": + return "length" + case "tool-calls": + return "tool_calls" + case "content-filter": + return "content_filter" + case undefined: + return null + default: + // "error" and "unknown" have no OpenAI equivalent. Reporting them as + // "stop" would tell the caller the turn ended cleanly when it did not. + return reason + } +} + +/** The engine parses tool-call input; OpenAI clients expect a JSON string. */ +export function toWireToolCalls( + calls: ReadonlyArray<{ readonly id: string; readonly name: string; readonly input: unknown }>, +): ReadonlyArray<{ + readonly id: string + readonly type: "function" + readonly function: { readonly name: string; readonly arguments: string } +}> { + return calls.map((call) => ({ + id: call.id, + type: "function" as const, + function: { name: call.name, arguments: JSON.stringify(call.input ?? {}) }, + })) +} + +/** `max_tokens` wins over `max_completion_tokens` when a caller sends both. */ +export function toGeneration(request: ChatCompletionRequest): GenerationInput | undefined { + const generation: { + maxTokens?: number + temperature?: number + topP?: number + seed?: number + frequencyPenalty?: number + presencePenalty?: number + stop?: ReadonlyArray + } = {} + const maxTokens = request.max_tokens ?? request.max_completion_tokens + if (maxTokens !== undefined) generation.maxTokens = maxTokens + if (request.temperature !== undefined) generation.temperature = request.temperature + if (request.top_p !== undefined) generation.topP = request.top_p + if (request.seed !== undefined) generation.seed = request.seed + if (request.frequency_penalty !== undefined) generation.frequencyPenalty = request.frequency_penalty + if (request.presence_penalty !== undefined) generation.presencePenalty = request.presence_penalty + if (request.stop !== undefined) { + generation.stop = typeof request.stop === "string" ? [request.stop] : [...request.stop] + } + return Object.keys(generation).length > 0 ? generation : undefined +} + +/** The subset of the engine's generation options this route can be asked for. */ +export interface GenerationInput { + readonly maxTokens?: number + readonly temperature?: number + readonly topP?: number + readonly seed?: number + readonly frequencyPenalty?: number + readonly presencePenalty?: number + readonly stop?: ReadonlyArray +} diff --git a/packages/server/src/handlers/chat-completion.ts b/packages/server/src/handlers/chat-completion.ts new file mode 100644 index 0000000000..da056c4da0 --- /dev/null +++ b/packages/server/src/handlers/chat-completion.ts @@ -0,0 +1,317 @@ +/** + * `POST /v1/chat/completions` — see docs/LOCAL-ENGINE-API.md and + * protocol/groups/chat-completion.ts for the contract. + * + * This handler deliberately does NOT touch Session, Agent or the tool runtime. + * It resolves a provider the way the agent does, makes one model call, and + * translates. Everything stateful belongs to the session API. + * + * `handleRaw` rather than `handle`, because the reply is JSON or + * text/event-stream depending on `stream`, and only the raw form can return an + * HttpServerResponse of its own choosing. + */ +import { Catalog } from "@redrob-code/core/catalog" +import { Integration } from "@redrob-code/core/integration" +import { ModelV2 } from "@redrob-code/core/model" +import { ProviderV2 } from "@redrob-code/core/provider" +import { fromCatalogModel } from "@redrob-code/core/session/runner/model" +import { LLM, LLMError } from "@redrob-code/llm" +import type { LLMEvent, LLMResponse } from "@redrob-code/llm" +import type { Credential } from "@redrob-code/schema/credential" +import { ChatCompletionRequest } from "@redrob-code/protocol/groups/chat-completion" +import { Effect, Stream } from "effect" +import { HttpServerRequest, HttpServerResponse } from "effect/unstable/http" +import { HttpApiBuilder } from "effect/unstable/httpapi" +import * as Sse from "effect/unstable/encoding/Sse" +import { Api } from "../api" +import { + ConversionError, + toFinishReason, + toGeneration, + toLLMMessages, + toLLMTools, + toWireToolCalls, +} from "./chat-completion-wire" + +/** OpenAI ids are `chatcmpl-`; clients log and correlate on them. */ +function completionID(): string { + return `chatcmpl-${crypto.randomUUID().replaceAll("-", "")}` +} + +function seconds(): number { + return Math.floor(Date.now() / 1000) +} + +interface WireError { + readonly status: 400 | 401 | 429 | 502 + readonly message: string + readonly type: string + readonly code: string | null +} + +/** + * Maps an engine failure onto the OpenAI error envelope. + * + * `engine_not_authenticated` is the load-bearing value: it is the only code a + * client may turn into a sign-in prompt, and it covers both a missing credential + * and one the provider rejected — from the caller's side those are the same + * problem, and neither is fixed by retrying. + * + * Note what is NOT mapped to auth. The executor classifies a content-policy body + * BEFORE it looks at the status, so an upstream 401 whose body mentions safety + * arrives as ContentPolicy. That ordering is upstream's and this reads whatever + * it produced rather than second-guessing it — reclassifying here would report a + * policy refusal as a login problem. + */ +function toWireError(error: unknown): WireError { + if (error instanceof ConversionError) { + return { status: 400, message: error.message, type: "invalid_request_error", code: null } + } + if (error instanceof LLMError) { + const reason = error.reason + switch (reason._tag) { + case "Authentication": + return { + status: 401, + message: reason.message, + type: "authentication_error", + code: "engine_not_authenticated", + } + case "InvalidRequest": + return { status: 400, message: reason.message, type: "invalid_request_error", code: null } + case "RateLimit": + return { status: 429, message: reason.message, type: "rate_limit_error", code: "rate_limit_exceeded" } + case "QuotaExceeded": + return { status: 429, message: reason.message, type: "insufficient_quota", code: "insufficient_quota" } + case "ContentPolicy": + return { status: 400, message: reason.message, type: "invalid_request_error", code: "content_policy_violation" } + default: + return { status: 502, message: reason.message, type: "api_error", code: null } + } + } + // A body that does not decode against ChatCompletionRequest is the CALLER's + // error, so it must not fall through to the 502 catch-all below. + // `schemaBodyJson` fails with `HttpServerError | Schema.SchemaError`, and both + // mean the request never reached a provider — reporting them as `api_error` told + // a caller the upstream had failed when in fact their own payload was malformed, + // which sends them debugging the wrong system. + if (isRequestDecodeError(error)) { + return { + status: 400, + message: error instanceof Error ? error.message : String(error), + type: "invalid_request_error", + code: null, + } + } + const message = error instanceof Error ? error.message : String(error) + return { status: 502, message, type: "api_error", code: null } +} + +/** + * Whether this failure happened while reading the request, before any provider was + * involved. + * + * Matched structurally rather than with `instanceof`: `Schema.SchemaError` and the + * HTTP request errors are separate hierarchies, and an `instanceof` chain over both + * silently stops matching when either is re-exported through a different module + * instance. The `_tag` values are part of those errors' public shape. + */ +function isRequestDecodeError(error: unknown): boolean { + if (typeof error !== "object" || error === null) return false + const tag = (error as { _tag?: unknown })._tag + return tag === "SchemaError" || tag === "RequestError" || tag === "HttpServerError" +} + +function errorResponse(error: unknown): Effect.Effect { + const wire = toWireError(error) + return HttpServerResponse.json( + { error: { message: wire.message, type: wire.type, code: wire.code, param: null } }, + { status: wire.status }, + ).pipe(Effect.orDie) +} + +/** + * Resolves the requested model the way the agent does: catalog entry plus the + * active integration credential. `redrob/auto` is the Console route; any other + * `provider/model` string resolves against the same registry, which is how a + * locally served model becomes reachable without a second code path. + * + * A caller-supplied base URL is deliberately NOT accepted. The config layer only + * admits a new openai-compatible provider at a local address (isLocalEndpoint); + * honouring an arbitrary URL here would bypass that check and forward the + * engine's own credential to whatever host the caller named. + */ +const resolveModel = Effect.fn("chat.resolveModel")(function* (model: string) { + const [providerPart, ...rest] = model.split("/") + const providerID = ProviderV2.ID.make(rest.length > 0 ? providerPart! : "redrob") + const modelID = ModelV2.ID.make(rest.length > 0 ? rest.join("/") : model) + + const catalog = yield* Catalog.Service + const info = yield* catalog.model.get(providerID, modelID) + if (info === undefined) { + return yield* Effect.fail(new ConversionError(`model ${model} is not available on this engine`, "model")) + } + + const integrations = yield* Integration.Service + let credential: Credential.Value | undefined + // `active` has no error channel; only `resolve` can fail, and a failure there + // means "no usable credential", which the provider layer reports better than a + // 500 from here would. + const connection = yield* integrations.connection.active(Integration.ID.make(providerID)) + if (connection !== undefined) { + credential = yield* integrations.connection.resolve(connection).pipe(Effect.catch(() => Effect.succeed(undefined))) + } + return yield* fromCatalogModel(info, credential) +}) + +const buildRequest = Effect.fn("chat.buildRequest")(function* (payload: ChatCompletionRequest) { + const model = yield* resolveModel(payload.model) + const messages = toLLMMessages(payload) + const tools = toLLMTools(payload) + const generation = toGeneration(payload) + return LLM.request({ + model, + messages, + // Absent tools is what selects engine-owned behaviour; an empty array would + // tell the provider "you may call nothing", which is a different thing. + ...(tools ? { tools: tools.definitions, ...(tools.choice ? { toolChoice: tools.choice } : {}) } : {}), + ...(generation ? { generation } : {}), + }) +}) + +function toJsonBody(id: string, model: string, response: LLMResponse): unknown { + const toolCalls = toWireToolCalls( + response.toolCalls.map((call) => ({ id: call.id, name: call.name, input: call.input })), + ) + const usage = response.usage + return { + id, + object: "chat.completion", + created: seconds(), + model, + choices: [ + { + index: 0, + message: { + role: "assistant", + content: response.text.length > 0 ? response.text : null, + ...(toolCalls.length > 0 ? { tool_calls: toolCalls } : {}), + }, + finish_reason: toFinishReason(response.finishReason), + }, + ], + ...(usage + ? { + usage: { + prompt_tokens: usage.inputTokens ?? 0, + completion_tokens: usage.outputTokens ?? 0, + total_tokens: usage.totalTokens ?? 0, + }, + } + : {}), + } +} + +/** One SSE frame carrying an OpenAI chunk. */ +function chunkFrame(id: string, model: string, delta: unknown, finishReason: string | null): Sse.Event { + return { + _tag: "Event", + event: "message", + id: undefined, + data: JSON.stringify({ + id, + object: "chat.completion.chunk", + created: seconds(), + model, + choices: [{ index: 0, delta, finish_reason: finishReason }], + }), + } +} + +function doneFrame(): Sse.Event { + return { _tag: "Event", event: "message", id: undefined, data: "[DONE]" } +} + +/** + * Translates the engine's event stream into OpenAI chunks. + * + * Two things here are not obvious. Tool-call fragments are emitted from the + * COMPLETED `tool-call` event rather than from `tool-input-delta`, because the + * engine parses arguments and the wire wants a JSON string — re-emitting + * fragments would mean re-serializing a half-parsed value. And `provider-error` + * arrives as a stream ELEMENT, not a failure, so it is matched explicitly: left + * unhandled, an upstream 500 would end the stream with a clean `[DONE]` and the + * caller would read a truncated answer as a complete one. + */ +function toChunks(id: string, model: string, events: Stream.Stream) { + return events.pipe( + Stream.flatMap((event: LLMEvent) => Stream.fromIterable(eventFrames(id, model, event))), + Stream.concat(Stream.make(doneFrame())), + ) +} + +function eventFrames(id: string, model: string, event: LLMEvent): ReadonlyArray { + switch (event.type) { + case "text-delta": + return [chunkFrame(id, model, { content: event.text }, null)] + case "reasoning-delta": + return [chunkFrame(id, model, { reasoning_content: event.text }, null)] + case "tool-call": { + const [call] = toWireToolCalls([{ id: event.id, name: event.name, input: event.input }]) + return [chunkFrame(id, model, { tool_calls: [{ index: 0, ...call }] }, null)] + } + case "finish": + return [chunkFrame(id, model, {}, toFinishReason(event.reason))] + case "provider-error": + // Surfaced as data, not as a stream failure: headers are already sent, so + // this is the only way the caller learns the turn broke. + return [ + { + _tag: "Event" as const, + event: "message", + id: undefined, + data: JSON.stringify({ + error: { message: event.message, type: "api_error", code: null, param: null }, + }), + }, + ] + default: + return [] + } +} + +export const ChatCompletionHandler = HttpApiBuilder.group(Api, "server.chat", (handlers) => + Effect.gen(function* () { + return handlers.handleRaw("chat.completions", () => + Effect.gen(function* () { + // handleRaw does not decode the payload — that is the cost of being able + // to answer with either JSON or an event stream. The declared schema is + // still the contract, so it is applied here rather than trusting the body. + const payload = yield* HttpServerRequest.schemaBodyJson(ChatCompletionRequest) + const id = completionID() + const request = yield* buildRequest(payload) + + if (payload.stream !== true) { + const response = yield* LLM.generate(request) + return yield* HttpServerResponse.json(toJsonBody(id, payload.model, response)).pipe(Effect.orDie) + } + + // Streaming: the error envelope can only be sent before the first byte, + // so a failure after headers travels as an SSE error frame (see + // eventFrames) rather than as a status code. + const frames = toChunks(id, payload.model, LLM.stream(request)).pipe( + Stream.pipeThroughChannel(Sse.encode()), + Stream.encodeText, + ) + return HttpServerResponse.stream(frames, { + contentType: "text/event-stream", + headers: { + "Cache-Control": "no-cache, no-transform", + "X-Accel-Buffering": "no", + "X-Content-Type-Options": "nosniff", + }, + }) + }).pipe(Effect.catch((error) => errorResponse(error))), + ) + }), +) From be693b7fd6b7c873a6d5f617556278e3c06cf3f6 Mon Sep 17 00:00:00 2001 From: Janghoon Lee <44862514+its-janghoon@users.noreply.github.com> Date: Wed, 23 Sep 2026 02:13:18 +0900 Subject: [PATCH 03/33] docs: correct three recommendations that research overturned (#60) Both harness documents told the next reader to do things that are wrong. Fixed with the evidence, rather than quietly reworded, so the reversals are auditable. 1. dynamicTools is no longer "the better fit" LOCAL-HARNESS-BACKENDS.md called Codex app-server's dynamicTools the better path for host-owned tools. Withdrawn. OpenAI's stability warning got BROADER in three months: the 2026-06-24 snapshot carried no production warning and scoped "experimental" to the WebSocket transport, while today's page covers the app-server command itself. app-server has no row at all in the Feature Maturity table. dynamicTools is double-experimental - gated behind capabilities.experimentalApi AND absent from the generated ThreadStartParams bindings, so the field must be hand-written against no type - and it has already changed wire shape once, leaving a LegacyDynamicToolSpec compat path where exposeToContext became the INVERTED deferLoading. There is no protocol version, no breaking-change log, and an open regression in the -c MCP path under app-server mode (openai/codex#39537). The counter-signal is recorded rather than buried: Zed's ACP adapter moved ONTO app-server and OpenAI's own IDE surfaces run on it, which is why this is "revisit in a quarter" and not "avoid". MCP cannot give us a tool living in our own process memory; when Office needs that, budget for re-porting dynamicTools at least once. 2. Driving the unmodified binary is not a safe harbour The doc said it was "the documented exception". Too confident. Anthropic began blocking third-party harnesses from Claude subscription billing on 2026-04-04 and the restriction is described as extending to all of them; OpenClaw is researching a --method cli path precisely because direct OAuth use has been blocked since then, and the same issue warns that spawning claude through the Agent SDK IS the pattern being restricted. Anthropic's June article still says subscription funds claude -p and the metering change is paused - the two together are a trajectory, not a guarantee. Consequences, both load-bearing: BYOK with the user's own key becomes the FIRST deliverable rather than the fallback, and Codex comes before Claude Code for business reasons rather than technical ones. 3. The /v1/providers routes I proposed already exist PROVIDER-AUTH.md proposed four new routes to expose the engine's login flow. They are already there as the server.integration group: GET /api/integration returns Integration.Info with a methods union (OAuthMethod | KeyMethod | EnvMethod) and a connections array, and connect/key, connect/oauth, the attempt status/complete/cancel routes and DELETE /api/credential complete it. Building /v1/providers would have been a duplicate surface over one credential store with two code paths to keep in agreement. A mapping table now records which existing route replaces each proposed one. So the engine side of BYOK is DONE and the real work is product-side - making an app use those routes instead of its own store. Office, Design, the extension, Query and Recall keep five stores that never read each other, which is the whole reason a user logs in again in every app; Cowork already does it right and is named as the pattern. Also folded in, from research rather than guesswork: no off-the-shelf wrapper is reusable (the only mature tools mapping is stale Rust, the popular one says in its README that function calling is unsupported, the third is GPL-3.0, and the 52.8k-star one never spawns the CLI - it stores the OAuth token and calls the vendor backend, which is exactly what April blocked). Two specifics ARE worth copying: pooled long-lived CLI processes, and launching with --tools "" --setting-sources "" --system-prompt , which strips ~28k tokens of the harness's own agent prompt and is the concrete mechanism behind the chat tier. And claude -p's stream-json schema is now mapped, so the old "schema unverified" note is replaced by the real field locations, plus the warning that its credential precedence puts the OAuth login LAST - ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, apiKeyHelper, CLAUDE_CODE_OAUTH_TOKEN and ANTHROPIC_PROFILE in the environment all silently take over billing, and --bare never reads the OAuth login at all. Same class of bug as the Codex env trap, opposite direction. Co-authored-by: Janghoon Lee <44862514+savagemanage@users.noreply.github.com> --- docs/LOCAL-HARNESS-BACKENDS.md | 161 ++++++++++++++++++++++++++++----- docs/PROVIDER-AUTH.md | 45 +++++---- 2 files changed, 167 insertions(+), 39 deletions(-) diff --git a/docs/LOCAL-HARNESS-BACKENDS.md b/docs/LOCAL-HARNESS-BACKENDS.md index 0ffdaec883..4b8efde0f3 100644 --- a/docs/LOCAL-HARNESS-BACKENDS.md +++ b/docs/LOCAL-HARNESS-BACKENDS.md @@ -93,14 +93,96 @@ chat tier: - **No tool protocol** — query, recall, reblend. Chat tier is the whole story. - **Host-owned tools** — office, browser, canvas. Chat tier works today. Giving the - harness their tools needs MCP servers (`~/.codex/config.toml`, or per-invocation - `--config mcp_servers.…`) or Codex app-server's `dynamicTools`, which lets the - tool stay in the host process. `dynamicTools` is the better fit and is labelled - experimental by OpenAI, so it is not a foundation to build on yet. + harness their tools means a **stdio MCP server** (`~/.codex/config.toml`, a + project-scoped `.codex/config.toml`, or per-invocation + `--config mcp_servers.…`; Claude Code takes `--mcp-config` plus + `--strict-mcp-config`). Codex app-server's `dynamicTools` would let the tool stay + inside the host process instead, and an earlier draft of this document called that + "the better fit" — that recommendation is **withdrawn** on evidence: + + - OpenAI's stability warning got *broader*, not narrower. The + [2026-06-24 snapshot](http://web.archive.org/web/20260624142043/https://developers.openai.com/codex/app-server) + carried no production warning at all and scoped "experimental" to the WebSocket + transport; today's page says "The app-server command and WebSocket transport are + experimental and aren't supported for production workloads." + - app-server has **no row at all** in OpenAI's + [Feature Maturity](https://developers.openai.com/codex/feature-maturity) table. + - `dynamicTools` is double-experimental: gated behind + `capabilities.experimentalApi`, and **absent from the generated + `ThreadStartParams` bindings**, so the field you must send is one you hand-write + against no type. It has already changed wire shape once — `LegacyDynamicToolSpec` + exists as the compat scar, with `exposeToContext` replaced by the *inverted* + `deferLoading`. + - No protocol version, no breaking-change log, and an open regression in the `-c` + MCP path under app-server mode + ([openai/codex#39537](https://github.com/openai/codex/issues/39537)). + + The counter-signal is real and is why this is "revisit in a quarter" rather than + "avoid": Zed's ACP adapter *moved onto* app-server + ([agentclientprotocol/codex-acp](https://github.com/agentclientprotocol/codex-acp)), + and OpenAI's own IDE surfaces run on it. The one thing MCP cannot give us is a tool + whose implementation lives in our own process memory. When Office genuinely needs + that, budget for re-porting `dynamicTools` at least once. - **Engine-owned tools** — cowork, cad, extension, design. Under a harness backend the engine's own tools are not in the loop at all. That is the agent tier's problem to solve. +## The vendor risk is asymmetric, and it sets the order + +An earlier draft of this document said driving the unmodified binary is "the +documented exception". That was too confident, and the correction matters enough to +change sequencing. + +Anthropic began blocking third-party harnesses from Claude **subscription** billing +on **2026-04-04**, and the restriction is described as being extended to *all* +third-party harnesses +([claude-mem#1826](https://github.com/thedotmack/claude-mem/issues/1826), which links +the press coverage). OpenClaw is researching a `--method cli` path precisely because +direct OAuth token use has been blocked since then — so CLI-driving is the surviving +workaround, **not a safe harbour**. The same issue warns that spawning a `claude` +subprocess through the Agent SDK "is exactly the 'third-party harness' pattern +Anthropic is restricting". Anthropic's own June support article still says +subscription limits fund `claude -p` and third-party apps and that the metering +change is paused; treat the two together as a trajectory, not a guarantee. + +OpenAI points the other way: it publishes the integration surface as a platform, and +LiteLLM openly ships a `chatgpt/` provider for subscription access. + +Two consequences, both load-bearing: + +1. **BYOK with the user's own API key is the first deliverable, not the fallback.** + It is unaffected by any of the above, it is the path Anthropic and Google both + name as the supported one for a third-party tool, and Office's policy is already + open for it. +2. **Codex before Claude Code**, for business reasons rather than technical ones. + +## Nothing off the shelf is reusable + +Surveyed 2026-09-22. No existing project is production-credible for our requirement +— drive the local CLI *and* pass caller-supplied tools through: + +- [claude-code-api-rs](https://github.com/ZhangHanDong/claude-code-api-rs) (177★, + MIT) is the only mature native `tools` → `tool_calls` mapping, and it is Rust, + five months without a commit, and ships with + `use_interactive_sessions = false # Disabled by default due to stability issues`. +- [claude-code-openai-wrapper](https://github.com/RichardAtCT/claude-code-openai-wrapper) + (622★) states in its own README that function calling is not supported. +- [codingworkflow/claude-code-api](https://github.com/codingworkflow/claude-code-api) + (331★) is **GPL-3.0** — viral, so unusable in a shipped product. +- [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) (52.8k★) looks like the + obvious answer and is not: it never spawns the CLI. It performs the CLI's OAuth + flow itself, stores the token, and calls the vendor backend — the exact + architecture Anthropic blocked in April. +- LiteLLM, OpenRouter, Portkey, Helicone: no provider that drives a local Claude Code + or Codex CLI. + +Two specifics are worth copying rather than the repos: the **pooled long-lived CLI +process** model and the native tools mapping from claude-code-api-rs, and the trick of +launching with `--tools "" --setting-sources "" --system-prompt `, which +strips roughly 28k tokens of the harness's own agent prompt and built-in tools out of +every request. That second one is the concrete mechanism behind the chat tier — it is +what turns an agent harness into something that behaves like a completion endpoint. + ## What the products have to do Almost nothing, which is the point. @@ -114,33 +196,66 @@ into "unavailable" strands the user, because the remedy differs and we are not allowed to offer the vendor's login ourselves. The remedy we may show is "run `codex login`" or "run `claude`". +Both states are readable by ASKING the runtime, never by reading its credential +store: `claude auth status` exits 0 when signed in and 1 when not (its JSON field +names are undocumented, so the exit code is the contract), and Codex app-server +exposes `account/read`. A `claude -p` run also reports `apiKeySource` in its +`system`/`init` event. + Credentials need no work anywhere. The harness holds its own; the engine holds none for it. Cowork already demonstrates the pattern for the BYOK case — it stores no provider key and treats the engine's `auth.json` as the single source of truth. ## Sequencing -1. **Land `/v1/chat/completions`.** It is the receiving route for everything above - and it is not merged: the handler exists on `feature/v1-chat-completions`, and - `test:httpapi` fails without an `httpapi-exercise` scenario. Also needs the - `chatCompletions` capability flag, the SSE OpenAPI patch, and a route test. - Nothing here can ship before it. -2. **Build the shim with both backends.** One local chat-completions server; two +1. **BYOK with the user's own API key.** Moved to the front — see the vendor-risk + section. The engine side is **already built** — `GET /api/integration` returns + each integration's `methods` (OAuth / Key / Env) and its `connections`, and + `connect/key`, `connect/oauth`, the attempt routes and `DELETE /api/credential` + complete the set. An earlier draft of `docs/PROVIDER-AUTH.md` proposed a parallel + `/v1/providers` surface; that is withdrawn, because it would duplicate these over + the same store. The remaining work is product-side: make an app use them instead + of its own key store. Cowork already does (`apps/server/src/redrob-auth.ts`). +2. **Use the user's OWN binary.** The Codex SDK pins `@openai/codex` exactly and + resolves the executable from its own bundled platform packages unless + `codexPathOverride` is passed — so the default behaviour runs a SECOND copy we + downloaded, not the one the user signed in to. That re-creates the duplicate-engine + problem this whole effort exists to remove. Detect the user's install and pass the + path explicitly. Claude Code is proprietary with no redistribution grant, so it is + the only option there anyway, which makes both runtimes the same shape. +3. **Build the shim with both backends.** One local chat-completions server; two normalizers behind it. The prototype's split — event folding separated from the subprocess — is what makes the second runtime a second normalizer rather than a second architecture. Reuse it rather than re-deriving it. -3. **Register through the local-provider path**, and surface the backends in - `/v1/providers` with their three states so a product can render settings - without hardcoding a list. -4. **Verify against real binaries.** Neither runtime is installed on the build +4. **Give the harness our tools through a stdio MCP server.** Not app-server. Know + the defaults before wiring: Codex's `startup_timeout_sec` is 10, `tool_timeout_sec` + is 60, and `required = true` makes `codex exec` exit with an error rather than + silently running without our server. +5. **Register through the local-provider path**, and report the harness backends + through `GET /api/integration` alongside every other integration, with their + three states, so a product renders settings from one list rather than hardcoding + a vendor set that goes stale. +6. **Verify against real binaries.** Neither runtime is installed on the build host, so the live path — a real subscription actually paying for a turn — is unproven until someone runs it on a machine with `codex` and `claude` signed in. -5. **Then the agent tier**, for cowork, code and cad, over ACP. - -Two things to carry forward rather than discover later. `claude -p`'s -`stream-json` event schema has not been checked against the real binary — only the -flags are confirmed — so step 2 starts by reading it, not by assuming it mirrors -Codex's JSONL. And Anthropic has announced, then paused, a change that moves -third-party subscription usage onto a capped monthly credit; it currently still -draws from the subscription, but the trajectory is known, so the Claude backend -should surface usage state rather than assume it is free. +7. **Then the agent tier**, for cowork, code and cad, over ACP. + +One thing to carry forward rather than discover later: Anthropic has announced, then +paused, a change moving third-party subscription usage onto a capped monthly credit, +so the Claude backend should surface usage state rather than assume it is free. Note +also that `claude -p`'s credential precedence puts the OAuth login **last** — +`ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN`, `apiKeyHelper`, `CLAUDE_CODE_OAUTH_TOKEN` +and `ANTHROPIC_PROFILE` in the environment all silently take over billing, and +`--bare` never reads the OAuth login at all. That is the same class of bug as the +Codex env trap, in the opposite direction: the shim must build the child environment +from an allow-list for both runtimes. + +`claude -p`'s `stream-json` schema IS now mapped, so step 3 does not start by +guessing: `system`/`init` carries `session_id`, `tools`, `mcp_servers` and +`apiKeySource`; assistant text and `tool_use` blocks are in +`assistant.message.content`; and the final `result` message carries +`subtype` (`success` or `error_*`), `is_error`, `total_cost_usd` and `modelUsage`. +Read cost from `modelUsage`, not `usage` — `usage` covers the main loop only and +undercounts subagents. Pin non-interactive behaviour with +`--permission-mode dontAsk --permission-prompts none`, and read-only with +`--tools "Read,Glob,Grep" --disallowedTools "Edit" "Write" "NotebookEdit" "Bash" "mcp__*"`. diff --git a/docs/PROVIDER-AUTH.md b/docs/PROVIDER-AUTH.md index e478ac6413..a0429839b2 100644 --- a/docs/PROVIDER-AUTH.md +++ b/docs/PROVIDER-AUTH.md @@ -123,22 +123,35 @@ key cannot leak one, log one, or sync one. `redrob providers login` already implements both shapes — a generic OAuth flow with `authorize()` plus `auto` and `code` callbacks, and an API-key path -(`packages/redrob/src/cli/cmd/providers.ts`). What is missing is not the flow but -its exposure: a product cannot drive it today. - -Add, on the v2 surface beside the completions route: - -- `GET /v1/providers` — what this engine can use, each entry declaring which auth - methods it accepts (`oauth`, `api-key`) and whether a credential is present. - This is what lets an app render a settings page without hardcoding a vendor - list that goes stale. -- `POST /v1/providers/:id/login` — starts a flow. For OAuth it returns the - authorization URL and an opaque attempt id; for an API key it accepts the key. -- `POST /v1/providers/:id/login/:attempt` — completes an OAuth attempt with the - authorization code, or reports that the loopback callback already completed it. -- `DELETE /v1/providers/:id/credential` — disconnect. - -Three rules on those routes: +(`packages/redrob/src/cli/cmd/providers.ts`). + +**An earlier draft of this section proposed four new `/v1/providers` routes to expose +it. That was wrong: the routes already exist.** They are the `server.integration` +group on the v2 surface, and they already carry everything a settings page needs. +Building the `/v1/providers` set would have been a duplicate surface over the same +credential store, with two code paths to keep in agreement. + +What exists, and what it replaces from that proposal: + +| proposed | already exists | +| --- | --- | +| `GET /v1/providers` | `GET /api/integration` — returns `Integration.Info`, whose `methods` array is a union of `OAuthMethod`, `KeyMethod` and `EnvMethod`, plus `connections` for what is already connected | +| `POST /v1/providers/:id/login` (key) | `POST /api/integration/:integrationID/connect/key` | +| `POST /v1/providers/:id/login` (oauth) | `POST /api/integration/:integrationID/connect/oauth` | +| `POST /v1/providers/:id/login/:attempt` | `POST /api/integration/attempt/:attemptID/complete`, with `GET /api/integration/attempt/:attemptID` for status and `DELETE` to cancel | +| `DELETE /v1/providers/:id/credential` | `DELETE /api/credential/:credentialID` | + +So the engine-side work for BYOK is **done**, and the real work is product-side: +making an app use these routes instead of its own store. Office keeps keys in +`userData/ai-settings.json`, Design in the macOS keychain, the extension in +`chrome.storage.local`, Query and Recall in separate keyring services — five stores +that never read each other, which is the whole reason a user logs in again in every +app. Cowork already does it the right way: it stores no provider key and posts the +key once to the engine (`apps/server/src/redrob-auth.ts`), treating the engine's +`auth.json` as the single source of truth. That is the pattern to copy. + +Three rules still hold for those routes, and they are worth restating because they +are what makes the single store safe: 1. **The key never comes back out.** A response says a credential is present and names it; it never returns the secret. A product that cannot read the key From ec8ae0aad702a80cb9bbe4be2735ca93c2de65ec Mon Sep 17 00:00:00 2001 From: Janghoon Lee <44862514+its-janghoon@users.noreply.github.com> Date: Wed, 23 Sep 2026 15:13:25 +0900 Subject: [PATCH 04/33] feat(protocol): document the chat-completions SSE response (#61) * feat(protocol): document the chat-completions SSE response The route answers with `application/json` or with `text/event-stream` depending on `stream`, and HttpApi can only declare one success schema. So the generated spec described only the JSON, and a client built from it looks for choices[].message.content on a streaming response and finds nothing on every frame - because a chunk carries choices[].delta instead. That is the same gap /event has, and the same fix site. Adds ChatCompletionChunk, which is genuinely a different shape rather than a reuse: the object discriminator is chat.completion.chunk, and each choice carries a partial delta instead of a complete message. Every delta field is optional because OpenAI's stream uses the shape sparsely - the first frame typically carries only role, middle frames only content, a tool call arrives spread across frames, and the terminator carries an empty delta with finish_reason set. A schema that required content would reject the terminator. Declared for documentation and codegen only. The handler stays handleRaw and writes its own SSE frames, because choosing between JSON and a stream per request is exactly what HttpApi cannot express. Both content types are declared on the one 200 rather than the SSE replacing the JSON, so the spec says what actually happens instead of lying in the other direction. Registered through HttpApi.AdditionalSchemas. A $ref to a schema no endpoint references dangles otherwise, and a dangling ref generates a client with a missing type rather than failing loudly - which is why one of the three new tests asserts the ref resolves to a registered component rather than only that the ref is present. Coverage 211/0/0, route and wire tests 28/28, openapi tests 21/21, protocol and redrob typecheck clean. The generated artifacts are NOT regenerated here. `bun script/generate.ts` also rewrites packages/sdk/openapi.json, which is stale on clean develop by 991 lines including routes that are not mine, so committing it would sweep another session's unregenerated surface into this change. It needs its own regeneration commit, which is being raised separately. * chore(sdk): regenerate the stale generated artifacts No source change. `bun script/generate.ts` output only. Two things land together, and the split matters for review: The chat-completions SSE documentation from the previous commit - the 200 now declares both application/json and text/event-stream, and ChatCompletionChunk appears as a component. Pre-existing drift that was already in the repo before this branch. packages/sdk/openapi.json was stale on clean develop by roughly 900 lines, including routes that are not mine (/api/variant/paraphrase among them). Those were regenerated here because there is no way to regenerate only part of the file, and leaving them would mean the committed spec still does not describe the served API. This is why it is a separate commit: the source change is reviewable on its own, and this one is verifiable by re-running the generator. There is no CI check for this drift, which is why it accumulated. The client generator IS checked (packages/client check:generated fails on any diff) and passes; the SDK spec is not. Worth adding a check, but that is a change to CI rather than to artifacts and does not belong here. Also deliberately NOT committed: the ~54 files `generate.ts` reformats as a side effect, mostly markdown table alignment and line wrapping in untouched sources. Those are the generator running a formatter over the repo, not generated output, and sweeping them in would bury the three files that actually matter. --------- Co-authored-by: Janghoon Lee <44862514+savagemanage@users.noreply.github.com> --- .../protocol/src/groups/chat-completion.ts | 43 + .../src/server/routes/instance/httpapi/api.ts | 6 + .../server/routes/instance/httpapi/public.ts | 22 + .../server/httpapi-public-openapi.test.ts | 45 + packages/sdk/js/src/v2/gen/sdk.gen.ts | 41 + packages/sdk/js/src/v2/gen/types.gen.ts | 143 +++ packages/sdk/openapi.json | 1022 ++++++++++++++++- 7 files changed, 1261 insertions(+), 61 deletions(-) diff --git a/packages/protocol/src/groups/chat-completion.ts b/packages/protocol/src/groups/chat-completion.ts index dc1ebdde43..373baea9d8 100644 --- a/packages/protocol/src/groups/chat-completion.ts +++ b/packages/protocol/src/groups/chat-completion.ts @@ -132,6 +132,49 @@ export const ChatCompletionResponse = Schema.Struct({ }).annotate({ identifier: "ChatCompletionResponse" }) export type ChatCompletionResponse = typeof ChatCompletionResponse.Type +/** + * One frame of the `stream: true` response. + * + * A DIFFERENT shape from `ChatCompletionResponse`, which is why documenting the streaming + * path needed this rather than reusing the JSON one: the object discriminator is + * `chat.completion.chunk`, and each choice carries a partial `delta` instead of a complete + * `message`. A generated client that assumed the non-streaming shape would look for + * `choices[].message.content` and find nothing on every frame. + * + * Declared for documentation and codegen only — the handler is `handleRaw` and writes SSE + * frames itself, because one request answers with JSON and another with + * `text/event-stream`. This type is what makes the OpenAPI honest about the second. + * + * Every field on `delta` is optional because OpenAI's stream uses the shape sparsely: the + * first frame typically carries only `role`, middle frames only `content`, a tool call + * arrives spread across frames, and the terminator carries an empty delta with + * `finish_reason` set. A schema that required `content` would reject the terminator. + */ +export const ChatCompletionChunk = Schema.Struct({ + id: Schema.String, + object: Schema.Literal("chat.completion.chunk"), + created: Schema.Int, + model: Schema.String, + choices: Schema.Array( + Schema.Struct({ + index: Schema.Int, + delta: Schema.Struct({ + role: Schema.optional(Schema.Literal("assistant")), + content: Schema.optional(Schema.String), + tool_calls: Schema.optional(Schema.Array(ToolCall)), + reasoning_content: Schema.optional(Schema.String), + }), + finish_reason: Schema.NullOr(Schema.String), + }), + ), + /** + * Sent on the final frame by providers that report it. Optional because many do not, + * and a client must not wait for a frame that never comes. + */ + usage: Schema.optional(Usage), +}).annotate({ identifier: "ChatCompletionChunk" }) +export type ChatCompletionChunk = typeof ChatCompletionChunk.Type + /** * The OpenAI error envelope. * diff --git a/packages/redrob/src/server/routes/instance/httpapi/api.ts b/packages/redrob/src/server/routes/instance/httpapi/api.ts index 8010729d18..93418db36e 100644 --- a/packages/redrob/src/server/routes/instance/httpapi/api.ts +++ b/packages/redrob/src/server/routes/instance/httpapi/api.ts @@ -28,6 +28,7 @@ import { WorkspaceApi } from "./groups/workspace" import { makeApi } from "@redrob-code/protocol/api" import { LocationMiddleware } from "@redrob-code/server/location" import { SessionLocationMiddleware } from "@redrob-code/server/middleware/session-location" +import { ChatCompletionChunk } from "@redrob-code/protocol/groups/chat-completion" import { GlobalApi } from "./groups/global" import { Authorization } from "./middleware/authorization" import { SchemaErrorMiddleware } from "./middleware/schema-error" @@ -91,6 +92,11 @@ export const RedrobHttpApi = HttpApi.make("redrob") Integration.Method, Integration.Ref, SkillV2.Source, + // Not referenced by any endpoint's declared success type -- the streaming shape is + // patched onto /v1/chat/completions in public.ts, because HttpApi cannot express "JSON + // or SSE depending on a request field". Registering it here is what makes that patch's + // $ref resolve instead of dangling in the generated spec. + ChatCompletionChunk, ]) export type RootHttpApiType = typeof RootHttpApi diff --git a/packages/redrob/src/server/routes/instance/httpapi/public.ts b/packages/redrob/src/server/routes/instance/httpapi/public.ts index 27d43b226c..9c64dd3729 100644 --- a/packages/redrob/src/server/routes/instance/httpapi/public.ts +++ b/packages/redrob/src/server/routes/instance/httpapi/public.ts @@ -169,6 +169,28 @@ function matchLegacyOpenApi(input: Record) { }, } } + if (path === "/v1/chat/completions" && method === "post") { + // Same gap, and worse consequences here: this route answers with JSON or with + // text/event-stream depending on `stream`, and the declared success schema only + // describes the JSON. A generated client built from the unpatched spec looks for + // choices[].message.content on a streaming response and finds nothing on every + // frame, because a chunk carries choices[].delta instead. + // + // Both content types are declared on the one 200 so the spec says what actually + // happens, rather than replacing the JSON shape and lying in the other direction. + const existing = operation.responses?.["200"] as + | { description?: string; content?: Record } + | undefined + operation.responses!["200"] = { + description: + "Chat completion. `application/json` when `stream` is false or absent; " + + "`text/event-stream` of `ChatCompletionChunk` frames terminated by `data: [DONE]` when true.", + content: { + ...(existing?.content ?? {}), + "text/event-stream": { schema: { $ref: "#/components/schemas/ChatCompletionChunk" } }, + }, + } + } const route = `${method.toUpperCase()} ${path}` for (const param of operation.parameters ?? []) normalizeParameter(param, route) } diff --git a/packages/redrob/test/server/httpapi-public-openapi.test.ts b/packages/redrob/test/server/httpapi-public-openapi.test.ts index 310ebae595..9b43c09aec 100644 --- a/packages/redrob/test/server/httpapi-public-openapi.test.ts +++ b/packages/redrob/test/server/httpapi-public-openapi.test.ts @@ -10,6 +10,8 @@ type OpenApiSchema = { readonly enum?: readonly unknown[] readonly properties?: Record readonly required?: readonly string[] + /** Element schema of an array property, e.g. a chunk's `choices`. */ + readonly items?: OpenApiSchema readonly contentSchema?: OpenApiSchema readonly contentMediaType?: string } @@ -70,6 +72,49 @@ function isBuiltInEndpointError(name: string) { } describe("PublicApi OpenAPI v2 errors", () => { + test("documents both content types on the chat-completions 200", () => { + // The route answers with JSON or with text/event-stream depending on `stream`, and + // HttpApi can only declare one success schema. Without the patch the spec describes + // only the JSON, so a generated client looks for choices[].message.content on a + // streaming response and finds nothing on every frame -- a chunk carries + // choices[].delta instead. + const spec = OpenApi.fromApi(PublicApi) as OpenApiSpec + const response = spec.paths["/v1/chat/completions"]?.post?.responses?.["200"] + + expect(Object.keys(response?.content ?? {}).sort()).toEqual(["application/json", "text/event-stream"]) + // Both, not one replacing the other: dropping the JSON would lie in the other direction. + expect(response?.content?.["application/json"]).toBeDefined() + expect(response?.description).toContain("[DONE]") + }) + + test("the chat-completions SSE ref resolves to a registered component", () => { + // A $ref to a schema no endpoint references dangles unless it is registered through + // HttpApi.AdditionalSchemas. A dangling ref generates a client with a missing type + // rather than failing loudly, so this is the assertion that keeps the patch honest. + const spec = OpenApi.fromApi(PublicApi) as OpenApiSpec + const ref = spec.paths["/v1/chat/completions"]?.post?.responses?.["200"]?.content?.[ + "text/event-stream" + ]?.schema?.$ref + + expect(ref).toBe("#/components/schemas/ChatCompletionChunk") + expect(Object.keys(spec.components.schemas)).toContain("ChatCompletionChunk") + }) + + test("a chunk is shaped as a delta, not as a complete message", () => { + const spec = OpenApi.fromApi(PublicApi) as OpenApiSpec + const chunk = spec.components.schemas.ChatCompletionChunk + const choice = chunk?.properties?.choices?.items as OpenApiSchema | undefined + + expect(choice?.properties?.delta).toBeDefined() + // The discriminator differs from the non-streaming object, which is the whole reason a + // separate schema was needed. + expect(chunk?.properties?.object?.enum).toEqual(["chat.completion.chunk"]) + // Every delta field is optional: the first frame carries only role, middle frames only + // content, and the terminator an empty delta. Requiring content would reject the + // terminator. + expect(choice?.properties?.delta?.required ?? []).toEqual([]) + }) + test("includes plugin-facing core schemas", () => { const spec = OpenApi.fromApi(PublicApi) as OpenApiSpec diff --git a/packages/sdk/js/src/v2/gen/sdk.gen.ts b/packages/sdk/js/src/v2/gen/sdk.gen.ts index 0b90a58f55..ce3837420c 100644 --- a/packages/sdk/js/src/v2/gen/sdk.gen.ts +++ b/packages/sdk/js/src/v2/gen/sdk.gen.ts @@ -15,6 +15,7 @@ import type { AuthRemoveResponses, AuthSetErrors, AuthSetResponses, + ChatCompletionRequest, CommandListErrors, CommandListResponses, Config as Config3, @@ -263,6 +264,8 @@ import type { TuiShowToastResponses, TuiSubmitPromptErrors, TuiSubmitPromptResponses, + V1ChatCompletionsErrors, + V1ChatCompletionsResponses, V2AgentListErrors, V2AgentListResponses, V2CommandListErrors, @@ -7137,6 +7140,39 @@ export class V2 extends HeyApiClient { } } +export class Chat extends HeyApiClient { + /** + * Create a chat completion + * + * OpenAI-compatible inference against the engine's own credential and provider registry. Declaring `tools` makes the caller responsible for executing them: the turn ends with finish_reason tool_calls and the results come back as role:tool messages. `stream: true` returns text/event-stream instead of this JSON body. Failures use OpenAI's error envelope -- {error:{message,type,code,param}} -- with 400 invalid_request_error, 401 authentication_error (code engine_not_authenticated, the only status a client should offer a sign-in action for), 429 rate_limit_error or insufficient_quota, and 502 api_error. They are written by the handler rather than declared as endpoint errors, because this is a raw handler and the envelope carries no discriminator field for codegen to branch on. + */ + public completions( + parameters?: { + chatCompletionRequest?: ChatCompletionRequest + }, + options?: Options, + ) { + const params = buildClientParams([parameters], [{ args: [{ key: "chatCompletionRequest", map: "body" }] }]) + return (options?.client ?? this.client).post({ + url: "/v1/chat/completions", + ...options, + ...params, + headers: { + "Content-Type": "application/json", + ...options?.headers, + ...params.headers, + }, + }) + } +} + +export class V1 extends HeyApiClient { + private _chat?: Chat + get chat(): Chat { + return (this._chat ??= new Chat({ client: this.client })) + } +} + export class RedrobClient extends HeyApiClient { public static readonly __registry = new HeyApiRegistry() @@ -7279,4 +7315,9 @@ export class RedrobClient extends HeyApiClient { get v2(): V2 { return (this._v2 ??= new V2({ client: this.client })) } + + private _v1?: V1 + get v1(): V1 { + return (this._v1 ??= new V1({ client: this.client })) + } } diff --git a/packages/sdk/js/src/v2/gen/types.gen.ts b/packages/sdk/js/src/v2/gen/types.gen.ts index ac9249b06d..11e8c0238a 100644 --- a/packages/sdk/js/src/v2/gen/types.gen.ts +++ b/packages/sdk/js/src/v2/gen/types.gen.ts @@ -106,6 +106,35 @@ export type QuestionRejected = { requestID: string } +export type ChatCompletionChunk = { + id: string + object: "chat.completion.chunk" + created: number + model: string + choices: Array<{ + index: number + delta: { + role?: "assistant" + content?: string + tool_calls?: Array<{ + id: string + type: "function" + function: { + name: string + arguments: string + } + }> + reasoning_content?: string + } + finish_reason: string + }> + usage?: { + prompt_tokens: number + completion_tokens: number + total_tokens: number + } +} + export type OAuth = { type: "oauth" refresh: string @@ -2142,8 +2171,15 @@ export type Provider = { } } +export type ChatCompletionsCapability = { + version: number + callerTools: boolean + stream: boolean +} + export type ExperimentalCapabilities = { backgroundSubagents: boolean + chatCompletions: ChatCompletionsCapability } export type ConsoleState = { @@ -2973,6 +3009,88 @@ export type ProjectCopyError = { } } +export type ChatCompletionRequest = { + model: string + messages: Array<{ + role: "system" | "developer" | "user" | "assistant" | "tool" + content?: + | string + | Array<{ + [key: string]: unknown + }> + tool_calls?: Array<{ + id: string + type: "function" + function: { + name: string + arguments: string + } + }> + tool_call_id?: string + name?: string + }> + tools?: Array<{ + type: "function" + function: { + name: string + description?: string + parameters?: { + [key: string]: unknown + } + } + }> + tool_choice?: + | "auto" + | "none" + | "required" + | { + type: "function" + function: { + name: string + } + } + stream?: boolean + max_tokens?: number + max_completion_tokens?: number + temperature?: number | "NaN" | "Infinity" | "-Infinity" | "Infinity" | "-Infinity" | "NaN" + top_p?: number | "NaN" | "Infinity" | "-Infinity" | "Infinity" | "-Infinity" | "NaN" + stop?: string | Array + seed?: number + frequency_penalty?: number | "NaN" | "Infinity" | "-Infinity" | "Infinity" | "-Infinity" | "NaN" + presence_penalty?: number | "NaN" | "Infinity" | "-Infinity" | "Infinity" | "-Infinity" | "NaN" + reasoning_effort?: string + user?: string +} + +export type ChatCompletionResponse = { + id: string + object: "chat.completion" + created: number + model: string + choices: Array<{ + index: number + message: { + role: "assistant" + content: string + tool_calls?: Array<{ + id: string + type: "function" + function: { + name: string + arguments: string + } + }> + reasoning_content?: string + } + finish_reason: string + }> + usage?: { + prompt_tokens: number + completion_tokens: number + total_tokens: number + } +} + export type EffectHttpApiErrorForbidden = { _tag: "Forbidden" } @@ -13720,6 +13838,31 @@ export type V2ProjectCopyRefreshResponses = { export type V2ProjectCopyRefreshResponse = V2ProjectCopyRefreshResponses[keyof V2ProjectCopyRefreshResponses] +export type V1ChatCompletionsData = { + body?: ChatCompletionRequest + path?: never + query?: never + url: "/v1/chat/completions" +} + +export type V1ChatCompletionsErrors = { + /** + * Bad request + */ + 400: BadRequestError +} + +export type V1ChatCompletionsError = V1ChatCompletionsErrors[keyof V1ChatCompletionsErrors] + +export type V1ChatCompletionsResponses = { + /** + * Chat completion. `application/json` when `stream` is false or absent; `text/event-stream` of `ChatCompletionChunk` frames terminated by `data: [DONE]` when true. + */ + 200: ChatCompletionResponse +} + +export type V1ChatCompletionsResponse = V1ChatCompletionsResponses[keyof V1ChatCompletionsResponses] + export type PtyConnectData = { body?: never path: { diff --git a/packages/sdk/openapi.json b/packages/sdk/openapi.json index 2e5d02c714..cd55bbe820 100644 --- a/packages/sdk/openapi.json +++ b/packages/sdk/openapi.json @@ -11947,6 +11947,190 @@ ] } }, + "/api/variant/paraphrase": { + "post": { + "tags": ["variants"], + "operationId": "v2.variant.paraphrase", + "parameters": [], + "security": [], + "responses": { + "200": { + "description": "Variant.Result", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/VariantResult" + } + } + } + }, + "400": { + "description": "InvalidRequestError", + "content": { + "application/json": { + "schema": { + "anyOf": [ + { + "$ref": "#/components/schemas/InvalidRequestError" + }, + { + "$ref": "#/components/schemas/InvalidRequestError" + } + ] + } + } + } + }, + "401": { + "description": "UnauthorizedError", + "content": { + "application/json": { + "schema": { + "anyOf": [ + { + "$ref": "#/components/schemas/UnauthorizedError" + }, + { + "$ref": "#/components/schemas/UnauthorizedError" + } + ] + } + } + } + }, + "404": { + "description": "ProviderNotFoundError", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProviderNotFoundError" + } + } + } + }, + "503": { + "description": "ServiceUnavailableError", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ServiceUnavailableError" + } + } + } + } + }, + "description": "Send the same text to two or more models and get each rewrite back, with the model that actually answered and what that slot cost.", + "summary": "Rewrite one text with several models", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/VariantParaphraseRequest" + } + } + }, + "required": true + }, + "x-codeSamples": [ + { + "lang": "js", + "source": "import { createRedrobClient } from \"@redrob-labs/sdk\n\nconst client = createRedrobClient()\nawait client.v2.variant.paraphrase({\n ...\n})" + } + ] + } + }, + "/api/variant/compare": { + "post": { + "tags": ["variants"], + "operationId": "v2.variant.compare", + "parameters": [], + "security": [], + "responses": { + "200": { + "description": "Variant.Result", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/VariantResult" + } + } + } + }, + "400": { + "description": "InvalidRequestError", + "content": { + "application/json": { + "schema": { + "anyOf": [ + { + "$ref": "#/components/schemas/InvalidRequestError" + }, + { + "$ref": "#/components/schemas/InvalidRequestError" + } + ] + } + } + } + }, + "401": { + "description": "UnauthorizedError", + "content": { + "application/json": { + "schema": { + "anyOf": [ + { + "$ref": "#/components/schemas/UnauthorizedError" + }, + { + "$ref": "#/components/schemas/UnauthorizedError" + } + ] + } + } + } + }, + "404": { + "description": "ProviderNotFoundError", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ProviderNotFoundError" + } + } + } + }, + "503": { + "description": "ServiceUnavailableError", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ServiceUnavailableError" + } + } + } + } + }, + "description": "Send the same messages to two or more models and get each answer back, so a caller can offer them as a choice.", + "summary": "Answer one conversation with several models", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/VariantCompareRequest" + } + } + }, + "required": true + }, + "x-codeSamples": [ + { + "lang": "js", + "source": "import { createRedrobClient } from \"@redrob-labs/sdk\n\nconst client = createRedrobClient()\nawait client.v2.variant.compare({\n ...\n})" + } + ] + } + }, "/api/integration": { "get": { "tags": ["integrations"], @@ -15169,6 +15353,57 @@ ] } }, + "/v1/chat/completions": { + "post": { + "tags": ["chat completions"], + "operationId": "v1.chat.completions", + "parameters": [], + "responses": { + "200": { + "description": "Chat completion. `application/json` when `stream` is false or absent; `text/event-stream` of `ChatCompletionChunk` frames terminated by `data: [DONE]` when true.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ChatCompletionResponse" + } + }, + "text/event-stream": { + "schema": { + "$ref": "#/components/schemas/ChatCompletionChunk" + } + } + } + }, + "400": { + "description": "Bad request", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/BadRequestError" + } + } + } + } + }, + "description": "OpenAI-compatible inference against the engine's own credential and provider registry. Declaring `tools` makes the caller responsible for executing them: the turn ends with finish_reason tool_calls and the results come back as role:tool messages. `stream: true` returns text/event-stream instead of this JSON body. Failures use OpenAI's error envelope -- {error:{message,type,code,param}} -- with 400 invalid_request_error, 401 authentication_error (code engine_not_authenticated, the only status a client should offer a sign-in action for), 429 rate_limit_error or insufficient_quota, and 502 api_error. They are written by the handler rather than declared as endpoint errors, because this is a raw handler and the envelope carries no discriminator field for codegen to branch on.", + "summary": "Create a chat completion", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ChatCompletionRequest" + } + } + } + }, + "x-codeSamples": [ + { + "lang": "js", + "source": "import { createRedrobClient } from \"@redrob-labs/sdk\n\nconst client = createRedrobClient()\nawait client.v1.chat.completions({\n ...\n})" + } + ] + } + }, "/pty/{ptyID}/connect": { "get": { "tags": ["pty"], @@ -15565,6 +15800,104 @@ "required": ["sessionID", "requestID"], "additionalProperties": false }, + "ChatCompletionChunk": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "object": { + "type": "string", + "enum": ["chat.completion.chunk"] + }, + "created": { + "type": "integer" + }, + "model": { + "type": "string" + }, + "choices": { + "type": "array", + "items": { + "type": "object", + "properties": { + "index": { + "type": "integer" + }, + "delta": { + "type": "object", + "properties": { + "role": { + "type": "string", + "enum": ["assistant"] + }, + "content": { + "type": "string" + }, + "tool_calls": { + "type": "array", + "items": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "type": { + "type": "string", + "enum": ["function"] + }, + "function": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "arguments": { + "type": "string" + } + }, + "required": ["name", "arguments"], + "additionalProperties": false + } + }, + "required": ["id", "type", "function"], + "additionalProperties": false + } + }, + "reasoning_content": { + "type": "string" + } + }, + "additionalProperties": false + }, + "finish_reason": { + "type": "string" + } + }, + "required": ["index", "delta", "finish_reason"], + "additionalProperties": false + } + }, + "usage": { + "type": "object", + "properties": { + "prompt_tokens": { + "type": "integer" + }, + "completion_tokens": { + "type": "integer" + }, + "total_tokens": { + "type": "integer" + } + }, + "required": ["prompt_tokens", "completion_tokens", "total_tokens"], + "additionalProperties": false + } + }, + "required": ["id", "object", "created", "model", "choices"], + "additionalProperties": false + }, "OAuth": { "type": "object", "properties": { @@ -16324,6 +16657,12 @@ "cost": { "type": "number" }, + "routedModel": { + "type": "string" + }, + "upstreamProvider": { + "type": "string" + }, "tokens": { "type": "object", "properties": { @@ -17271,7 +17610,65 @@ "properties": { "type": { "type": "string", - "enum": ["idle"] + "enum": ["idle"] + } + }, + "required": ["type"], + "additionalProperties": false + }, + { + "type": "object", + "properties": { + "type": { + "type": "string", + "enum": ["retry"] + }, + "attempt": { + "type": "integer", + "minimum": 0 + }, + "message": { + "type": "string" + }, + "action": { + "type": "object", + "properties": { + "reason": { + "type": "string" + }, + "provider": { + "type": "string" + }, + "title": { + "type": "string" + }, + "message": { + "type": "string" + }, + "label": { + "type": "string" + }, + "link": { + "type": "string" + } + }, + "required": ["reason", "provider", "title", "message", "label"], + "additionalProperties": false + }, + "next": { + "type": "integer", + "minimum": 0 + } + }, + "required": ["type", "attempt", "message", "next"], + "additionalProperties": false + }, + { + "type": "object", + "properties": { + "type": { + "type": "string", + "enum": ["busy"] } }, "required": ["type"], @@ -17282,11 +17679,7 @@ "properties": { "type": { "type": "string", - "enum": ["retry"] - }, - "attempt": { - "type": "integer", - "minimum": 0 + "enum": ["blocked"] }, "message": { "type": "string" @@ -17315,24 +17708,9 @@ }, "required": ["reason", "provider", "title", "message", "label"], "additionalProperties": false - }, - "next": { - "type": "integer", - "minimum": 0 - } - }, - "required": ["type", "attempt", "message", "next"], - "additionalProperties": false - }, - { - "type": "object", - "properties": { - "type": { - "type": "string", - "enum": ["busy"] } }, - "required": ["type"], + "required": ["type", "message"], "additionalProperties": false } ] @@ -21515,6 +21893,13 @@ "reserved": { "type": "integer", "minimum": 0 + }, + "threshold": { + "type": "integer", + "minimum": 0 + }, + "maxTurnInputCostUsd": { + "type": "number" } }, "additionalProperties": false @@ -21859,14 +22244,34 @@ "required": ["id", "name", "source", "env", "options", "models"], "additionalProperties": false }, + "ChatCompletionsCapability": { + "type": "object", + "properties": { + "version": { + "type": "integer", + "minimum": 0 + }, + "callerTools": { + "type": "boolean" + }, + "stream": { + "type": "boolean" + } + }, + "required": ["version", "callerTools", "stream"], + "additionalProperties": false + }, "ExperimentalCapabilities": { "type": "object", "properties": { "backgroundSubagents": { "type": "boolean" + }, + "chatCompletions": { + "$ref": "#/components/schemas/ChatCompletionsCapability" } }, - "required": ["backgroundSubagents"], + "required": ["backgroundSubagents", "chatCompletions"], "additionalProperties": false }, "ConsoleState": { @@ -24252,66 +24657,420 @@ { "$ref": "#/components/schemas/WorkspaceFailed" }, - { - "$ref": "#/components/schemas/WorkspaceStatus" + { + "$ref": "#/components/schemas/WorkspaceStatus" + }, + { + "$ref": "#/components/schemas/WorktreeReady" + }, + { + "$ref": "#/components/schemas/WorktreeFailed" + }, + { + "$ref": "#/components/schemas/ServerConnected" + }, + { + "$ref": "#/components/schemas/GlobalDisposed" + } + ] + }, + "V2EventStream": { + "type": "string", + "contentSchema": { + "$ref": "#/components/schemas/V2Event" + }, + "contentMediaType": "application/json" + }, + "ForbiddenError": { + "type": "object", + "properties": { + "_tag": { + "type": "string", + "enum": ["ForbiddenError"] + }, + "message": { + "type": "string" + } + }, + "required": ["_tag", "message"], + "additionalProperties": false + }, + "ProjectCopyError": { + "type": "object", + "properties": { + "name": { + "type": "string", + "enum": ["ProjectCopyError"] + }, + "data": { + "type": "object", + "properties": { + "message": { + "type": "string" + }, + "forceRequired": { + "type": "boolean" + } + }, + "required": ["message"], + "additionalProperties": false + } + }, + "required": ["name", "data"], + "additionalProperties": false + }, + "ChatCompletionRequest": { + "type": "object", + "properties": { + "model": { + "type": "string" + }, + "messages": { + "type": "array", + "items": { + "type": "object", + "properties": { + "role": { + "type": "string", + "enum": ["system", "developer", "user", "assistant", "tool"] + }, + "content": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "array", + "items": { + "type": "object" + } + } + ] + }, + "tool_calls": { + "type": "array", + "items": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "type": { + "type": "string", + "enum": ["function"] + }, + "function": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "arguments": { + "type": "string" + } + }, + "required": ["name", "arguments"], + "additionalProperties": false + } + }, + "required": ["id", "type", "function"], + "additionalProperties": false + } + }, + "tool_call_id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "required": ["role"], + "additionalProperties": false + } + }, + "tools": { + "type": "array", + "items": { + "type": "object", + "properties": { + "type": { + "type": "string", + "enum": ["function"] + }, + "function": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "description": { + "type": "string" + }, + "parameters": { + "type": "object" + } + }, + "required": ["name"], + "additionalProperties": false + } + }, + "required": ["type", "function"], + "additionalProperties": false + } + }, + "tool_choice": { + "anyOf": [ + { + "type": "string", + "enum": ["auto", "none", "required"] + }, + { + "type": "object", + "properties": { + "type": { + "type": "string", + "enum": ["function"] + }, + "function": { + "type": "object", + "properties": { + "name": { + "type": "string" + } + }, + "required": ["name"], + "additionalProperties": false + } + }, + "required": ["type", "function"], + "additionalProperties": false + } + ] + }, + "stream": { + "type": "boolean" + }, + "max_tokens": { + "type": "integer" + }, + "max_completion_tokens": { + "type": "integer" + }, + "temperature": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "string", + "enum": ["NaN"] + }, + { + "type": "string", + "enum": ["Infinity"] + }, + { + "type": "string", + "enum": ["-Infinity"] + }, + { + "type": "string", + "enum": ["Infinity", "-Infinity", "NaN"] + } + ] + }, + "top_p": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "string", + "enum": ["NaN"] + }, + { + "type": "string", + "enum": ["Infinity"] + }, + { + "type": "string", + "enum": ["-Infinity"] + }, + { + "type": "string", + "enum": ["Infinity", "-Infinity", "NaN"] + } + ] + }, + "stop": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "array", + "items": { + "type": "string" + } + } + ] }, - { - "$ref": "#/components/schemas/WorktreeReady" + "seed": { + "type": "integer" }, - { - "$ref": "#/components/schemas/WorktreeFailed" + "frequency_penalty": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "string", + "enum": ["NaN"] + }, + { + "type": "string", + "enum": ["Infinity"] + }, + { + "type": "string", + "enum": ["-Infinity"] + }, + { + "type": "string", + "enum": ["Infinity", "-Infinity", "NaN"] + } + ] }, - { - "$ref": "#/components/schemas/ServerConnected" + "presence_penalty": { + "anyOf": [ + { + "type": "number" + }, + { + "type": "string", + "enum": ["NaN"] + }, + { + "type": "string", + "enum": ["Infinity"] + }, + { + "type": "string", + "enum": ["-Infinity"] + }, + { + "type": "string", + "enum": ["Infinity", "-Infinity", "NaN"] + } + ] }, - { - "$ref": "#/components/schemas/GlobalDisposed" - } - ] - }, - "V2EventStream": { - "type": "string", - "contentSchema": { - "$ref": "#/components/schemas/V2Event" - }, - "contentMediaType": "application/json" - }, - "ForbiddenError": { - "type": "object", - "properties": { - "_tag": { - "type": "string", - "enum": ["ForbiddenError"] + "reasoning_effort": { + "type": "string" }, - "message": { + "user": { "type": "string" } }, - "required": ["_tag", "message"], + "required": ["model", "messages"], "additionalProperties": false }, - "ProjectCopyError": { + "ChatCompletionResponse": { "type": "object", "properties": { - "name": { + "id": { + "type": "string" + }, + "object": { "type": "string", - "enum": ["ProjectCopyError"] + "enum": ["chat.completion"] }, - "data": { + "created": { + "type": "integer" + }, + "model": { + "type": "string" + }, + "choices": { + "type": "array", + "items": { + "type": "object", + "properties": { + "index": { + "type": "integer" + }, + "message": { + "type": "object", + "properties": { + "role": { + "type": "string", + "enum": ["assistant"] + }, + "content": { + "type": "string" + }, + "tool_calls": { + "type": "array", + "items": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "type": { + "type": "string", + "enum": ["function"] + }, + "function": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "arguments": { + "type": "string" + } + }, + "required": ["name", "arguments"], + "additionalProperties": false + } + }, + "required": ["id", "type", "function"], + "additionalProperties": false + } + }, + "reasoning_content": { + "type": "string" + } + }, + "required": ["role", "content"], + "additionalProperties": false + }, + "finish_reason": { + "type": "string" + } + }, + "required": ["index", "message", "finish_reason"], + "additionalProperties": false + } + }, + "usage": { "type": "object", "properties": { - "message": { - "type": "string" + "prompt_tokens": { + "type": "integer" }, - "forceRequired": { - "type": "boolean" + "completion_tokens": { + "type": "integer" + }, + "total_tokens": { + "type": "integer" } }, - "required": ["message"], + "required": ["prompt_tokens", "completion_tokens", "total_tokens"], "additionalProperties": false } }, - "required": ["name", "data"], + "required": ["id", "object", "created", "model", "choices"], "additionalProperties": false }, "effect_HttpApiError_Forbidden": { @@ -27873,6 +28632,12 @@ "cost": { "type": "number" }, + "routedModel": { + "type": "string" + }, + "upstreamProvider": { + "type": "string" + }, "tokens": { "type": "object", "properties": { @@ -30039,6 +30804,133 @@ "required": ["id", "name", "api", "request"], "additionalProperties": false }, + "VariantModel": { + "type": "object", + "properties": { + "model": { + "type": "string" + }, + "variant": { + "type": "string" + } + }, + "required": ["model"], + "additionalProperties": false + }, + "VariantParaphraseRequest": { + "type": "object", + "properties": { + "text": { + "type": "string" + }, + "models": { + "type": "array", + "items": { + "$ref": "#/components/schemas/VariantModel" + } + }, + "requestId": { + "type": "string" + } + }, + "required": ["text", "models"], + "additionalProperties": false + }, + "VariantProvenance": { + "type": "object", + "properties": { + "requestId": { + "type": "string" + }, + "routedModel": { + "type": "string" + }, + "upstreamProvider": { + "type": "string" + }, + "latencyMs": { + "type": "number" + }, + "costUsd": { + "type": "number" + } + }, + "additionalProperties": false + }, + "VariantSlot": { + "type": "object", + "properties": { + "slot": { + "type": "integer" + }, + "model": { + "type": "string" + }, + "text": { + "type": "string" + }, + "error": { + "type": "string" + }, + "redrob": { + "$ref": "#/components/schemas/VariantProvenance" + } + }, + "required": ["slot", "model"], + "additionalProperties": false + }, + "VariantResult": { + "type": "object", + "properties": { + "variants": { + "type": "array", + "items": { + "$ref": "#/components/schemas/VariantSlot" + } + }, + "totalCostUsd": { + "type": "number" + } + }, + "required": ["variants", "totalCostUsd"], + "additionalProperties": false + }, + "VariantMessage": { + "type": "object", + "properties": { + "role": { + "type": "string", + "enum": ["system", "user", "assistant"] + }, + "content": { + "type": "string" + } + }, + "required": ["role", "content"], + "additionalProperties": false + }, + "VariantCompareRequest": { + "type": "object", + "properties": { + "messages": { + "type": "array", + "items": { + "$ref": "#/components/schemas/VariantMessage" + } + }, + "models": { + "type": "array", + "items": { + "$ref": "#/components/schemas/VariantModel" + } + }, + "requestId": { + "type": "string" + } + }, + "required": ["messages", "models"], + "additionalProperties": false + }, "IntegrationWhen": { "type": "object", "properties": { @@ -36898,6 +37790,10 @@ "name": "providers", "description": "Experimental provider routes." }, + { + "name": "variants", + "description": "Experimental multi-model routes. Available while the Redrob provider is connected." + }, { "name": "integrations", "description": "Integration discovery and authentication routes." @@ -36942,6 +37838,10 @@ "name": "projectCopy", "description": "Project copy management routes." }, + { + "name": "chat completions", + "description": "OpenAI-compatible inference route. The shared engine surface for Redrob products." + }, { "name": "pty", "description": "PTY websocket route." From b7bc394ebb52ba0793c71fdcf4455035a6f92543 Mon Sep 17 00:00:00 2001 From: Janghoon Lee <44862514+its-janghoon@users.noreply.github.com> Date: Fri, 2 Oct 2026 22:42:09 +0900 Subject: [PATCH 05/33] feat(skill): K-1 code-mode domain globals and K-2 skill auto-arming (#62) * feat(codemode): K-1 typed domain objects as code-mode globals A skill document is cheap only because the objects it drives already exist. Code Mode could previously only be written as `tools..(...)`, so a skill had to describe a tool call instead of naming a method. Mechanism, in @redrob-code/codemode: `globals` on ExecuteOptions names top-level tool namespaces that are ALSO bound as bare identifiers in the interpreter's global scope. A global is an alias for the same tool path, so there is one implementation and one authorization point, not two. Code Mode stays host-neutral: it never learns what a given name means. The builtin global seeding moves into seedBuiltinGlobals(), and BUILTIN_GLOBAL_NAMES is derived from it, so a builtin added later cannot silently become available as a host global name. assertValidGlobals refuses a name that is not a namespace of the tool tree, one that shadows a builtin, a duplicate, a non-identifier, and a tool rather than a namespace. The generated instructions gain a "Domain globals" section stating the bare and tools forms are the same call. Objects, in packages/redrob/src/tool/domain.ts: Page and Channel as real TypeScript interfaces with a single implementation each, injected under the names `page` and `channel`. - Channel is implemented: `channel.send` appends a text part to the assistant message that owns the execution, so every surface the session is attached to renders it without the engine knowing which surface that is. `channel.id` returns the session id. - Page is NOT implemented, and says so. Nothing in the engine process controls a browser page today, so its single implementation is unavailablePage, whose every method fails with the capability named: "the engine process has no browser-page control surface". It returns no plausible value, so a skill written against the interface fails loudly rather than reading an empty string. The domain namespaces are spread after the MCP catalog deliberately: a connected MCP server named `page` must not shadow the object a skill is written against. * feat(skill): K-2 auto-arming from frontmatter keywords and URL globs Two optional frontmatter keys beside name/description/slash: icon: string # a URL autoInject: keywords: [string] # matched against the prompt url: [string] # globs, e.g. docs.google.com/document/** Both lists are optional and an empty list matches nothing, so `autoInject: {}` is a skill that still never arms rather than one that arms on everything. The arming decision is a pure function in packages/core/src/skill/arming.ts: SkillArming.arm(input: Input): ReadonlyArray SkillArming.armedNames(input: Input): ReadonlyArray It takes the loaded skills, the current prompt text and the current tab URL, and returns which skills arm, with the patterns that armed each one. It reads nothing and caches nothing, so the decision is identical in the prompt builder, in a UI preview and in a test. Keyword and URL matching are both case-insensitive; keywords match on word boundaries so `ai` does not match inside `said`; `*` and `?` stop at a `/` while `**` crosses segments; a `.` in a glob is a literal dot and not a wildcard. The scheme is ignored on both sides, so globs are written without one. A skill with NO autoInject block never auto-arms. It stays explicitly loadable, which is the point: a skill that arms on everything is always in the prompt. Loading decodes one document at a time, so a malformed frontmatter costs that one document and not the set. The decoders are the Result-returning form rather than the Option form, because the Option form answers only "no" and makes the cause unfindable; every rejection is logged at warning level with the file and the decoder's own reason. A malformed autoInject is narrower still: it is ignored with its own warning and the skill loads without it, since that leaves the skill exactly where a skill with no block already sits. The reasons are in the log MESSAGE rather than in annotations, because the default logger renders an annotation object as [object Object] and loses precisely that detail. Tests in packages/core/test/skill/arming.test.ts: 13 pass, 0 fail. They cover a keyword hit, a URL-glob hit, three glob near-misses that must not arm, a skill with no block, an empty block, case-insensitivity in both directions, and a malformed autoInject that is ignored rather than fatal while a sibling bad document is dropped with a logged reason. Each of five restored defects (explicit-only guard removed, `*` crossing `/`, glob compiled as a regex, malformed autoInject made fatal, reason text dropped from the log) was confirmed to fail the suite. Note: packages/schema has 2 failing tests in test/event-manifest.test.ts. They fail identically on a clean tree (git stash push -u, 13 pass / 2 fail both ways) and are unrelated to this change. * feat(client): K-2 regenerate the client types for the new skill keys `packages/client` commits its generated SDK and gates it with `check:generated` (bun run generate && git diff --exit-code), so adding `icon` and `autoInject` to `SkillV2.Info` in packages/schema left the committed artifact behind and that gate red on CI - the core (linux) job passed its tests and then failed on this step, and unit (linux) is only the aggregation job reporting it. Regenerated with the repo's own generator (`bun run generate` in packages/client), not hand-edited, because the gate rejects a hand edit by construction. The whole diff is the two new optional fields on SkillsListOutput, which is exactly the propagation of the schema change: readonly icon?: string readonly autoInject?: { readonly keywords?: ReadonlyArray readonly url?: ReadonlyArray } Measured after the change: packages/client typecheck passes; its suite is 15 pass / 1 fail, and that one failure ("exposes every standard HTTP API group") also fails on a clean origin/develop worktree, so it is not from this change. --------- Co-authored-by: Janghoon Lee <44862514+savagemanage@users.noreply.github.com> --- packages/client/src/generated/types.ts | 2 + packages/codemode/src/codemode.ts | 15 +- packages/codemode/src/interpreter/runtime.ts | 131 ++++++++--- packages/codemode/src/tool-runtime.ts | 27 ++- packages/codemode/test/globals.test.ts | 83 +++++++ packages/core/src/skill.ts | 83 ++++++- packages/core/src/skill/arming.ts | 131 +++++++++++ packages/core/test/skill/arming.test.ts | 170 ++++++++++++++ packages/redrob/src/tool/code-mode.ts | 26 ++- packages/redrob/src/tool/domain.ts | 232 +++++++++++++++++++ packages/redrob/test/tool/code-mode.test.ts | 9 +- packages/redrob/test/tool/domain.test.ts | 185 +++++++++++++++ packages/schema/src/skill.ts | 18 ++ 13 files changed, 1060 insertions(+), 52 deletions(-) create mode 100644 packages/codemode/test/globals.test.ts create mode 100644 packages/core/src/skill/arming.ts create mode 100644 packages/core/test/skill/arming.test.ts create mode 100644 packages/redrob/src/tool/domain.ts create mode 100644 packages/redrob/test/tool/domain.test.ts diff --git a/packages/client/src/generated/types.ts b/packages/client/src/generated/types.ts index 5db37e3579..7cc6071d18 100644 --- a/packages/client/src/generated/types.ts +++ b/packages/client/src/generated/types.ts @@ -2609,6 +2609,8 @@ export type SkillsListOutput = { readonly name: string readonly description?: string readonly slash?: boolean + readonly icon?: string + readonly autoInject?: { readonly keywords?: ReadonlyArray; readonly url?: ReadonlyArray } readonly location: string readonly content: string }> diff --git a/packages/codemode/src/codemode.ts b/packages/codemode/src/codemode.ts index 14782d7c8a..1cd36e56c6 100644 --- a/packages/codemode/src/codemode.ts +++ b/packages/codemode/src/codemode.ts @@ -1,5 +1,5 @@ import { Effect, Schema } from "effect" -import { executeWithLimits } from "./interpreter/runtime.js" +import { assertValidGlobals, executeWithLimits } from "./interpreter/runtime.js" import { type HostTools, type Services, type ToolDescription, ToolRuntime } from "./tool-runtime.js" import type { Definition } from "./tool.js" @@ -38,6 +38,14 @@ export type ExecuteOptions = {}> = { code: string /** Explicit tool tree exposed to the program as `tools`. */ tools?: Tools & ToolTree> + /** + * Top-level tool namespaces ALSO bound as bare globals, so a skill document can be + * written as `await page.text()` rather than `await tools.page.text()`. Each name must + * be a namespace at the top level of `tools` and must not shadow a builtin global. + * A global is an alias for the same tool path: one implementation, one authorization + * point. Code Mode stays host-neutral - it never knows what `page` means. + */ + globals?: ReadonlyArray /** Per-execution overrides for the default resource limits. */ limits?: ExecutionLimits /** Observes decoded tool input immediately before tool execution. */ @@ -139,6 +147,7 @@ export const execute = >( ): Effect.Effect> => { const tools = (options.tools ?? {}) as HostTools> ToolRuntime.assertValidTools(tools) + assertValidGlobals(tools, options.globals ?? []) return executeWithLimits(options, resolveExecutionLimits(options.limits), ToolRuntime.searchIndex(tools)) } @@ -148,8 +157,10 @@ export const make = = {}>( ): Runtime> => { const tools = (options.tools ?? {}) as HostTools> ToolRuntime.assertValidTools(tools) + const globals = options.globals ?? [] + assertValidGlobals(tools, globals) const limits = resolveExecutionLimits(options.limits) - const prepared = ToolRuntime.prepare(tools, options.discovery?.catalogBudget) + const prepared = ToolRuntime.prepare(tools, options.discovery?.catalogBudget, globals) return { catalog: () => prepared.catalog, diff --git a/packages/codemode/src/interpreter/runtime.ts b/packages/codemode/src/interpreter/runtime.ts index 093f577765..573cdc493b 100644 --- a/packages/codemode/src/interpreter/runtime.ts +++ b/packages/codemode/src/interpreter/runtime.ts @@ -13,6 +13,8 @@ import { type Services, } from "../tool-runtime.js" import { ToolError } from "../tool-error.js" +import { isDefinition as isToolDefinition } from "../tool.js" +import { identifierSegment } from "../tool-schema.js" import type { DataValue, Diagnostic, @@ -599,6 +601,85 @@ const collectPatternNames = (pattern: AstNode, out: Array = []): Array => { + const globalScope = new Map() + globalScope.set("tools", { mutable: false, value: new ToolReference([]) }) + globalScope.set("Promise", { mutable: false, value: new PromiseNamespace() }) + globalScope.set("undefined", { mutable: false, value: undefined }) + globalScope.set("Object", { mutable: false, value: new GlobalNamespace("Object") }) + globalScope.set("Math", { mutable: false, value: new GlobalNamespace("Math") }) + globalScope.set("JSON", { mutable: false, value: new GlobalNamespace("JSON") }) + globalScope.set("Number", { mutable: false, value: new CoercionFunction("Number") }) + globalScope.set("String", { mutable: false, value: new CoercionFunction("String") }) + globalScope.set("Boolean", { mutable: false, value: new CoercionFunction("Boolean") }) + globalScope.set("Array", { mutable: false, value: new GlobalNamespace("Array") }) + globalScope.set("console", { mutable: false, value: new GlobalNamespace("console") }) + globalScope.set("parseInt", { mutable: false, value: new CoercionFunction("parseInt") }) + globalScope.set("parseFloat", { mutable: false, value: new CoercionFunction("parseFloat") }) + globalScope.set("Date", { mutable: false, value: new GlobalNamespace("Date") }) + globalScope.set("RegExp", { mutable: false, value: new GlobalNamespace("RegExp") }) + globalScope.set("Map", { mutable: false, value: new GlobalNamespace("Map") }) + globalScope.set("Set", { mutable: false, value: new GlobalNamespace("Set") }) + globalScope.set("URL", { mutable: false, value: new GlobalNamespace("URL") }) + globalScope.set("URLSearchParams", { mutable: false, value: new GlobalNamespace("URLSearchParams") }) + globalScope.set("encodeURI", { mutable: false, value: new UriFunction("encodeURI") }) + globalScope.set("encodeURIComponent", { mutable: false, value: new UriFunction("encodeURIComponent") }) + globalScope.set("decodeURI", { mutable: false, value: new UriFunction("decodeURI") }) + globalScope.set("decodeURIComponent", { mutable: false, value: new UriFunction("decodeURIComponent") }) + // Error constructors are real values, so `x instanceof Error` works and `Error("msg")` + // (with or without `new`) constructs a branded { name, message } error object. + for (const name of errorConstructors) { + globalScope.set(name, { mutable: false, value: new ErrorConstructorReference(name) }) + } + // NaN/Infinity flow as ordinary in-sandbox values (normalized to null only at the data + // boundary - see copyOut), so their global bindings must exist too, e.g. `reduce(max, -Infinity)`. + globalScope.set("NaN", { mutable: false, value: NaN }) + globalScope.set("Infinity", { mutable: false, value: Infinity }) + return globalScope +} + +/** + * Every name the interpreter seeds into the builtin global scope. A host global + * (see `ExecuteOptions.globals`) may not shadow one of these. + */ +export const BUILTIN_GLOBAL_NAMES: ReadonlySet = new Set(seedBuiltinGlobals().keys()) + +/** + * Validates the names a host asks to bind as bare globals. A global is an alias for a + * top-level namespace of the tool tree, so it must name one, must not shadow a builtin + * global, and must be a plain identifier (a bare binding cannot be written in bracket + * notation). A duplicate is refused rather than deduplicated: a repeated name in a host's + * list is a configuration mistake worth surfacing. + * + * Lives here, beside `seedBuiltinGlobals`, so the reserved-name check reads the same set + * the interpreter actually seeds. + */ +export const assertValidGlobals = (tools: HostTools, globals: ReadonlyArray): void => { + const seen = new Set() + for (const name of globals) { + if (!identifierSegment.test(name)) { + throw new Error(`Global '${name}' is not a plain identifier and cannot be bound as a global.`) + } + if (BUILTIN_GLOBAL_NAMES.has(name)) { + throw new Error(`Global '${name}' collides with a builtin global.`) + } + if (seen.has(name)) throw new Error(`Global '${name}' is listed more than once.`) + seen.add(name) + if (!Object.hasOwn(tools, name)) { + throw new Error(`Global '${name}' is not a top-level namespace of the tool tree.`) + } + const namespace = tools[name] + if (typeof namespace === "function" || isToolDefinition(namespace)) { + throw new Error(`Global '${name}' must be a namespace of tools, not a tool itself.`) + } + } +} + class Interpreter { private scopes: Array> private readonly invokeTool: (path: ReadonlyArray, args: Array) => Effect.Effect @@ -618,46 +699,28 @@ class Interpreter { invokeTool: (path: ReadonlyArray, args: Array) => Effect.Effect, toolKeys: (path: ReadonlyArray) => ReadonlyArray, logs: Array = [], + /** + * Host tool namespaces additionally bound at the top level, so a program writes + * `page.text()` instead of `tools.page.text()`. Each name resolves to the same tool + * path, so there is one implementation and one authorization point, not two. + */ + hostGlobals: ReadonlyArray = [], ) { - const globalScope = new Map() + const globalScope = seedBuiltinGlobals() this.scopes = [globalScope] this.invokeTool = invokeTool this.toolKeys = toolKeys this.logs = logs this.lastValue = undefined this.callPermits = Semaphore.makeUnsafe(TOOL_CALL_CONCURRENCY) - globalScope.set("tools", { mutable: false, value: new ToolReference([]) }) - globalScope.set("Promise", { mutable: false, value: new PromiseNamespace() }) - globalScope.set("undefined", { mutable: false, value: undefined }) - globalScope.set("Object", { mutable: false, value: new GlobalNamespace("Object") }) - globalScope.set("Math", { mutable: false, value: new GlobalNamespace("Math") }) - globalScope.set("JSON", { mutable: false, value: new GlobalNamespace("JSON") }) - globalScope.set("Number", { mutable: false, value: new CoercionFunction("Number") }) - globalScope.set("String", { mutable: false, value: new CoercionFunction("String") }) - globalScope.set("Boolean", { mutable: false, value: new CoercionFunction("Boolean") }) - globalScope.set("Array", { mutable: false, value: new GlobalNamespace("Array") }) - globalScope.set("console", { mutable: false, value: new GlobalNamespace("console") }) - globalScope.set("parseInt", { mutable: false, value: new CoercionFunction("parseInt") }) - globalScope.set("parseFloat", { mutable: false, value: new CoercionFunction("parseFloat") }) - globalScope.set("Date", { mutable: false, value: new GlobalNamespace("Date") }) - globalScope.set("RegExp", { mutable: false, value: new GlobalNamespace("RegExp") }) - globalScope.set("Map", { mutable: false, value: new GlobalNamespace("Map") }) - globalScope.set("Set", { mutable: false, value: new GlobalNamespace("Set") }) - globalScope.set("URL", { mutable: false, value: new GlobalNamespace("URL") }) - globalScope.set("URLSearchParams", { mutable: false, value: new GlobalNamespace("URLSearchParams") }) - globalScope.set("encodeURI", { mutable: false, value: new UriFunction("encodeURI") }) - globalScope.set("encodeURIComponent", { mutable: false, value: new UriFunction("encodeURIComponent") }) - globalScope.set("decodeURI", { mutable: false, value: new UriFunction("decodeURI") }) - globalScope.set("decodeURIComponent", { mutable: false, value: new UriFunction("decodeURIComponent") }) - // Error constructors are real values, so `x instanceof Error` works and `Error("msg")` - // (with or without `new`) constructs a branded { name, message } error object. - for (const name of errorConstructors) { - globalScope.set(name, { mutable: false, value: new ErrorConstructorReference(name) }) - } - // NaN/Infinity flow as ordinary in-sandbox values (normalized to null only at the data - // boundary - see copyOut), so their global bindings must exist too, e.g. `reduce(max, -Infinity)`. - globalScope.set("NaN", { mutable: false, value: NaN }) - globalScope.set("Infinity", { mutable: false, value: Infinity }) + for (const name of hostGlobals) { + // Validated by ToolRuntime.assertValidGlobals before construction; a collision + // here would silently replace a builtin, so refuse rather than overwrite. + if (BUILTIN_GLOBAL_NAMES.has(name)) { + throw new Error(`Host global '${name}' collides with a builtin global.`) + } + globalScope.set(name, { mutable: false, value: new ToolReference([name]) }) + } } run(program: ProgramNode): Effect.Effect { @@ -3365,7 +3428,7 @@ export const executeWithLimits = >( const operation = Effect.gen(function* () { const program = parseProgram(options.code) - const interpreter = new Interpreter>(tools.invoke, tools.keys, logs) + const interpreter = new Interpreter>(tools.invoke, tools.keys, logs, options.globals ?? []) const value = yield* interpreter.run(program) const result = copyOut(copyIn(value, "Execution result"), true) as DataValue return { diff --git a/packages/codemode/src/tool-runtime.ts b/packages/codemode/src/tool-runtime.ts index f4ccc61d4c..a5f70823a0 100644 --- a/packages/codemode/src/tool-runtime.ts +++ b/packages/codemode/src/tool-runtime.ts @@ -481,6 +481,24 @@ export const assertValidTools = (tools: HostTools): void => { } } +/** + * Renders the host-global aliases. Each listed namespace is reachable both as a bare + * identifier and under `tools`, so a skill document can open with `await page.text()` + * without the model having to be told that `page` is "really" a tool namespace. + */ +const globalsSection = (globals: ReadonlyArray): Array => { + if (globals.length === 0) return [] + const ordered = [...globals].sort((left, right) => left.localeCompare(right)) + return [ + "", + "## Domain globals", + "", + `These namespaces are also bound as bare globals: ${ordered.map((name) => `\`${name}\``).join(", ")}.`, + "`.(input)` and `tools..(input)` are the same call; prefer the bare form.", + "A global's tools are listed under its namespace in the catalog below, like any other tool.", + ] +} + /** * Budgeted catalog: every namespace is always listed with its tool count; full call * signatures are inlined against the `catalogBudget` (estimated tokens, @@ -492,7 +510,12 @@ export const assertValidTools = (tools: HostTools): void => { * namespace. Namespace stub lines are never budgeted: every namespace appears with its * tool count even at budget 0. */ -export const prepare = (tools: HostTools, catalogBudget = defaultCatalogBudget): DiscoveryPlan => { +export const prepare = ( + tools: HostTools, + catalogBudget = defaultCatalogBudget, + /** Namespaces also bound as bare globals; rendered so the model knows it may write `page.text()`. */ + globals: ReadonlyArray = [], +): DiscoveryPlan => { if (!Number.isSafeInteger(catalogBudget) || catalogBudget < 0) { throw new RangeError("discovery.catalogBudget must be a non-negative safe integer") } @@ -639,7 +662,7 @@ export const prepare = (tools: HostTools, catalogBudget = defaultCatalogBu } } - const lines = [...intro, ...workflow, ...rules, ...language, ...toolSection] + const lines = [...intro, ...workflow, ...rules, ...globalsSection(globals), ...language, ...toolSection] return { catalog: described, instructions: lines.join("\n"), diff --git a/packages/codemode/test/globals.test.ts b/packages/codemode/test/globals.test.ts new file mode 100644 index 0000000000..c27d9bc1a6 --- /dev/null +++ b/packages/codemode/test/globals.test.ts @@ -0,0 +1,83 @@ +import { describe, expect, test } from "bun:test" +import { Effect, Schema } from "effect" +import { CodeMode, Tool } from "../src/index.js" + +const text = Tool.make({ + description: "Visible text of the page", + input: Schema.Struct({}), + output: Schema.String, + run: () => Effect.succeed("hello from the page"), +}) + +const click = Tool.make({ + description: "Click the first node matching a selector", + input: Schema.Struct({ selector: Schema.String }), + output: Schema.Boolean, + run: (input) => Effect.succeed(input.selector === "button"), +}) + +const page = { text, click } + +const run = (code: string, globals: ReadonlyArray = ["page"]) => + Effect.runPromise(CodeMode.make({ tools: { page }, globals }).execute(code)) + +describe("CodeMode host globals", () => { + test("a bare global resolves to its tool namespace", async () => { + const result = await run("return await page.text({})") + expect(result).toMatchObject({ ok: true, value: "hello from the page" }) + }) + + test("the bare form and the tools form are the same call", async () => { + const result = await run( + 'return [await page.click({ selector: "button" }), await tools.page.click({ selector: "a" })]', + ) + expect(result).toMatchObject({ ok: true, value: [true, false] }) + expect(result.toolCalls.map((call) => call.name)).toStrictEqual(["page.click", "page.click"]) + }) + + test("a global is enumerable like any namespace", async () => { + const result = await run("return Object.keys(page).toSorted()") + expect(result).toMatchObject({ ok: true, value: ["click", "text"] }) + }) + + test("a top-level declaration still shadows a global, as in a JS module", async () => { + const result = await run('const page = "shadowed"; return page') + expect(result).toMatchObject({ ok: true, value: "shadowed" }) + }) + + test("an unlisted namespace is not bound as a global", async () => { + const result = await Effect.runPromise(CodeMode.make({ tools: { page } }).execute("return await page.text({})")) + expect(result.ok).toBe(false) + expect(result.ok ? "" : result.error.message).toContain("page") + }) + + test("the instructions name the globals and the equivalence", async () => { + const instructions = CodeMode.make({ tools: { page }, globals: ["page"] }).instructions() + expect(instructions).toContain("## Domain globals") + expect(instructions).toContain("`page`") + expect(instructions).toContain("`tools..(input)` are the same call") + }) + + test("no globals section is rendered when no globals are declared", () => { + expect(CodeMode.make({ tools: { page } }).instructions()).not.toContain("## Domain globals") + }) + + describe("refuses a global a host cannot honestly bind", () => { + const cases: ReadonlyArray, string]> = [ + ["a name that is not in the tool tree", ["screen"], "not a top-level namespace"], + ["a builtin global", ["Math"], "collides with a builtin global"], + ["the tools binding itself", ["tools"], "collides with a builtin global"], + ["a duplicate", ["page", "page"], "listed more than once"], + ["a non-identifier", ["page-object"], "not a plain identifier"], + ] + for (const [name, globals, message] of cases) { + test(name, () => { + expect(() => CodeMode.make({ tools: { page }, globals })).toThrow(message) + }) + } + + test("a tool rather than a namespace", () => { + expect(() => CodeMode.make({ tools: { text }, globals: ["text"] })).toThrow("not a tool itself") + }) + }) +}) diff --git a/packages/core/src/skill.ts b/packages/core/src/skill.ts index a0ffeae896..8c82d56a46 100644 --- a/packages/core/src/skill.ts +++ b/packages/core/src/skill.ts @@ -2,7 +2,7 @@ export * as SkillV2 from "./skill" import { makeLocationNode } from "./effect/app-node" import path from "path" -import { Context, Effect, Layer, Schema, Types } from "effect" +import { Context, Effect, Layer, Result, Schema, Types } from "effect" import { Skill } from "@redrob-code/schema/skill" import { AgentV2 } from "./agent" import { ConfigMarkdown } from "./config/markdown" @@ -27,6 +27,9 @@ export type Source = typeof Source.Type export const Info = Skill.Info export type Info = Skill.Info +export const AutoInject = Skill.AutoInject +export type AutoInject = Skill.AutoInject + export const available = (skills: ReadonlyArray, agent: AgentV2.Info) => skills.filter((skill) => PermissionV2.evaluate("skill", skill.name, agent.permissions).effect !== "deny") @@ -34,8 +37,25 @@ const Frontmatter = Schema.Struct({ name: Schema.String.pipe(Schema.optional), description: Schema.String.pipe(Schema.optional), slash: Schema.Boolean.pipe(Schema.optional), + icon: Schema.String.pipe(Schema.optional), }) -const decodeFrontmatter = Schema.decodeUnknownOption(Frontmatter) +/** + * `decodeUnknownResult`, not `decodeUnknownOption`: the Option form answers only "no", which + * makes a malformed document unfindable among the ones that loaded. The Result form carries + * the issue, so the log below can name the file AND why it was rejected. + */ +const decodeFrontmatter = Schema.decodeUnknownResult(Frontmatter) + +/** + * K-2's auto-arm block is decoded SEPARATELY from the rest of the frontmatter, so a + * malformed `autoInject` costs only the arming behaviour: the skill still loads and stays + * explicitly loadable, instead of the whole document being dropped for a typo in an + * optional block. + */ +const decodeAutoInject = Schema.decodeUnknownResult(Skill.AutoInject) + +/** The reason one document did not become a skill, kept so the log can say why. */ +type Drop = { readonly file: string; readonly reason: string } export type Data = { sources: Types.DeepMutable[] @@ -74,33 +94,82 @@ const layer = Layer.effect( const skills: Info[] = [] if (source.type === "embedded") return [source.skill] const directories = source.type === "directory" ? [source.path] : yield* discovery.pull(source.url) + // Decoded ONE DOCUMENT AT A TIME: a malformed frontmatter costs that document and not + // the set, and every rejection is recorded with its reason so the cause is findable. + const drops: Drop[] = [] + let ignoredAutoInject = 0 for (const directory of directories) { const files = yield* fs .glob("{*.md,**/SKILL.md}", { cwd: directory, absolute: true, include: "file", symlink: true, dot: true }) .pipe(Effect.catch(() => Effect.succeed([] as string[]))) for (const filepath of files.toSorted()) { const content = yield* fs.readFileStringSafe(filepath).pipe(Effect.catch(() => Effect.succeed(undefined))) - if (!content) continue + if (!content) { + drops.push({ file: filepath, reason: "unreadable: the file could not be read" }) + continue + } const markdown = ConfigMarkdown.parseOption(content) - if (!markdown) continue - const frontmatter = decodeFrontmatter(markdown.data).valueOrUndefined - if (!frontmatter) continue + if (!markdown) { + drops.push({ file: filepath, reason: "unparsable: no valid markdown frontmatter block" }) + continue + } + const decoded = decodeFrontmatter(markdown.data) + if (Result.isFailure(decoded)) { + drops.push({ file: filepath, reason: `frontmatter rejected: ${decoded.failure.message}` }) + continue + } + const frontmatter = decoded.success const name = frontmatter.name !== undefined ? frontmatter.name : path.dirname(filepath) === directory ? path.basename(filepath, ".md") : undefined - if (!name) continue + if (!name) { + drops.push({ + file: filepath, + reason: "unnamed: no `name` in frontmatter and the path gives no fallback name", + }) + continue + } + // A malformed `autoInject` is IGNORED, not fatal: the skill loads and stays + // explicitly loadable, which is what a skill without the block does anyway. + let autoInject: AutoInject | undefined + const declared = (markdown.data as Record | undefined)?.["autoInject"] + if (declared !== undefined) { + const block = decodeAutoInject(declared) + if (Result.isFailure(block)) { + ignoredAutoInject += 1 + // The reason goes in the MESSAGE, not only in an annotation: the default logger + // renders an annotation object as `[object Object]`, which loses exactly the + // detail that makes this findable. + yield* Effect.logWarning( + `SkillV2.load ignored a malformed autoInject block in ${filepath} (skill "${name}"): ${block.failure.message}`, + ) + } else autoInject = block.success + } skills.push({ name, description: frontmatter.description, slash: frontmatter.slash, + icon: frontmatter.icon, + autoInject, location: AbsolutePath.make(filepath), content: markdown.content, }) } } + if (drops.length > 0) { + // Warning, not debug: a dropped skill is silently missing behaviour, so the count and + // every individual reason have to reach a default log level to be findable at all. + // Both are in the message text for the same reason the per-block warning above is. + const total = skills.length + drops.length + yield* Effect.logWarning( + `SkillV2.load dropped ${drops.length} of ${total} skill documents from ${Source.key(source)} ` + + `(loaded ${skills.length}, ignoredAutoInject ${ignoredAutoInject}): ` + + drops.map((drop) => `${drop.file}: ${drop.reason}`).join("; "), + ) + } return skills }) diff --git a/packages/core/src/skill/arming.ts b/packages/core/src/skill/arming.ts new file mode 100644 index 0000000000..ca5e7a59e9 --- /dev/null +++ b/packages/core/src/skill/arming.ts @@ -0,0 +1,131 @@ +export * as SkillArming from "./arming" + +import type { Skill } from "@redrob-code/schema/skill" + +/** + * K-2: the arming decision, as a pure function. + * + * Given the loaded skills, the current prompt text and the current tab URL, `arm` returns + * which skills auto-arm. It reads nothing, caches nothing and logs nothing, so the decision + * is testable on its own and identical wherever it is asked: the session prompt builder, a + * UI preview, or a test. + * + * Two rules, both case-insensitive: + * + * - keyword: a keyword matches when it appears in the prompt on word boundaries, so `ai` + * does not match `said` but `docs.google.com` still matches when followed by a slash. + * - url: a glob matches the current tab URL in full. `*` and `?` stop at `/`; `**` crosses + * path segments. The scheme is ignored on both sides, so a glob is written + * `docs.google.com/document/**` rather than `https://docs.google.com/document/**`. + * + * A skill with no `autoInject` block NEVER auto-arms. It stays explicitly loadable, which + * is the point: a skill that arms on everything is a skill that is always in the prompt. + */ + +/** The shape `arm` needs from a skill. Narrower than `Skill.Info` so callers can test it directly. */ +export type Armable = { + readonly name: string + readonly autoInject?: Skill.AutoInject | undefined +} + +/** One skill that armed, with the patterns that armed it. */ +export type Armed = { + readonly name: string + /** Keywords from the skill's block that matched the prompt. */ + readonly keywords: ReadonlyArray + /** URL globs from the skill's block that matched the current tab URL. */ + readonly urls: ReadonlyArray +} + +export type Input = { + readonly skills: ReadonlyArray + /** The current prompt text. An empty prompt matches no keyword. */ + readonly prompt: string + /** The current tab URL, when the session is attached to one. Absent matches no glob. */ + readonly url?: string | undefined +} + +const isWordCharacter = (character: string | undefined): boolean => + character !== undefined && /[\p{L}\p{N}_]/u.test(character) + +/** + * Case-insensitive word-boundary containment. A boundary is required only where the + * keyword's own edge is a word character, so a keyword like `.pdf` or `docs.google.com` + * matches in running text while `ai` does not match inside `said`. + */ +const matchesKeyword = (prompt: string, keyword: string): boolean => { + const needle = keyword.trim().toLowerCase() + if (needle.length === 0) return false + const haystack = prompt.toLowerCase() + const checkStart = isWordCharacter(needle[0]) + const checkEnd = isWordCharacter(needle[needle.length - 1]) + let from = 0 + for (;;) { + const at = haystack.indexOf(needle, from) + if (at === -1) return false + const before = at === 0 ? undefined : haystack[at - 1] + const after = haystack[at + needle.length] + if ((!checkStart || !isWordCharacter(before)) && (!checkEnd || !isWordCharacter(after))) return true + from = at + 1 + } +} + +/** Drops a `scheme://` prefix and a trailing slash, so globs are written without a scheme. */ +const normalizeUrl = (value: string): string => + value + .trim() + .toLowerCase() + .replace(/^[a-z][a-z0-9+.-]*:\/\//, "") + .replace(/\/+$/, "") + +/** + * Compiles a glob to an anchored regular expression. `**` crosses `/`; `*` and `?` do not. + * Every other character is matched literally, so a `.` in a hostname is a dot and not a + * wildcard - the common authoring mistake if globs were treated as regular expressions. + */ +const globToRegExp = (glob: string): RegExp => { + let pattern = "" + for (let index = 0; index < glob.length; index += 1) { + const character = glob[index]! + if (character === "*") { + if (glob[index + 1] === "*") { + pattern += ".*" + index += 1 + continue + } + pattern += "[^/]*" + continue + } + if (character === "?") { + pattern += "[^/]" + continue + } + pattern += character.replace(/[.*+?^${}()|[\]\\]/g, "\\$&") + } + return new RegExp(`^${pattern}$`) +} + +const matchesUrlGlob = (url: string, glob: string): boolean => { + const normalized = normalizeUrl(glob) + if (normalized.length === 0) return false + return globToRegExp(normalized).test(normalizeUrl(url)) +} + +/** Returns the skills that auto-arm, in the order they were given. */ +export const arm = (input: Input): ReadonlyArray => { + const armed: Array = [] + for (const skill of input.skills) { + const block = skill.autoInject + // No block means never auto-arm. This is the explicit-only case, not a default-on one. + if (block === undefined) continue + const keywords = (block.keywords ?? []).filter((keyword) => matchesKeyword(input.prompt, keyword)) + const urls = + input.url === undefined ? [] : (block.url ?? []).filter((glob) => matchesUrlGlob(input.url as string, glob)) + if (keywords.length === 0 && urls.length === 0) continue + armed.push({ name: skill.name, keywords, urls }) + } + return armed +} + +/** Just the names that armed, for callers that only need the set. */ +export const armedNames = (input: Input): ReadonlyArray => arm(input).map((entry) => entry.name) diff --git a/packages/core/test/skill/arming.test.ts b/packages/core/test/skill/arming.test.ts new file mode 100644 index 0000000000..00dc901fd1 --- /dev/null +++ b/packages/core/test/skill/arming.test.ts @@ -0,0 +1,170 @@ +import fs from "fs/promises" +import path from "path" +import { describe, expect, test } from "bun:test" +import { Effect, Layer } from "effect" +import * as TestConsole from "effect/testing/TestConsole" +import { AgentV2 } from "@redrob-code/core/agent" +import { AppNodeBuilder } from "@redrob-code/core/effect/app-node-builder" +import { LayerNode } from "@redrob-code/core/effect/layer-node" +import { AbsolutePath } from "@redrob-code/core/schema" +import { SkillV2 } from "@redrob-code/core/skill" +import { SkillArming } from "@redrob-code/core/skill/arming" +import { SkillDiscovery } from "@redrob-code/core/skill/discovery" +import { tmpdir } from "../fixture/tmpdir" +import { testEffect } from "../lib/effect" + +/** `arm` is pure, so most of K-2 needs no layer, no filesystem and no clock. */ +const skill = (name: string, autoInject?: SkillV2.AutoInject): SkillArming.Armable => ({ name, autoInject }) + +const names = (input: SkillArming.Input) => SkillArming.armedNames(input) + +describe("SkillArming.arm", () => { + const docs = skill("google-docs", { url: ["docs.google.com/document/**"] }) + const deploy = skill("deploy", { keywords: ["deploy", "rollback"] }) + /** No `autoInject` block at all: explicitly loadable, never auto-armed. */ + const manual = skill("manual") + + test("arms on a keyword hit", () => { + expect(names({ skills: [deploy, manual], prompt: "can you deploy this branch" })).toEqual(["deploy"]) + }) + + test("reports which keyword armed the skill", () => { + expect(SkillArming.arm({ skills: [deploy], prompt: "time to ROLLBACK" })).toEqual([ + { name: "deploy", keywords: ["rollback"], urls: [] }, + ]) + }) + + test("keyword matching is case-insensitive in both directions", () => { + expect(names({ skills: [skill("x", { keywords: ["RollBack"] })], prompt: "rollback now" })).toEqual(["x"]) + }) + + test("a keyword does not match inside a longer word", () => { + expect(names({ skills: [skill("ai", { keywords: ["ai"] })], prompt: "she said nothing" })).toEqual([]) + }) + + test("arms on a URL-glob hit", () => { + expect( + names({ + skills: [docs, manual], + prompt: "summarise this", + url: "https://docs.google.com/document/d/abc123/edit", + }), + ).toEqual(["google-docs"]) + }) + + test("URL matching ignores the scheme and is case-insensitive", () => { + expect(names({ skills: [docs], prompt: "", url: "HTTPS://Docs.Google.com/Document/d/ABC" })).toEqual([ + "google-docs", + ]) + }) + + test("a glob NEAR-MISS does not arm", () => { + // `documents` is not `document`, and a single `*` does not cross a `/`. + expect(names({ skills: [docs], prompt: "", url: "https://docs.google.com/documents/d/abc" })).toEqual([]) + expect( + names({ + skills: [skill("one-segment", { url: ["docs.google.com/document/*"] })], + prompt: "", + url: "https://docs.google.com/document/d/abc", + }), + ).toEqual([]) + // A `.` in the glob is a literal dot, not a regex wildcard. + expect(names({ skills: [docs], prompt: "", url: "https://docsXgoogle.com/document/d/abc" })).toEqual([]) + }) + + test("a skill with no autoInject block NEVER arms", () => { + expect( + names({ skills: [manual], prompt: "manual deploy rollback everything", url: "docs.google.com/document/d/abc" }), + ).toEqual([]) + }) + + test("an empty autoInject block matches nothing", () => { + expect(names({ skills: [skill("empty", {})], prompt: "deploy", url: "docs.google.com/document/d/a" })).toEqual([]) + }) + + test("no current URL means no glob can arm", () => { + expect(names({ skills: [docs], prompt: "summarise this" })).toEqual([]) + }) + + test("keyword and URL hits are both reported on one skill", () => { + expect( + SkillArming.arm({ + skills: [skill("both", { keywords: ["summarise"], url: ["docs.google.com/**"] })], + prompt: "Summarise it", + url: "https://docs.google.com/document/d/a", + }), + ).toEqual([{ name: "both", keywords: ["summarise"], urls: ["docs.google.com/**"] }]) + }) + + test("skills are returned in the order given", () => { + expect(names({ skills: [deploy, manual, skill("also", { keywords: ["deploy"] })], prompt: "deploy" })).toEqual([ + "deploy", + "also", + ]) + }) +}) + +const urls = new Map() +const discovery = Layer.succeed( + SkillDiscovery.Service, + SkillDiscovery.Service.of({ pull: (url) => Effect.succeed(urls.get(url) ?? []) }), +) +const it = testEffect( + AppNodeBuilder.build(LayerNode.group([SkillV2.node, AgentV2.node]), [[SkillDiscovery.node, discovery]]), +) + +describe("SkillV2.load frontmatter", () => { + it.live("ignores a malformed autoInject instead of failing the load, and drops only the bad document", () => + Effect.acquireRelease( + Effect.promise(() => tmpdir()), + (tmp) => Effect.promise(() => tmp[Symbol.asyncDispose]()), + ).pipe( + Effect.flatMap((tmp) => + Effect.gen(function* () { + yield* Effect.promise(async () => { + // `autoInject` is a string where the schema wants an object: ignored, not fatal. + await fs.writeFile( + path.join(tmp.path, "broken-block.md"), + "---\nname: broken-block\nautoInject: nonsense\n---\n# broken-block", + ) + // A good neighbour, to prove one bad document does not cost the set. + await fs.writeFile( + path.join(tmp.path, "good.md"), + "---\nname: good\nicon: https://example.test/i.png\nautoInject:\n keywords: [deploy]\n url: [docs.google.com/document/**]\n---\n# good", + ) + // `slash` must be a boolean: this document IS dropped, with a logged reason. + await fs.writeFile(path.join(tmp.path, "bad-type.md"), "---\nname: bad-type\nslash: yes please\n---\n# bad") + }) + + const skills = yield* SkillV2.Service + yield* skills.transform((editor) => editor.source({ type: "directory", path: AbsolutePath.make(tmp.path) })) + const loaded = yield* skills.list() + + // The undecodable document is gone; the two others survived. + expect(loaded.map((item) => item.name).toSorted()).toEqual(["broken-block", "good"]) + + const broken = loaded.find((item) => item.name === "broken-block")! + expect(broken.autoInject).toBeUndefined() + // Still loadable, just never auto-armed. + expect(names({ skills: [broken], prompt: "nonsense" })).toEqual([]) + + const good = loaded.find((item) => item.name === "good")! + expect(good.icon).toBe("https://example.test/i.png") + expect(good.autoInject).toEqual({ keywords: ["deploy"], url: ["docs.google.com/document/**"] }) + expect(names({ skills: [good], prompt: "please deploy" })).toEqual(["good"]) + expect(names({ skills: [good], prompt: "", url: "https://docs.google.com/document/d/x" })).toEqual(["good"]) + + // The drop must be FINDABLE: the count, the offending file and the reason all logged. + const logged = (yield* TestConsole.logLines).map((line) => String(line)).join("\n") + expect(logged).toContain("SkillV2.load dropped 1 of 3 skill documents") + expect(logged).toContain("bad-type.md") + expect(logged).toContain("frontmatter rejected") + expect(logged).toContain("ignored a malformed autoInject block") + expect(logged).toContain("broken-block.md") + // The documents that loaded are not reported as dropped. + expect(logged).not.toContain("good.md:") + }), + ), + ), + ) +}) diff --git a/packages/redrob/src/tool/code-mode.ts b/packages/redrob/src/tool/code-mode.ts index 98df376495..162a45b9c9 100644 --- a/packages/redrob/src/tool/code-mode.ts +++ b/packages/redrob/src/tool/code-mode.ts @@ -8,6 +8,7 @@ import { Agent } from "@/agent/agent" import { Session } from "@/session/session" import { Permission } from "@/permission" import { Plugin } from "@/plugin" +import { DOMAIN_GLOBALS, domainTools, sessionChannel, unavailableChannel, unavailablePage } from "./domain" export const CODE_MODE_TOOL = "execute" @@ -57,10 +58,16 @@ function groupByServer(mcpTools: Record, servers: readonly export function describeCatalog(mcpTools: Record, servers: readonly string[]): string { return CodeMode.make({ - tools: toolTree( - [...groupByServer(mcpTools, servers).values()].flat(), - () => () => Effect.fail(toolError("Tool preview is not executable.")), - ), + tools: { + ...toolTree( + [...groupByServer(mcpTools, servers).values()].flat(), + () => () => Effect.fail(toolError("Tool preview is not executable.")), + ), + // The preview must list the same domain globals a live execution binds, so the + // description a model reads matches the scope it will actually run in. + ...domainTools({ page: unavailablePage(), channel: unavailableChannel() }), + }, + globals: DOMAIN_GLOBALS, }).instructions() } @@ -213,6 +220,11 @@ export const CodeModeTool = Tool.define( const calls: CallEntry[] = [] const attachments: Attachment[] = [] + // K-1 domain objects for this execution. `channel` posts into the session that owns + // the run; `page` has no working implementation in the engine process yet and + // refuses by name rather than returning a plausible value. + const page = unavailablePage() + const channel = sessionChannel({ sessions, sessionID: ctx.sessionID, messageID: ctx.messageID }) const publish = () => ctx.metadata({ title: CODE_MODE_TOOL, metadata: { toolCalls: calls.map((c) => ({ ...c })) } }) @@ -237,7 +249,11 @@ export const CodeModeTool = Tool.define( ) const runtime = CodeMode.make({ - tools: toolTree(catalog, callTool), + // Domain namespaces are spread last deliberately: a connected MCP server named + // `page` must not shadow the typed domain object a skill document is written + // against. The domain objects are part of the language; the MCP catalog is not. + tools: { ...toolTree(catalog, callTool), ...domainTools({ page, channel }) }, + globals: DOMAIN_GLOBALS, onToolCallStart: ({ index, name, input }) => Effect.suspend(() => { const shown = (() => { diff --git a/packages/redrob/src/tool/domain.ts b/packages/redrob/src/tool/domain.ts new file mode 100644 index 0000000000..165a9aa9a8 --- /dev/null +++ b/packages/redrob/src/tool/domain.ts @@ -0,0 +1,232 @@ +/** + * K-1: the typed domain objects a skill document is written against. + * + * A skill is cheap only because these objects exist. Each one is a real TypeScript + * interface with a single implementation, exported from this module and injected into the + * Code Mode interpreter scope under its own name, so a skill body reads + * + * const body = await page.text() + * await channel.send({ text: body.slice(0, 200) }) + * + * rather than describing a tool call. Code Mode itself stays host-neutral: it is handed a + * namespace of tools plus the list of names to bind as globals, and never learns what + * `page` or `channel` mean (see packages/codemode/AGENTS.md). + * + * `page` has no implementation that can act today: nothing in the engine process controls + * a browser page. Its single implementation is therefore `unavailablePage`, which THROWS a + * message naming what is missing rather than returning a plausible value, so a skill + * written against the interface fails loudly instead of silently reading an empty string. + */ +import { SessionV1 } from "@redrob-code/core/v1/session" +import { Effect, Schema } from "effect" +import { Tool as SandboxTool, toolError } from "@redrob-code/codemode" +import { PartID, type MessageID, type SessionID } from "../session/schema" +import type { Session } from "@/session/session" + +/** One node returned by a page query. Kept to what a skill can act on without a handle. */ +export interface PageNode { + readonly selector: string + readonly text: string + readonly attributes: Readonly> +} + +/** + * A domain object the running engine cannot reach. Carries the missing capability by name + * so the model, the logs, and a person reading a transcript all see the same reason. + */ +export class DomainUnavailableError extends Schema.TaggedErrorClass()( + "CodeModeDomainUnavailable", + { object: Schema.String, missing: Schema.String }, +) { + override get message() { + return `The \`${this.object}\` domain object is not available in this session: ${this.missing}` + } +} + +/** + * The browser page the agent is acting in. + * + * Every method can fail with `DomainUnavailableError`, because whether a page exists is a + * property of the session, not of the call. + */ +export interface Page { + /** The page's current URL. */ + readonly url: () => Effect.Effect + /** The page's visible text. */ + readonly text: () => Effect.Effect + /** Nodes matching a CSS selector, in document order. */ + readonly query: (input: { + readonly selector: string + readonly limit?: number + }) => Effect.Effect, DomainUnavailableError> + /** Clicks the first node matching a selector. */ + readonly click: (input: { readonly selector: string }) => Effect.Effect + /** Types text into the first node matching a selector, optionally submitting afterwards. */ + readonly type: (input: { + readonly selector: string + readonly text: string + readonly submit?: boolean + }) => Effect.Effect + /** Navigates the page and returns the URL actually landed on. */ + readonly navigate: (input: { readonly url: string }) => Effect.Effect +} + +/** The channel this session belongs to: where a message or a result is delivered. */ +export interface Channel { + /** Identifier of the conversation this session posts into. */ + readonly id: () => Effect.Effect + /** Posts a message visible to the person in the conversation. Returns the part id written. */ + readonly send: (input: { readonly text: string }) => Effect.Effect +} + +/** + * The single `Page` implementation available in the engine process: none of it works. + * + * Kept deliberately rather than omitted, so the interface, the tool schemas, the generated + * instructions, and the skills written against them all exist and are exercised before the + * browser side lands. `missing` names the capability, not the symptom. + */ +export const unavailablePage = ( + missing = "the engine process has no browser-page control surface; the browser must expose page control to the engine first", +): Page => { + const refuse = () => + Effect.fail(new DomainUnavailableError({ object: "page", missing })) as Effect.Effect + return { + url: () => refuse(), + text: () => refuse(), + query: () => refuse>(), + click: () => refuse(), + type: () => refuse(), + navigate: () => refuse(), + } +} + +/** + * The single `Channel` implementation: posts into the session the program is running in, + * by appending a text part to the assistant message that owns the execution. Every surface + * the session is attached to renders that part, so this is the channel in the sense the + * skill means it, without the engine having to know which surface is attached. + */ +export const sessionChannel = (input: { + readonly sessions: Session.Interface + readonly sessionID: SessionID + readonly messageID: MessageID +}): Channel => ({ + id: () => Effect.succeed(input.sessionID), + send: ({ text }) => + Effect.gen(function* () { + const part = yield* input.sessions.updatePart({ + id: PartID.ascending(), + messageID: input.messageID, + sessionID: input.sessionID, + type: "text", + text, + } satisfies SessionV1.TextPart) + return part.id + }), +}) + +/** + * A `Channel` that refuses. Used where the catalog is described rather than executed, so + * the preview instructions list the same globals a real execution binds without a preview + * being able to post anything into a conversation. + */ +export const unavailableChannel = (missing = "this is a catalog preview, not a live execution"): Channel => { + const refuse = () => + Effect.fail(new DomainUnavailableError({ object: "channel", missing })) as Effect.Effect + return { id: () => refuse(), send: () => refuse() } +} + +const Empty = Schema.Struct({}) + +/** Narrow JSON projection of a page node, so results cross the sandbox data boundary. */ +const PageNodeSchema = Schema.Struct({ + selector: Schema.String, + text: Schema.String, + attributes: Schema.Record(Schema.String, Schema.String), +}) + +/** + * Lifts a domain call into a Code Mode tool. A `DomainUnavailableError` becomes a + * model-safe tool failure carrying the same sentence, so an unavailable object reads as a + * precise refusal in the program rather than as an opaque execution failure. + */ +const lift = + (call: (input: I) => Effect.Effect) => + (input: I) => + call(input).pipe(Effect.mapError((error) => toolError(error.message, error))) + +/** `page` as a Code Mode namespace. The tool names are the interface's method names. */ +export const pageTools = (page: Page) => ({ + url: SandboxTool.make({ + description: "Current URL of the page the agent is acting in.", + input: Empty, + output: Schema.String, + run: lift(() => page.url()), + }), + text: SandboxTool.make({ + description: "Visible text of the current page.", + input: Empty, + output: Schema.String, + run: lift(() => page.text()), + }), + query: SandboxTool.make({ + description: "Nodes matching a CSS selector, in document order.", + input: Schema.Struct({ + selector: Schema.String.annotate({ description: "CSS selector." }), + limit: Schema.optionalKey(Schema.Number.annotate({ description: "Maximum nodes to return." })), + }), + output: Schema.Array(PageNodeSchema), + run: lift((input: { selector: string; limit?: number }) => page.query(input)), + }), + click: SandboxTool.make({ + description: "Click the first node matching a CSS selector.", + input: Schema.Struct({ selector: Schema.String.annotate({ description: "CSS selector." }) }), + output: Schema.Null, + run: lift((input: { selector: string }) => page.click(input).pipe(Effect.as(null))), + }), + type: SandboxTool.make({ + description: "Type text into the first node matching a CSS selector.", + input: Schema.Struct({ + selector: Schema.String.annotate({ description: "CSS selector." }), + text: Schema.String.annotate({ description: "Text to type." }), + submit: Schema.optionalKey(Schema.Boolean.annotate({ description: "Submit the field afterwards." })), + }), + output: Schema.Null, + run: lift((input: { selector: string; text: string; submit?: boolean }) => page.type(input).pipe(Effect.as(null))), + }), + navigate: SandboxTool.make({ + description: "Navigate the page and return the URL landed on.", + input: Schema.Struct({ url: Schema.String.annotate({ description: "Absolute URL to open." }) }), + output: Schema.String, + run: lift((input: { url: string }) => page.navigate(input)), + }), +}) + +/** `channel` as a Code Mode namespace. */ +export const channelTools = (channel: Channel) => ({ + id: SandboxTool.make({ + description: "Identifier of the conversation this session posts into.", + input: Empty, + output: Schema.String, + run: lift(() => channel.id()), + }), + send: SandboxTool.make({ + description: "Post a message into the conversation this session belongs to.", + input: Schema.Struct({ text: Schema.String.annotate({ description: "Message text." }) }), + output: Schema.String, + run: lift((input: { text: string }) => channel.send(input)), + }), +}) + +/** The names bound as bare globals in the interpreter scope, in the order they are declared. */ +export const DOMAIN_GLOBALS = ["channel", "page"] as const + +/** + * The domain namespaces for one execution, keyed by the global name each is bound to. + * Returned as one object so the tool tree and `DOMAIN_GLOBALS` cannot drift apart. + */ +export const domainTools = (input: { readonly page: Page; readonly channel: Channel }) => ({ + channel: channelTools(input.channel), + page: pageTools(input.page), +}) diff --git a/packages/redrob/test/tool/code-mode.test.ts b/packages/redrob/test/tool/code-mode.test.ts index 16cc08bbec..053cad8cc3 100644 --- a/packages/redrob/test/tool/code-mode.test.ts +++ b/packages/redrob/test/tool/code-mode.test.ts @@ -256,7 +256,7 @@ describe("code mode execute", () => { expect(output.metadata.toolCalls).toEqual([]) }) - test("Object.keys(tools) enumerates the MCP server and CodeMode namespaces", async () => { + test("Object.keys(tools) enumerates the MCP server, domain, and CodeMode namespaces", async () => { const tool = await build({ github_list_issues: mcpTool("list_issues", () => ""), linear_search: mcpTool("search", () => ""), @@ -267,7 +267,12 @@ describe("code mode execute", () => { ctx, ), ) - expect(JSON.parse(output.output)).toEqual({ namespaces: ["github", "linear", "$codemode"], count: 3 }) + // The K-1 domain namespaces (`channel`, `page`) are part of the language, so they are + // always present alongside whatever MCP servers are connected. + expect(JSON.parse(output.output)).toEqual({ + namespaces: ["github", "linear", "channel", "page", "$codemode"], + count: 5, + }) }) test("calls a namespaced MCP tool and flows its text result back into the program", async () => { diff --git a/packages/redrob/test/tool/domain.test.ts b/packages/redrob/test/tool/domain.test.ts new file mode 100644 index 0000000000..9315ac2864 --- /dev/null +++ b/packages/redrob/test/tool/domain.test.ts @@ -0,0 +1,185 @@ +import { describe, expect, test } from "bun:test" +import { Agent } from "@/agent/agent" +import { MCP } from "@/mcp" +import { Plugin } from "@/plugin" +import { Session } from "@/session/session" +import { Tool } from "@/tool/tool" +import * as Truncate from "@/tool/truncate" +import { CODE_MODE_TOOL, CodeModeTool, describeCatalog } from "@/tool/code-mode" +import { + DOMAIN_GLOBALS, + DomainUnavailableError, + channelTools, + domainTools, + pageTools, + sessionChannel, + unavailableChannel, + unavailablePage, +} from "@/tool/domain" +import { MessageID, PartID, SessionID } from "@/session/schema" +import type { SessionV1 } from "@redrob-code/core/v1/session" +import { Cause, Effect, Exit, Layer } from "effect" + +const sessionID = SessionID.make("ses_domain") +const messageID = MessageID.make("msg_domain") + +const ctx: Tool.Context = { + sessionID, + messageID, + agent: "build", + abort: new AbortController().signal, + callID: "call_domain", + messages: [], + metadata: () => Effect.void, + ask: () => Effect.void, +} + +/** Collects the parts a program writes, so `channel.send` is asserted on its effect. */ +function recordingSessions() { + const parts: SessionV1.Part[] = [] + const sessions = { + get: () => Effect.succeed({ permission: [] } as any), + updatePart: (part: SessionV1.Part) => + Effect.sync(() => { + parts.push(part) + return part + }), + } + return { parts, sessions } +} + +function harness(sessions: Record) { + return Layer.mergeAll( + Layer.mock(Plugin.Service, { + trigger: ((_name, _input, output) => Effect.succeed(output)) as Plugin.Interface["trigger"], + }), + Layer.mock(Truncate.Service, { + output: (text: string) => Effect.succeed({ content: text, truncated: false as const }), + }), + Layer.mock(Agent.Service, { get: () => Effect.succeed({ name: "build", permission: [] } as any) }), + Layer.mock(Session.Service, sessions as any), + Layer.mock(MCP.Service, { tools: () => Effect.succeed({}), clients: () => Effect.succeed({}) }), + ) +} + +/** Runs one Code Mode program through the real `execute` tool. */ +function run(code: string, sessions: Record) { + return Effect.runPromise( + CodeModeTool.pipe( + Effect.flatMap(Tool.init), + Effect.flatMap((def) => def.execute({ code }, ctx)), + Effect.provide(harness(sessions)), + ), + ) +} + +/** Program failures die at the tool boundary; recover the defect for message assertions. */ +async function failureOf(code: string, sessions: Record) { + const exit = await Effect.runPromise( + CodeModeTool.pipe( + Effect.flatMap(Tool.init), + Effect.flatMap((def) => def.execute({ code }, ctx)), + Effect.provide(harness(sessions)), + Effect.exit, + ), + ) + if (Exit.isSuccess(exit)) throw new Error("expected the program to fail") + return (Cause.squash(exit.cause) as Error).message +} + +describe("K-1 typed domain objects", () => { + test("every global names a namespace of the domain tool tree", () => { + const tools = domainTools({ page: unavailablePage(), channel: unavailableChannel() }) + expect([...(DOMAIN_GLOBALS as ReadonlyArray)].toSorted()).toStrictEqual(Object.keys(tools).toSorted()) + }) + + test("the tool names are the interface method names", () => { + expect(Object.keys(pageTools(unavailablePage())).toSorted()).toStrictEqual([ + "click", + "navigate", + "query", + "text", + "type", + "url", + ]) + expect(Object.keys(channelTools(unavailableChannel())).toSorted()).toStrictEqual(["id", "send"]) + }) + + test("the globals are advertised in the catalog description", () => { + const instructions = describeCatalog({}, []) + expect(instructions).toContain("## Domain globals") + expect(instructions).toContain("`page`") + expect(instructions).toContain("`channel`") + }) + + describe("channel", () => { + test("send posts a text part into the session that owns the run", async () => { + const { parts, sessions } = recordingSessions() + const result = await run('return await channel.send({ text: "posted by a skill" })', sessions) + expect(result.output).toStartWith("prt_") + expect(parts).toHaveLength(1) + expect(parts[0]).toMatchObject({ + type: "text", + text: "posted by a skill", + sessionID, + messageID, + }) + }) + + test("id returns the session the program is bound to", async () => { + const { sessions } = recordingSessions() + expect((await run("return await channel.id({})", sessions)).output).toBe(sessionID) + }) + + test("the bare global and the tools path are the same call", async () => { + const { parts, sessions } = recordingSessions() + await run('await channel.send({ text: "a" }); return await tools.channel.send({ text: "b" })', sessions) + expect(parts.map((part) => (part.type === "text" ? part.text : undefined))).toStrictEqual(["a", "b"]) + }) + }) + + describe("page is unavailable rather than faked", () => { + test("every method refuses, naming the missing capability", async () => { + const page = unavailablePage() + const calls = [ + page.url(), + page.text(), + page.query({ selector: "h1" }), + page.click({ selector: "h1" }), + page.type({ selector: "input", text: "x" }), + page.navigate({ url: "https://example.invalid" }), + ] + for (const call of calls) { + const exit = await Effect.runPromise(call.pipe(Effect.exit)) + expect(Exit.isFailure(exit)).toBe(true) + const error = Exit.isFailure(exit) ? Cause.squash(exit.cause) : undefined + expect(error).toBeInstanceOf(DomainUnavailableError) + expect((error as DomainUnavailableError).message).toContain("no browser-page control surface") + } + }) + + test("a program calling page.text fails with that reason, not an empty string", async () => { + const { sessions } = recordingSessions() + const message = await failureOf("return await page.text({})", sessions) + expect(message).toContain("`page` domain object is not available") + expect(message).toContain("no browser-page control surface") + }) + + test("the refusal carries the named object", () => { + const error = new DomainUnavailableError({ object: "page", missing: "nothing drives a page here" }) + expect(error.message).toBe( + "The `page` domain object is not available in this session: nothing drives a page here", + ) + }) + }) + + test("the execute tool still reports its own name", async () => { + const { sessions } = recordingSessions() + const result = await run('return await channel.send({ text: "t" })', sessions) + expect(result.title).toBe(CODE_MODE_TOOL) + }) + + test("PartID is the id shape channel.send returns", () => { + expect(PartID.ascending()).toStartWith("prt_") + }) +}) diff --git a/packages/schema/src/skill.ts b/packages/schema/src/skill.ts index ec299180ed..4dfe2afdba 100644 --- a/packages/schema/src/skill.ts +++ b/packages/schema/src/skill.ts @@ -16,11 +16,29 @@ export const UrlSource = Schema.Struct({ url: Schema.String, }).annotate({ identifier: "SkillV2.UrlSource" }) +/** + * K-2 auto-arming block. A skill carrying one arms itself when the current prompt or the + * current tab URL matches; a skill WITHOUT one never auto-arms and stays explicitly + * loadable. Both lists are optional and an empty list matches nothing, so + * `autoInject: {}` is a skill that still never arms rather than one that arms on + * everything - a skill that arms on everything is a skill that is always in the prompt. + */ +export interface AutoInject extends Schema.Schema.Type {} +export const AutoInject = Schema.Struct({ + /** Case-insensitive keywords matched against the prompt text on word boundaries. */ + keywords: Schema.Array(Schema.String).pipe(optional), + /** Globs matched against the current tab URL, e.g. `docs.google.com/document/**`. */ + url: Schema.Array(Schema.String).pipe(optional), +}).annotate({ identifier: "SkillV2.AutoInject" }) + export interface Info extends Schema.Schema.Type {} export const Info = Schema.Struct({ name: Schema.String, description: Schema.String.pipe(optional), slash: Schema.Boolean.pipe(optional), + /** Icon URL declared in frontmatter, for surfaces that list skills. */ + icon: Schema.String.pipe(optional), + autoInject: AutoInject.pipe(optional), location: AbsolutePath, content: Schema.String, }).annotate({ identifier: "SkillV2.Info" }) From d72c7599e84895a4b63e41c935d3f6d22c89ccd4 Mon Sep 17 00:00:00 2001 From: Janghoon Lee <44862514+its-janghoon@users.noreply.github.com> Date: Fri, 2 Oct 2026 23:01:04 +0900 Subject: [PATCH 06/33] K-3 document toolchain in code mode, and K-5 a skill that writes skills (#63) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(codemode): K-3 a real document toolchain in code mode Four typed globals in the same shape as K-1's `page` — `docx`, `xlsx`, `pptx` and `pdf` — each with a `read` and a `write`, bound into the Code Mode interpreter scope and described to the model by the catalog preview. The rule the module is built around: a write produces a real file or it fails. Each write serialises through a format library, lands bytes on disk, then stats the file and returns the size the filesystem reports — so the byte count is a reading of the artefact, not of the buffer we hoped to write. All four formats can genuinely write, PDF included; nothing here is stubbed and nothing returns a stringified summary of a document it did not produce. Reads use the zip reader already vendored for OOXML (`@zip.js/zip.js`) for `docx` and `pptx`, ExcelJS for `xlsx`, and pdf-lib plus a content-stream text scanner for `pdf`. `pdf.read` is the one with an honest limit, and it states it per page rather than hiding it. Text is extracted from content streams, not rendered, so a page whose text is drawn with a composite (Type0) font carries glyph identifiers instead of characters: those runs are SKIPPED and the page comes back with `textComplete: false` and a reason naming the font. A scanned page has no text operators at all and is reported the same way — there is no OCR in the engine process. Returning glyph soup as text would be exactly the fake success this module exists to refuse. Writes are gated by the same external-directory check the `write` tool uses, so a program cannot drop a document outside the instance directory without the user being asked, and a refusal leaves no file behind. `DomainUnavailableError` moves to its own leaf module. `domain.ts` folds the new globals into `DOMAIN_GLOBALS` at module scope and `document.ts` needs the error class, so importing it from `domain.ts` made the two a cycle whose evaluation order decided whether `DOCUMENT_GLOBALS` was defined when `domain.ts` read it — it was not, and the suite failed on the TDZ. A leaf module has no such order to get wrong. Dependencies, pinned exactly, all MIT, all the canonical package for their job: - docx 9.8.1 (dolanmiu) — writes the Word package. - exceljs 4.4.0 (exceljs) — reads and writes the Excel workbook. - pptxgenjs 4.0.1 (gitbrent) — writes the PowerPoint deck. - pdf-lib 1.17.1 (Hopding) — writes the PDF, and gives the object graph the text scanner walks on read. The K-3 skill document ships as a builtin skill, auto-armed on the words a request for this work actually contains and on the three Google editors plus Office on the web. A second test holds that document to the code BOTH ways: every `.` it names is a real tool, every real tool is named by it, and the result fields it promises are compared against the keys real calls return rather than a list retyped in the test. Verified: 25 unit tests over real files in a temp directory, including a program run through the real `execute` tool whose written `.docx` is then stat'd and magic-checked from outside the sandbox; full engine suite 3432 pass / 0 fail; core 993 pass / 0 fail; `packages/client` `check:generated` clean, as no schema changed. * feat(skill): K-5 a skill that writes skills A builtin skill document whose body is the instruction for producing a new `SKILL.md`: the two layouts the loader's `{*.md,**/SKILL.md}` glob actually finds, every frontmatter key its decoder actually reads, what K-2's `autoInject` block does and why an empty one still never arms, and a template to copy. It deliberately carries no `autoInject` of its own. Writing a skill is something a person asks for by name, and a skill that armed on the word "skill" would be in the prompt of every session that mentions one. The frontmatter the skill tells an agent to write is not asserted by eye, and it is not a copy of something checked elsewhere. The test EXTRACTS the template out of the shipped document — anchored on the document's own `` marker, so adding an earlier example cannot silently repoint it at the wrong fence — writes it where the loader globs, and asserts the decoded `SkillV2.Info`: name, description, slash, icon, and both `autoInject` lists. So a key the loader does not read, a shape its decoder rejects, or a typo in the document's own template fails the test. Reverse-verified: misspelling one frontmatter key in the document turns the suite red, and restoring it turns it green. Three further claims the document makes are tested as behaviour rather than prose: the loaded skill re-decodes through `Info` (the loader builds it with `make`, which does not run the decoder, so this is the only place the whole value meets the schema); a `SKILL.md` in a subdirectory with no `name` is dropped, which is the most common way a new skill silently fails to appear; and a malformed `autoInject` costs only the arming while the skill still loads. The builtin skills become an exported `BuiltinSkills` list the plugin iterates, rather than three inline `draft.source` calls, so a test can run the real decoder over every skill the engine ships. Verified: core 993 pass / 0 fail, measured on this tree in this change. * fix(K-5): read the skill template whatever line endings it was checked out with The test pulls K-5's template out of the skill document with a regex anchored on \n. There is no `* text=auto` in .gitattributes, so a Windows checkout converts the document to CRLF, the pattern then matches no fence at all, and the helper throws before any assertion runs. It failed only in the Windows core job, which is the single place it could show: on Linux the document is LF and the pattern matches. Reverse-verified rather than assumed. With the document converted to CRLF and the normalisation removed, the suite reproduces the CI error exactly ('no ```markdown fence follows the template marker', 0 pass); with the normalisation in place it is 6 pass against the CRLF document. --------- Co-authored-by: Janghoon Lee <44862514+savagemanage@users.noreply.github.com> --- bun.lock | 202 +++- packages/core/src/plugin/skill.ts | 65 +- .../src/plugin/skill/document-toolchain.md | 125 +++ .../core/src/plugin/skill/skill-writer.md | 104 ++ packages/core/test/skill/skill-writer.test.ts | 172 ++++ packages/redrob/package.json | 4 + packages/redrob/src/tool/code-mode.ts | 33 +- packages/redrob/src/tool/document.ts | 930 ++++++++++++++++++ packages/redrob/src/tool/domain-error.ts | 23 + packages/redrob/src/tool/domain.ts | 28 +- packages/redrob/test/tool/code-mode.test.ts | 9 +- .../redrob/test/tool/document-skill.test.ts | 101 ++ packages/redrob/test/tool/document.test.ts | 449 +++++++++ 13 files changed, 2213 insertions(+), 32 deletions(-) create mode 100644 packages/core/src/plugin/skill/document-toolchain.md create mode 100644 packages/core/src/plugin/skill/skill-writer.md create mode 100644 packages/core/test/skill/skill-writer.test.ts create mode 100644 packages/redrob/src/tool/document.ts create mode 100644 packages/redrob/src/tool/domain-error.ts create mode 100644 packages/redrob/test/tool/document-skill.test.ts create mode 100644 packages/redrob/test/tool/document.test.ts diff --git a/bun.lock b/bun.lock index 2774c1a1fe..94ec851482 100644 --- a/bun.lock +++ b/bun.lock @@ -372,8 +372,10 @@ "cross-spawn": "catalog:", "decimal.js": "10.5.0", "diff": "catalog:", + "docx": "9.8.1", "drizzle-orm": "catalog:", "effect": "catalog:", + "exceljs": "4.4.0", "fuzzysort": "3.1.0", "gitlab-ai-provider": "6.12.1", "glob": "13.0.5", @@ -391,6 +393,8 @@ "opencode-poe-auth": "0.0.1", "opentui-spinner": "catalog:", "partial-json": "0.1.7", + "pdf-lib": "1.17.1", + "pptxgenjs": "4.0.1", "remeda": "catalog:", "semver": "^7.6.3", "solid-js": "catalog:", @@ -852,8 +856,6 @@ "@clack/prompts": ["@clack/prompts@1.0.0-alpha.1", "", { "dependencies": { "@clack/core": "1.0.0-alpha.1", "picocolors": "^1.0.0", "sisteransi": "^1.0.5" } }, "sha512-07MNT0OsxjKOcyVfX8KhXBhJiyUbDP1vuIAcHc+nx5v93MJO23pX3X/k3bWz6T3rpM9dgWPq90i4Jq7gZAyMbw=="], - "@cloudflare/workers-types": ["@cloudflare/workers-types@4.20251008.0", "", {}, "sha512-dZLkO4PbCL0qcCSKzuW7KE4GYe49lI12LCfQ5y9XeSwgYBoAUbwH4gmJ6A0qUIURiTJTkGkRkhVPqpq2XNgYRA=="], - "@drizzle-team/brocli": ["@drizzle-team/brocli@0.11.0", "", {}, "sha512-hD3pekGiPg0WPCCGAZmusBBJsDqGUR66Y452YgQsZOnkdQ7ViEPKuyP4huUGEZQefp8g34RRodXYmJ2TbCH+tg=="], "@effect/opentelemetry": ["@effect/opentelemetry@4.0.0-beta.83", "", { "peerDependencies": { "@opentelemetry/api": "^1.9", "@opentelemetry/api-logs": ">=0.203.0 <0.300.0", "@opentelemetry/resources": "^2.0.0", "@opentelemetry/sdk-logs": ">=0.203.0 <0.300.0", "@opentelemetry/sdk-metrics": "^2.0.0", "@opentelemetry/sdk-trace-base": "^2.0.0", "@opentelemetry/sdk-trace-node": "^2.0.0", "@opentelemetry/sdk-trace-web": "^2.0.0", "@opentelemetry/semantic-conventions": "^1.33.0", "effect": "^4.0.0-beta.83" }, "optionalPeers": ["@opentelemetry/api", "@opentelemetry/api-logs", "@opentelemetry/resources", "@opentelemetry/sdk-logs", "@opentelemetry/sdk-metrics", "@opentelemetry/sdk-trace-base", "@opentelemetry/sdk-trace-node", "@opentelemetry/sdk-trace-web"] }, "sha512-cPfCfp/ghu0itbX6Dqjdr4N0rbjng5ON4sUpnLHV5JJySG8zZpWmuOZLWIrfrNKT2ctYR1BYmp1aYCgkItaJLw=="], @@ -916,6 +918,10 @@ "@esbuild/win32-x64": ["@esbuild/win32-x64@0.25.12", "", { "os": "win32", "cpu": "x64" }, "sha512-alJC0uCZpTFrSL0CCDjcgleBXPnCrEAhTBILpeAp7M/OFgoqtAetfBzX0xM00MUsVVPpVjlPuMbREqnZCXaTnA=="], + "@fast-csv/format": ["@fast-csv/format@4.3.5", "", { "dependencies": { "@types/node": "^14.0.1", "lodash.escaperegexp": "^4.1.2", "lodash.isboolean": "^3.0.3", "lodash.isequal": "^4.5.0", "lodash.isfunction": "^3.0.9", "lodash.isnil": "^4.0.0" } }, "sha512-8iRn6QF3I8Ak78lNAa+Gdl5MJJBM5vRHivFtMRUWINdevNo00K7OXxS2PshawLKTejVwieIlPmK5YlLu6w4u8A=="], + + "@fast-csv/parse": ["@fast-csv/parse@4.3.6", "", { "dependencies": { "@types/node": "^14.0.1", "lodash.escaperegexp": "^4.1.2", "lodash.groupby": "^4.6.0", "lodash.isfunction": "^3.0.9", "lodash.isnil": "^4.0.0", "lodash.isundefined": "^3.0.1", "lodash.uniq": "^4.5.0" } }, "sha512-uRsLYksqpbDmWaSmzvJcuApSEe38+6NQZBUsuAyMZKqHxH0g1wcJgsKUvN3WC8tewaqFjBMMGrkHmC+T7k8LvA=="], + "@fastify/ajv-compiler": ["@fastify/ajv-compiler@4.0.6", "", { "dependencies": { "ajv": "^8.12.0", "ajv-formats": "^3.0.1", "fast-uri": "^4.0.0" } }, "sha512-NtuzM0SfaMJbGlnjr9LWQUN5LzgSrbB8tf/wRZNas+4E1O/Nmzl53e7ruT61HDZyRCJGC6FxIogmNZO1c5ETBA=="], "@fastify/busboy": ["@fastify/busboy@2.1.1", "", {}, "sha512-vBZP4NlzfOlerQTnba4aqZoMhE/a9HY7HRqoOPaETQcSQuWEIyZMHGfVu6w9wGtGK5fED5qRs2DteVCjOH60sA=="], @@ -1232,6 +1238,10 @@ "@parcel/watcher-win32-x64": ["@parcel/watcher-win32-x64@2.5.1", "", { "os": "win32", "cpu": "x64" }, "sha512-9lHBdJITeNR++EvSQVUcaZoWupyHfXe1jZvGZ06O/5MflPcuPLtEphScIBL+AiCWBO46tDSHzWyD0uDmmZqsgA=="], + "@pdf-lib/standard-fonts": ["@pdf-lib/standard-fonts@1.0.0", "", { "dependencies": { "pako": "^1.0.6" } }, "sha512-hU30BK9IUN/su0Mn9VdlVKsWBS6GyhVfqjwl1FjZN4TxP6cCw0jP2w7V3Hf5uX7M0AZJ16vey9yE0ny7Sa59ZA=="], + + "@pdf-lib/upng": ["@pdf-lib/upng@1.0.1", "", { "dependencies": { "pako": "^1.0.10" } }, "sha512-dQK2FUMQtowVP00mtIksrlZhdFXQZPC+taih1q4CvPZ5vqdxR/LKBaFg0oAfzd1GlHZXXSPdQfzQnt+ViGvEIQ=="], + "@pierre/diffs": ["@pierre/diffs@1.2.10", "", { "dependencies": { "@pierre/theme": "1.0.3", "@pierre/theming": "0.0.1", "@shikijs/transformers": "^3.0.0 || ^4.0.0", "diff": "8.0.3", "hast-util-to-html": "9.0.5", "lru_map": "0.4.1", "shiki": "^3.0.0 || ^4.0.0" }, "peerDependencies": { "react": "^18.3.1 || ^19.0.0", "react-dom": "^18.3.1 || ^19.0.0" } }, "sha512-rPeAmDWarxFVTQpaf4y6wTxjZxU44xKJKoJti2zU21P06DVd9nRHZX+xSIObLB307Qjpaesyb1x/j0z94t7vLw=="], "@pierre/theme": ["@pierre/theme@1.0.3", "", {}, "sha512-sWHv11TMoqKxKDgTIk5VbhQjdPhs8DCcBxbjh3mRlS3YOM/OcrWoGX6MM8eBGn9cUu3M46Py0JnxsG2nJaFTuA=="], @@ -1658,12 +1668,18 @@ "before-after-hook": ["before-after-hook@2.2.3", "", {}, "sha512-NzUnlZexiaH/46WDhANlyR2bXRopNg4F/zuSA3OpZnllCUgRaOF2znDioDWrmbNVsuZk6l9pMquQB38cfBZwkQ=="], + "big-integer": ["big-integer@1.6.52", "", {}, "sha512-QxD8cf2eVqJOOz63z6JIN9BzvVs/dlySa5HGSBH5xtR8dPteIRQnBxxKqkNTiT6jbDTF6jAfrd4oMcND9RGbQg=="], + "bignumber.js": ["bignumber.js@9.3.1", "", {}, "sha512-Ko0uX15oIUS7wJ3Rb30Fs6SkVbLmPBAKdlm7q9+ak9bbIeFf0MwuBsQV6z7+X768/cHsfg+WlysDWJcmthjsjQ=="], "bin-links": ["bin-links@6.0.2", "", { "dependencies": { "cmd-shim": "^8.0.0", "npm-normalize-package-bin": "^5.0.0", "proc-log": "^6.0.0", "read-cmd-shim": "^6.0.0", "write-file-atomic": "^7.0.0" } }, "sha512-frE1t78WOwJ45PKV2cF2tNPjTcs9L1J9s6VkrV59wanRP4GlaomuxYPVma7BwthMg8WnfSory4w5PTE6FZZ81w=="], "binary": ["binary@0.3.0", "", { "dependencies": { "buffers": "~0.1.1", "chainsaw": "~0.1.0" } }, "sha512-D4H1y5KYwpJgK8wk1Cue5LLPgmwHKYSChkbspQg5JtVuR5ulGckxfR62H3AE9UDkdMC8yyXlqYihuz3Aqg2XZg=="], + "bl": ["bl@4.1.0", "", { "dependencies": { "buffer": "^5.5.0", "inherits": "^2.0.4", "readable-stream": "^3.4.0" } }, "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w=="], + + "bluebird": ["bluebird@3.4.7", "", {}, "sha512-iD3898SR7sWVRHbiQv+sHUtHnMvC1o3nW5rAcqnq3uOn07DSAppZYUkIGslDz6gXC7HfunPe7YVBgoEJASPcHA=="], + "body-parser": ["body-parser@1.20.6", "", { "dependencies": { "bytes": "~3.1.2", "content-type": "~1.0.5", "debug": "2.6.9", "depd": "2.0.0", "destroy": "~1.2.0", "http-errors": "~2.0.1", "iconv-lite": "~0.4.24", "on-finished": "~2.4.1", "qs": "~6.15.1", "raw-body": "~2.5.3", "type-is": "~1.6.18", "unpipe": "~1.0.0" } }, "sha512-p5tAzS57i5MV9fZFDj9LeIiTZEufbSe2eDozP+ElheSUq1m74CRq1jI4mYNDdVs9vQztXFLuk/Gd6BWTdwRJ5g=="], "bonjour-service": ["bonjour-service@1.3.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "multicast-dns": "^7.2.5" } }, "sha512-3YuAUiSkWykd+2Azjgyxei8OWf8thdn8AITIog2M4UICzoqfjlqr64WIjEXZllf/W6vK1goqleSR6brGomxQqA=="], @@ -1684,6 +1700,8 @@ "buffer-equal-constant-time": ["buffer-equal-constant-time@1.0.1", "", {}, "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA=="], + "buffer-indexof-polyfill": ["buffer-indexof-polyfill@1.0.2", "", {}, "sha512-I7wzHwA3t1/lwXQh+A5PbNvJxgfo5r3xulgpYDB5zckTu/Z9oUK9biouBKQUjEqzaz3HnAT6TYoovmE+GqSf7A=="], + "buffers": ["buffers@0.1.1", "", {}, "sha512-9q/rDEGSb/Qsvv2qvzIzdluL5k7AaJOTrw23z9reQthrbF7is4CtlT0DXyO1oei2DCp4uojjzQ7igaSHp1kAEQ=="], "bun-ffi-structs": ["bun-ffi-structs@0.2.4", "", { "peerDependencies": { "typescript": "^5" } }, "sha512-AJzsqoVFs1KBbJbWHIYrVZLDC3NhTqqh25awRXqzoLzmBAKr5oqk6+CwuYHAekKx+VBCYVohBoKuRq40dV+TYg=="], @@ -1750,6 +1768,8 @@ "compress-commons": ["compress-commons@6.0.2", "", { "dependencies": { "crc-32": "^1.2.0", "crc32-stream": "^6.0.0", "is-stream": "^2.0.1", "normalize-path": "^3.0.0", "readable-stream": "^4.0.0" } }, "sha512-6FqVXeETqWPoGcfzrXb37E50NP0LXT8kAMu5ooZayhWWdgEY4lBEEcbQNXtkuKQsGduxiIcI4gOTsxTmuq/bSg=="], + "concat-map": ["concat-map@0.0.1", "", {}, "sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg=="], + "confbox": ["confbox@0.2.4", "", {}, "sha512-ysOGlgTFbN2/Y6Cg3Iye8YKulHw+R2fNXHrgSmXISQdMnomY6eNDprVdW9R5xBguEqI954+S6709UyiO7B+6OQ=="], "consola": ["consola@3.4.2", "", {}, "sha512-5IKcdX0nnYavi6G7TtOhwkYzyjfJlatbjMjuLSfE2kYT5pMDOilZ4OvMhi637CcDICTmz3wARPoyhqyX1Y+XvA=="], @@ -1788,6 +1808,8 @@ "data-view-byte-offset": ["data-view-byte-offset@1.0.1", "", { "dependencies": { "call-bound": "^1.0.2", "es-errors": "^1.3.0", "is-data-view": "^1.0.1" } }, "sha512-BS8PfmtDGnrgYdOonGZQdLZslWIeCGFP9tpan0hi1Co2Zr2NKADsvGYA8XxuG/4UWgJ6Cjtv+YJnB6MM69QGlQ=="], + "dayjs": ["dayjs@1.11.23", "", {}, "sha512-QDTCU0M0MxR3hQfnlDJfwekQiaanm1ubOD231u73WBckQ/fsamwRLiE2GBz6D3a/xF1NgfiDLJjXBa1hYOYTtQ=="], + "debug": ["debug@4.4.3", "", { "dependencies": { "ms": "^2.1.3" } }, "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA=="], "decimal.js": ["decimal.js@10.5.0", "", {}, "sha512-8vDa8Qxvr/+d94hSh5P3IJwI5t8/c0KsMp+g8bNw9cY2icONa5aPfvKeieW1WlG0WQYwwhJ7mjui2xtiePQSXw=="], @@ -1826,6 +1848,8 @@ "dns-packet": ["dns-packet@5.6.1", "", { "dependencies": { "@leichtgewicht/ip-codec": "^2.0.1" } }, "sha512-l4gcSouhcgIKRvyy99RNVOgxXiicE+2jZoNmaNmZ6JXiGajBOJAesk1OBlJuM5k2c+eudGdLxDqXuPCKIj6kpw=="], + "docx": ["docx@9.8.1", "", { "dependencies": { "@types/node": "^26.6.2", "hash.js": "^1.1.7", "jszip": "^3.10.1", "nanoid": "^6.0.1", "xml": "^1.0.1", "xml-js": "^1.6.8" } }, "sha512-bv4yDxgiV9aRyh327HG+V36wG0f23wk32SDhXGQpUhnykVtIPaR1K4ISi8E+DYJ7OcRQRkR8Y70QYlSS08V1cA=="], + "dom-serializer": ["dom-serializer@2.0.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.2", "entities": "^4.2.0" } }, "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg=="], "domelementtype": ["domelementtype@2.3.0", "", {}, "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw=="], @@ -1842,6 +1866,8 @@ "dunder-proto": ["dunder-proto@1.0.1", "", { "dependencies": { "call-bind-apply-helpers": "^1.0.1", "es-errors": "^1.3.0", "gopd": "^1.2.0" } }, "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A=="], + "duplexer2": ["duplexer2@0.1.4", "", { "dependencies": { "readable-stream": "^2.0.2" } }, "sha512-asLFVfWWtJ90ZyOUHMqk7/S2w2guQKxUI2itj3d92ADHhxUSbCMGi1f1cBcJ7xM1To+pE/Khbwo1yuNbMEPKeA=="], + "eastasianwidth": ["eastasianwidth@0.2.0", "", {}, "sha512-I88TYZWc9XiYHRQ4/3c5rjjfgkjhLyW2luGIheGERbNQ6OY7yTybanSpDXZa8y7VUP9YmDcYa+eyq4ca7iLqWA=="], "ecdsa-sig-formatter": ["ecdsa-sig-formatter@1.0.11", "", { "dependencies": { "safe-buffer": "^5.0.1" } }, "sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ=="], @@ -1856,6 +1882,8 @@ "encodeurl": ["encodeurl@2.0.0", "", {}, "sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg=="], + "end-of-stream": ["end-of-stream@1.4.5", "", { "dependencies": { "once": "^1.4.0" } }, "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg=="], + "engine.io-client": ["engine.io-client@6.6.6", "", { "dependencies": { "@socket.io/component-emitter": "~3.1.0", "debug": "~4.4.1", "engine.io-parser": "~5.2.1", "ws": "~8.21.0", "xmlhttprequest-ssl": "~2.1.1" } }, "sha512-iY6QdftLQ9pyiPoX082bpf/u1UewnOaJrtJIF9T0++QB34lZrj0uP+Q/bj8AlUsAxqhnkTV2BS8SBZSxOmoV5Q=="], "engine.io-parser": ["engine.io-parser@5.2.3", "", {}, "sha512-HqD3yTBfnBxIrbnM1DoD6Pcq8NECnh8d4As1Qgh0z5Gg3jRRIqijury0CL3ghu/edArpUYiYqQiDUQBIs4np3Q=="], @@ -1904,6 +1932,8 @@ "eventsource-parser": ["eventsource-parser@3.1.1", "", {}, "sha512-EKN1vKAMcZ8MlYMpaNuxN6R9yakzH6uajHcHVTqWJzvu5pWw9DyhbP35HH8MVBQ+dZjAfDxk+A8NiR9KWaXiyQ=="], + "exceljs": ["exceljs@4.4.0", "", { "dependencies": { "archiver": "^5.0.0", "dayjs": "^1.8.34", "fast-csv": "^4.3.1", "jszip": "^3.10.1", "readable-stream": "^3.6.0", "saxes": "^5.0.1", "tmp": "^0.2.0", "unzipper": "^0.10.11", "uuid": "^8.3.0" } }, "sha512-XctvKaEMaj1Ii9oDOqbW/6e1gXknSY4g/aLCDicOXqBE4M0nRWkUu0PTp++UPNzoFY12BNHMfs/VadKIS6llvg=="], + "execa": ["execa@8.0.1", "", { "dependencies": { "cross-spawn": "^7.0.3", "get-stream": "^8.0.1", "human-signals": "^5.0.0", "is-stream": "^3.0.0", "merge-stream": "^2.0.0", "npm-run-path": "^5.1.0", "onetime": "^6.0.0", "signal-exit": "^4.1.0", "strip-final-newline": "^3.0.0" } }, "sha512-VyhnebXciFV2DESc+p6B+y0LjSm0krU4OgJN44qFAhBY0TJ+1V61tYD2+wHusZ6F9n5K+vl8k0sTy7PEfV4qpg=="], "exponential-backoff": ["exponential-backoff@3.1.3", "", {}, "sha512-ZgEeZXj30q+I0EN+CbSSpIyPaJ5HVQD18Z1m+u1FXbAeT94mr1zw50q4q6jiiC447Nl/YTcIYSAftiGqetwXCA=="], @@ -1920,6 +1950,8 @@ "fast-check": ["fast-check@4.9.0", "", { "dependencies": { "pure-rand": "^8.0.0" } }, "sha512-7ms6T7SybUev/PQITciI0yLM2pOSFy5zpG8Ty7tQofcVaQUvrMXp6CBwqF6fThLCLOrfBtuHAtwq6Yu4XPCllg=="], + "fast-csv": ["fast-csv@4.3.6", "", { "dependencies": { "@fast-csv/format": "4.3.5", "@fast-csv/parse": "4.3.6" } }, "sha512-2RNSpuwwsJGP0frGsOmTb9oUF+VkFSM4SyLTDgwf2ciHWTarN0lQTC+F2f/t5J9QjW+c65VFIAAu85GsvMIusw=="], + "fast-decode-uri-component": ["fast-decode-uri-component@1.0.1", "", {}, "sha512-WKgKWg5eUxvRZGwW8FvfbaH7AXSh2cL+3j5fMGzUMCxWBJ3dV3a7Wz8y2f/uQ0e3B6WmodD3oS54jTQ9HVTIIg=="], "fast-deep-equal": ["fast-deep-equal@3.1.3", "", {}, "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q=="], @@ -1974,10 +2006,14 @@ "fresh": ["fresh@0.5.2", "", {}, "sha512-zJ2mQYM18rEFOudeV4GShTGIQ7RbzA7ozbU9I/XBpm7kqgMywgmylMwXHxZJmkVoYkna9d2pVXVXPdYTP9ej8Q=="], + "fs-constants": ["fs-constants@1.0.0", "", {}, "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow=="], + "fs-minipass": ["fs-minipass@3.0.3", "", { "dependencies": { "minipass": "^7.0.3" } }, "sha512-XUBA9XClHbnJWSfBzjkm6RvPsyg3sryZt06BEQoXcF7EK/xpGaQYJgQKDJSUH5SGZ76Y7pFx1QBnXz09rU5Fbw=="], "fs.realpath": ["fs.realpath@1.0.0", "", {}, "sha512-OO0pH2lK6a0hZnAdau5ItzHPI6pUlvI7jMVnxUQRtw4owF2wk8lOSabtGDCTP4Ggrg2MbGnWO9X8K1t4+fGMDw=="], + "fstream": ["fstream@1.0.12", "", { "dependencies": { "graceful-fs": "^4.1.2", "inherits": "~2.0.0", "mkdirp": ">=0.5 0", "rimraf": "2" } }, "sha512-WvJ193OHa0GHPEL+AycEJgxvBEwyfRkN1vhjca23OaPVMCaLCXTd5qAu82AjTcgP1UJmytkOKb63Ypde7raDIg=="], + "function-bind": ["function-bind@1.1.2", "", {}, "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA=="], "function.prototype.name": ["function.prototype.name@1.2.0", "", { "dependencies": { "call-bind": "^1.0.9", "call-bound": "^1.0.4", "es-define-property": "^1.0.1", "es-errors": "^1.3.0", "functions-have-names": "^1.2.3", "has-property-descriptors": "^1.0.2", "hasown": "^2.0.4", "is-callable": "^1.2.7", "is-document.all": "^1.0.0" } }, "sha512-jObKIik1P2QjPHP5nz5BaOtUlfgS0fWo8IUByNXkM+o+02sJOi94em77GwJKQSJ3gfPHdgzLNrHc1uokV4P/ew=="], @@ -2042,6 +2078,8 @@ "has-tostringtag": ["has-tostringtag@1.0.2", "", { "dependencies": { "has-symbols": "^1.0.3" } }, "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw=="], + "hash.js": ["hash.js@1.1.7", "", { "dependencies": { "inherits": "^2.0.3", "minimalistic-assert": "^1.0.1" } }, "sha512-taOaskGt4z4SOANNseOviYDvjEJinIkRgmp7LbKP2YTTmVxWBl87s/uzK9r+44BclBSp2X7K1hqeNfz9JbBeXA=="], + "hasown": ["hasown@2.0.4", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A=="], "hast-util-to-html": ["hast-util-to-html@9.0.5", "", { "dependencies": { "@types/hast": "^3.0.0", "@types/unist": "^3.0.0", "ccount": "^2.0.0", "comma-separated-tokens": "^2.0.0", "hast-util-whitespace": "^3.0.0", "html-void-elements": "^3.0.0", "mdast-util-to-hast": "^13.0.0", "property-information": "^7.0.0", "space-separated-tokens": "^2.0.0", "stringify-entities": "^4.0.0", "zwitch": "^2.0.4" } }, "sha512-OguPdidb+fbHQSU4Q4ZiLKnzWo8Wwsf5bZfbvu7//a9oTYoqD/fWpe96NuHkoS9h0ccGOTe0C4NGXdtS0iObOw=="], @@ -2066,6 +2104,8 @@ "http-proxy-agent": ["http-proxy-agent@7.0.2", "", { "dependencies": { "agent-base": "^7.1.0", "debug": "^4.3.4" } }, "sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig=="], + "https": ["https@1.0.0", "", {}, "sha512-4EC57ddXrkaF0x83Oj8sM6SLQHAWXw90Skqu2M4AEWENZ3F02dFJE/GARA8igO79tcgYqGrD7ae4f5L3um2lgg=="], + "https-proxy-agent": ["https-proxy-agent@7.0.6", "", { "dependencies": { "agent-base": "^7.1.2", "debug": "4" } }, "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw=="], "human-signals": ["human-signals@5.0.0", "", {}, "sha512-AXcZb6vzzrFAUE61HnN4mpLqd/cSIwNQjtNWR0euPm6y0iqx3G4gOXaIDdtdDwZmhwe82LA6+zinmW4UBWVePQ=="], @@ -2080,8 +2120,14 @@ "ignore-walk": ["ignore-walk@8.0.0", "", { "dependencies": { "minimatch": "^10.0.3" } }, "sha512-FCeMZT4NiRQGh+YkeKMtWrOmBgWjHjMJ26WQWrRQyoyzqevdaGSakUaJW5xQYmjLlUVk2qUnCjYVBax9EKKg8A=="], + "image-size": ["image-size@1.2.1", "", { "dependencies": { "queue": "6.0.2" }, "bin": { "image-size": "bin/image-size.js" } }, "sha512-rH+46sQJ2dlwfjfhCyNx5thzrv+dtmBIhPHk0zgRUukHzZ/kRueTJXoYYsclBaKcSMBWuGbOFXtioLpzTb5euw=="], + + "immediate": ["immediate@3.0.6", "", {}, "sha512-XXOFtyqDjNDAQxVfYxuF7g9Il/IbWmmlQg2MYKOH8ExIT1qg6xc4zyS3HaEEATgs1btfzxq15ciUiY7gjSXRGQ=="], + "immer": ["immer@11.1.4", "", {}, "sha512-XREFCPo6ksxVzP4E0ekD5aMdf8WMwmdNaz6vuvxgI40UaEiu6q3p8X52aU6GdyvLY3XXX/8R7JOTXStz/nBbRw=="], + "inflight": ["inflight@1.0.6", "", { "dependencies": { "once": "^1.3.0", "wrappy": "1" } }, "sha512-k92I/b08q4wvFscXCLvqfsHCrjrF7yiXsQuIVvVE7N82W3+aqpzuUdBbfhWcy/FZR3/4IgflMgKLOsvPDrGCJA=="], + "inherits": ["inherits@2.0.4", "", {}, "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ=="], "ini": ["ini@7.0.0", "", {}, "sha512-ifK0CgjALofS5bkrcTy4RaQ9Vx2Knf/eLeIO+NaswQEpH1UblrtTSCIvN71qQDMq0PeQ/SSPojvEJp9vvvfr+w=="], @@ -2170,7 +2216,7 @@ "is64bit": ["is64bit@2.0.0", "", { "dependencies": { "system-architecture": "^0.1.0" } }, "sha512-jv+8jaWCl0g2lSBkNSVXdzfBA0npK1HGC2KtWM9FumFRoGS94g3NbCCLVnCYHLjp4GrW2KZeeSTMo5ddtznmGw=="], - "isarray": ["isarray@2.0.5", "", {}, "sha512-xHjhDr3cNBK0BzdUJSPXZntQUx/mwMS5Rw4A7lPJ90XGAO6ISP/ePDNuo0vhqOZU+UD5JoodwCAAoZQd3FeAKw=="], + "isarray": ["isarray@1.0.0", "", {}, "sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ=="], "isexe": ["isexe@4.0.0", "", {}, "sha512-FFUtZMpoZ8RqHS3XeXEmHWLA4thH+ZxCv2lOiPIn1Xc7CxrqhWzNSDzD+/chS/zbYezmiwWLdQC09JdQKmthOw=="], @@ -2220,6 +2266,8 @@ "jsonwebtoken": ["jsonwebtoken@9.0.3", "", { "dependencies": { "jws": "^4.0.1", "lodash.includes": "^4.3.0", "lodash.isboolean": "^3.0.3", "lodash.isinteger": "^4.0.4", "lodash.isnumber": "^3.0.3", "lodash.isplainobject": "^4.0.6", "lodash.isstring": "^4.0.1", "lodash.once": "^4.0.0", "ms": "^2.1.1", "semver": "^7.5.4" } }, "sha512-MT/xP0CrubFRNLNKvxJ2BYfy53Zkm++5bX9dtuPbqAeQpTVe0MQTFhao8+Cp//EmJp244xt6Drw/GVEGCUj40g=="], + "jszip": ["jszip@3.10.2", "", { "dependencies": { "lie": "~3.3.0", "pako": "~1.0.2", "readable-stream": "~2.3.6", "setimmediate": "^1.0.5" } }, "sha512-3l+rb15IOWtUhU0H5MFqES/T6Kh7abYwjosBey/vD6hDt8zoEffkSC5Ws5SGtgVw3gBx2NEbhTeSW1+kWkpyTQ=="], + "just-diff": ["just-diff@6.0.2", "", {}, "sha512-S59eriX5u3/QhMNq3v/gm8Kd0w8OS6Tz2FS1NG4blv+z0MuQcBRJyFWjdovM0Rad4/P4aUPFtnkNjMjyMlMSYA=="], "just-diff-apply": ["just-diff-apply@5.5.0", "", {}, "sha512-OYTthRfSh55WOItVqwpefPtNt2VdKsq5AnAK6apdtR6yCH8pr0CmSr710J0Mf+WdQy7K/OzMy7K2MgAfdQURDw=="], @@ -2236,26 +2284,52 @@ "lazystream": ["lazystream@1.0.1", "", { "dependencies": { "readable-stream": "^2.0.5" } }, "sha512-b94GiNHQNy6JNTrt5w6zNyffMrNkXZb3KTkCZJb2V1xaEGCk093vkZ2jk3tpaeP33/OiXC+WvK9AxUebnf5nbw=="], + "lie": ["lie@3.3.0", "", { "dependencies": { "immediate": "~3.0.5" } }, "sha512-UaiMJzeWRlEujzAuw5LokY1L5ecNQYZKfmyZ9L7wDHb/p5etKaxXhohBcrw0EYby+G/NA52vRSN4N39dxHAIwQ=="], + "light-my-request": ["light-my-request@6.6.0", "", { "dependencies": { "cookie": "^1.0.1", "process-warning": "^4.0.0", "set-cookie-parser": "^2.6.0" } }, "sha512-CHYbu8RtboSIoVsHZ6Ye4cj4Aw/yg2oAFimlF7mNvfDV192LR7nDiKtSIfCuLT7KokPSTn/9kfVLm5OGN0A28A=="], + "listenercount": ["listenercount@1.0.1", "", {}, "sha512-3mk/Zag0+IJxeDrxSgaDPy4zZ3w05PRZeJNnlWhzFz5OkX49J4krc+A8X2d2M69vGMBEX0uyl8M+W+8gH+kBqQ=="], + "locate-path": ["locate-path@3.0.0", "", { "dependencies": { "p-locate": "^3.0.0", "path-exists": "^3.0.0" } }, "sha512-7AO748wWnIhNqAuaty2ZWHkQHRSNfPVIsPIfwEOWO22AmaoVrWavlOcMR5nzTLNYvp36X220/maaRsrec1G65A=="], "lodash": ["lodash@4.18.1", "", {}, "sha512-dMInicTPVE8d1e5otfwmmjlxkZoUpiVLwyeTdUsi/Caj/gfzzblBcCE5sRHV/AsjuCmxWrte2TNGSYuCeCq+0Q=="], + "lodash.defaults": ["lodash.defaults@4.2.0", "", {}, "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ=="], + + "lodash.difference": ["lodash.difference@4.5.0", "", {}, "sha512-dS2j+W26TQ7taQBGN8Lbbq04ssV3emRw4NY58WErlTO29pIqS0HmoT5aJ9+TUQ1N3G+JOZSji4eugsWwGp9yPA=="], + + "lodash.escaperegexp": ["lodash.escaperegexp@4.1.2", "", {}, "sha512-TM9YBvyC84ZxE3rgfefxUWiQKLilstD6k7PTGt6wfbtXF8ixIJLOL3VYyV/z+ZiPLsVxAsKAFVwWlWeb2Y8Yyw=="], + + "lodash.flatten": ["lodash.flatten@4.4.0", "", {}, "sha512-C5N2Z3DgnnKr0LOpv/hKCgKdb7ZZwafIrsesve6lmzvZIRZRGaZ/l6Q8+2W7NaT+ZwO3fFlSCzCzrDCFdJfZ4g=="], + + "lodash.groupby": ["lodash.groupby@4.6.0", "", {}, "sha512-5dcWxm23+VAoz+awKmBaiBvzox8+RqMgFhi7UvX9DHZr2HdxHXM/Wrf8cfKpsW37RNrvtPn6hSwNqurSILbmJw=="], + "lodash.includes": ["lodash.includes@4.3.0", "", {}, "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w=="], "lodash.isboolean": ["lodash.isboolean@3.0.3", "", {}, "sha512-Bz5mupy2SVbPHURB98VAcw+aHh4vRV5IPNhILUCsOzRmsTmSQ17jIuqopAentWoehktxGd9e/hbIXq980/1QJg=="], + "lodash.isequal": ["lodash.isequal@4.5.0", "", {}, "sha512-pDo3lu8Jhfjqls6GkMgpahsF9kCyayhgykjyLMNFTKWrpVdAQtYyB4muAMWozBB4ig/dtWAmsMxLEI8wuz+DYQ=="], + + "lodash.isfunction": ["lodash.isfunction@3.0.9", "", {}, "sha512-AirXNj15uRIMMPihnkInB4i3NHeb4iBtNg9WRWuK2o31S+ePwwNmDPaTL3o7dTJ+VXNZim7rFs4rxN4YU1oUJw=="], + "lodash.isinteger": ["lodash.isinteger@4.0.4", "", {}, "sha512-DBwtEWN2caHQ9/imiNeEA5ys1JoRtRfY3d7V9wkqtbycnAmTvRRmbHKDV4a0EYc678/dia0jrte4tjYwVBaZUA=="], + "lodash.isnil": ["lodash.isnil@4.0.0", "", {}, "sha512-up2Mzq3545mwVnMhTDMdfoG1OurpA/s5t88JmQX809eH3C8491iu2sfKhTfhQtKY78oPNhiaHJUpT/dUDAAtng=="], + "lodash.isnumber": ["lodash.isnumber@3.0.3", "", {}, "sha512-QYqzpfwO3/CWf3XP+Z+tkQsfaLL/EnUlXWVkIk5FUPc4sBdTehEqZONuyRt2P67PXAk+NXmTBcc97zw9t1FQrw=="], "lodash.isplainobject": ["lodash.isplainobject@4.0.6", "", {}, "sha512-oSXzaWypCMHkPC3NvBEaPHf0KsA5mvPrOPgQWDsbg8n7orZ290M0BmC/jgRZ4vcJ6DTAhjrsSYgdsW/F+MFOBA=="], "lodash.isstring": ["lodash.isstring@4.0.1", "", {}, "sha512-0wJxfxH1wgO3GrbuP+dTTk7op+6L41QCXbGINEmD+ny/G/eCqGzxyCsh7159S+mgDDcoarnBw6PC1PS5+wUGgw=="], + "lodash.isundefined": ["lodash.isundefined@3.0.1", "", {}, "sha512-MXB1is3s899/cD8jheYYE2V9qTHwKvt+npCwpD+1Sxm3Q3cECXCiYHjeHWXNwr6Q0SOBPrYUDxendrO6goVTEA=="], + "lodash.once": ["lodash.once@4.1.1", "", {}, "sha512-Sb487aTOCr9drQVL8pIxOzVhafOjZN9UU54hiN8PU3uAiSV7lx1yYNpbNmex2PK6dSJoNTSJUUswT651yww3Mg=="], + "lodash.union": ["lodash.union@4.6.0", "", {}, "sha512-c4pB2CdGrGdjMKYLA+XiRDO7Y0PRQbm/Gzg8qMj+QH+pFVAoTp5sBpO0odL3FjoPCGjK96p6qsP+yQoiLoOBcw=="], + + "lodash.uniq": ["lodash.uniq@4.5.0", "", {}, "sha512-xfBaXQd9ryd9dlSDvnvI0lvxfLJlYAZzXomUYzLKtUeOQvOP5piqAWuGtrhWeqaXK9hhoM/iyJc5AV+XfsX3HQ=="], + "long": ["long@5.3.2", "", {}, "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA=="], "loose-envify": ["loose-envify@1.4.0", "", { "dependencies": { "js-tokens": "^3.0.0 || ^4.0.0" }, "bin": { "loose-envify": "cli.js" } }, "sha512-lyuxPGr/Wfhrlem2CL/UcnUc1zcqKAImBDzukY7Y5F/yQiNdko6+fRLevlw1HgMySw7f611UIY408EtxRSoK3Q=="], @@ -2300,6 +2374,8 @@ "mimic-fn": ["mimic-fn@4.0.0", "", {}, "sha512-vqiC06CuhBTUdZH+RYl8sFrL096vA45Ok5ISO6sE/Mr1jRbGH4Csnhi8f3wKVl7x8mO4Au7Ir9D3Oyv1VYMFJw=="], + "minimalistic-assert": ["minimalistic-assert@1.0.1", "", {}, "sha512-UtJcAD4yEaGtjPezWuO9wC4nwUnVH/8/Im3yEHQP4b67cXlD/Qr9hdITCU1xDbSEXg2XKNaP8jsReV7vQd00/A=="], + "minimatch": ["minimatch@10.2.5", "", { "dependencies": { "brace-expansion": "^5.0.5" } }, "sha512-MULkVLfKGYDFYejP07QOurDLLQpcjk7Fw+7jXS2R2czRQzR56yHRveU5NDJEOviH+hETZKSkIk5c+T23GjFUMg=="], "minimist": ["minimist@1.2.8", "", {}, "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA=="], @@ -2332,6 +2408,8 @@ "nanoevents": ["nanoevents@7.0.1", "", {}, "sha512-o6lpKiCxLeijK4hgsqfR6CNToPyRU3keKyyI6uwuHRvpRTbZ0wXw51WRgyldVugZqoJfkGFrjrIenYH3bfEO3Q=="], + "nanoid": ["nanoid@6.0.1", "", { "bin": { "nanoid": "bin/nanoid.js" } }, "sha512-3wVS3i51pE2pi1k5FFL/95BGfVS0kSsvDVuGXHOtxox/TywUmtgq+3qiTOTbs9J7KfHaXPiN171k/A6dBnaXFw=="], + "negotiator": ["negotiator@1.1.0", "", { "dependencies": { "content-type": "^2.1.0" } }, "sha512-NMPBRMJgiQHjbd8phG3Vebdx4kZ1H121rbl5IkMqeOsahptB9BKo/d7oJ3zTXqTgagn2bWlNSXkh0QUGM31RYg=="], "node-addon-api": ["node-addon-api@7.1.1", "", {}, "sha512-5m3bsyrjFWE1xf7nz7YXdN4udnVtXK6/Yfgn5qnahL6bCkf2yKt4k3nuTKAtT4r3IG8JNR2ncsIMdZuAzJjHQQ=="], @@ -2436,6 +2514,8 @@ "pacote": ["pacote@21.5.1", "", { "dependencies": { "@gar/promise-retry": "^1.0.0", "@npmcli/git": "^7.0.0", "@npmcli/installed-package-contents": "^4.0.0", "@npmcli/package-json": "^7.0.0", "@npmcli/promise-spawn": "^9.0.0", "@npmcli/run-script": "^10.0.0", "cacache": "^20.0.0", "fs-minipass": "^3.0.0", "minipass": "^7.0.2", "npm-package-arg": "^13.0.0", "npm-packlist": "^10.0.1", "npm-pick-manifest": "^11.0.1", "npm-registry-fetch": "^19.0.0", "proc-log": "^6.0.0", "sigstore": "^4.0.0", "ssri": "^13.0.0", "tar": "^7.4.3" }, "bin": { "pacote": "bin/index.js" } }, "sha512-KvcJ9iy3crysCsgqc4+PknH/w6jkrp8JN36mpZBPwNaDRwTfMZD37YzRazNstiZUOhuF5pno9f78n9mEJBavwg=="], + "pako": ["pako@1.0.11", "", {}, "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw=="], + "parse-conflict-json": ["parse-conflict-json@5.0.1", "", { "dependencies": { "json-parse-even-better-errors": "^5.0.0", "just-diff": "^6.0.0", "just-diff-apply": "^5.2.0" } }, "sha512-ZHEmNKMq1wyJXNwLxyHnluPfRAFSIliBvbK/UiOceROt4Xh9Pz0fq49NytIaeaCUf5VR86hwQ/34FCcNU5/LKQ=="], "parse5": ["parse5@7.3.0", "", { "dependencies": { "entities": "^6.0.0" } }, "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw=="], @@ -2448,6 +2528,8 @@ "path-expression-matcher": ["path-expression-matcher@1.6.2", "", {}, "sha512-enSlaiat05iasnzmgNxRj8reFdj3puY2QpNgP1aPIaVfT6nn9ICuPoFlKHk8EN22HcwewshO+mN2DGbkCEOtqQ=="], + "path-is-absolute": ["path-is-absolute@1.0.1", "", {}, "sha512-AVbw3UJ2e9bq64vSaS9Am0fje1Pa8pbGqTTsmXfaIiMpnr5DlDhfJOuLj9Sf95ZPVDAUerDfEk88MPmPe7UCQg=="], + "path-key": ["path-key@3.1.1", "", {}, "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q=="], "path-parse": ["path-parse@1.0.7", "", {}, "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw=="], @@ -2458,6 +2540,8 @@ "pathe": ["pathe@2.0.3", "", {}, "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w=="], + "pdf-lib": ["pdf-lib@1.17.1", "", { "dependencies": { "@pdf-lib/standard-fonts": "^1.0.0", "@pdf-lib/upng": "^1.0.1", "pako": "^1.0.11", "tslib": "^1.11.1" } }, "sha512-V/mpyJAoTsN4cnP31vc0wfNA1+p20evqqnap0KLoRUN0Yk/p3wN52DOEsL4oBFcLdb76hlpKPtzJIgo67j/XLw=="], + "perfect-debounce": ["perfect-debounce@2.1.0", "", {}, "sha512-LjgdTytVFXeUgtHZr9WYViYSM/g8MkcTPYDlPa3cDqMirHjKiSZPYd6DoL7pK8AJQr+uWkQvCjHNdiMqsrJs+g=="], "picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="], @@ -2484,6 +2568,8 @@ "powershell-utils": ["powershell-utils@0.1.0", "", {}, "sha512-dM0jVuXJPsDN6DvRpea484tCUaMiXWjuCn++HGTqUWzGDjv5tZkEZldAJ/UMlqRYGFrD/etByo4/xOuC/snX2A=="], + "pptxgenjs": ["pptxgenjs@4.0.1", "", { "dependencies": { "@types/node": "^22.8.1", "https": "^1.0.0", "image-size": "^1.2.1", "jszip": "^3.10.1" } }, "sha512-TeJISr8wouAuXw4C1F/mC33xbZs/FuEG6nH9FG1Zj+nuPcGMP5YRHl6X+j3HSUnS1f3at6k75ZZXPMZlA5Lj9A=="], + "prettier": ["prettier@3.6.2", "", { "bin": { "prettier": "bin/prettier.cjs" } }, "sha512-I7AIg5boAr5R0FFtJ6rCfD+LFsWHp81dolrFD8S79U9tb8Az2nGrJncnMSnys+bpQJfRUzqs9hnA81OAA3hCuQ=="], "proc-log": ["proc-log@6.1.0", "", {}, "sha512-iG+GYldRf2BQ0UDUAd6JQ/RwzaQy6mXmsk/IzlYyal4A4SNFw54MeH4/tLkF4I5WoWG9SQwuqWzS99jaFQHBuQ=="], @@ -2514,6 +2600,8 @@ "qs": ["qs@6.15.3", "", { "dependencies": { "es-define-property": "^1.0.1", "side-channel": "^1.1.1" } }, "sha512-O9gl3zCl5h5blw1KGUzQKhA5oUXSl8rwUIM5o0S3nCXMliSvy5Dzx7/DJcI+SwgICv+IneSZwhBh1oSyEHA71A=="], + "queue": ["queue@6.0.2", "", { "dependencies": { "inherits": "~2.0.3" } }, "sha512-iHZWu+q3IdFZFX36ro/lKBkSvfkztY5Y7HMiPlOUjhupPcG2JMfst2KKEpu5XndviX/3UhFbRngUPNKtgvtZiA=="], + "quick-format-unescaped": ["quick-format-unescaped@4.0.4", "", {}, "sha512-tYC1Q1hgyRuHgloV/YXs2w15unPVh8qfu/qCTfhTYamaw7fyhumKa2yGpdSo87vY32rIclj+4fWYQXUMs9EHvg=="], "range-parser": ["range-parser@1.2.1", "", {}, "sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg=="], @@ -2528,7 +2616,7 @@ "read-cmd-shim": ["read-cmd-shim@6.0.0", "", {}, "sha512-1zM5HuOfagXCBWMN83fuFI/x+T/UhZ7k+KIzhrHXcQoeX5+7gmaDYjELQHmmzIodumBHeByBJT4QYS7ufAgs7A=="], - "readable-stream": ["readable-stream@4.7.0", "", { "dependencies": { "abort-controller": "^3.0.0", "buffer": "^6.0.3", "events": "^3.3.0", "process": "^0.11.10", "string_decoder": "^1.3.0" } }, "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg=="], + "readable-stream": ["readable-stream@3.6.2", "", { "dependencies": { "inherits": "^2.0.3", "string_decoder": "^1.1.1", "util-deprecate": "^1.0.1" } }, "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA=="], "readdir-glob": ["readdir-glob@1.1.3", "", { "dependencies": { "minimatch": "^5.1.0" } }, "sha512-v05I2k7xN8zXvPD9N+z/uhXPaj0sUFCe2rcWZIpBsqxfP7xXFQ0tipAd/wjj1YxWyWtUS5IDJpOG82JKt2EAVA=="], @@ -2592,6 +2680,10 @@ "safer-buffer": ["safer-buffer@2.1.2", "", {}, "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg=="], + "sax": ["sax@1.6.1", "", {}, "sha512-42tBVwLWnaQvW5zc4HbZrTuWccECCZfBi92FDuwtqxasH+JbPB3/FOKb1m222K42R4WxuxzzMsTswfzgtSu64Q=="], + + "saxes": ["saxes@5.0.1", "", { "dependencies": { "xmlchars": "^2.2.0" } }, "sha512-5LBh1Tls8c9xgGjw3QrMwETmTMVk0oFgvrFSvWx62llR2hcEInrKNZ2GZCCuuy2lvWrdl5jhbpeqc5hRYKFOcw=="], + "scheduler": ["scheduler@0.23.2", "", { "dependencies": { "loose-envify": "^1.1.0" } }, "sha512-UOShsPwz7NrMUqhR6t0hWjFduvOzbtv7toDH1/hIrfRNIDBnnBWd0CwJTGvTpngVlmwGCdP9/Zl/tVrDqcuYzQ=="], "section-matter": ["section-matter@1.0.0", "", { "dependencies": { "extend-shallow": "^2.0.1", "kind-of": "^6.0.0" } }, "sha512-vfD3pmTzGpufjScBh50YHKzEu2lxBWhVEHsNGoEXmCmn2hKGfeNLYMzCJpe8cD7gqX7TJluOVpBkAequ6dgMmA=="], @@ -2616,6 +2708,8 @@ "set-proto": ["set-proto@1.0.0", "", { "dependencies": { "dunder-proto": "^1.0.1", "es-errors": "^1.3.0", "es-object-atoms": "^1.0.0" } }, "sha512-RJRdvCo6IAnPdsvP/7m6bsQqNnn1FCBX5ZNtFL98MmFF/4xAIJTIg1YbHW5DC2W5SKZanrC6i4HsJqlajw/dZw=="], + "setimmediate": ["setimmediate@1.0.5", "", {}, "sha512-MATJdZp8sLqDl/68LfQmbP8zKPLQNV6BIZoIgrscFDQ+RsvK/BxeDQOgyxKKoh0y/8h3BqVFnCqQ/gd+reiIXA=="], + "setprototypeof": ["setprototypeof@1.2.0", "", {}, "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw=="], "shebang-command": ["shebang-command@2.0.0", "", { "dependencies": { "shebang-regex": "^3.0.0" } }, "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA=="], @@ -2736,6 +2830,8 @@ "tinyglobby": ["tinyglobby@0.2.17", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g=="], + "tmp": ["tmp@0.2.7", "", {}, "sha512-e0votIpp4Uo2AJYSzVHV6xCcawuiez3DzqDAbrTc3YxBkplN6e+dM13ZeIcZnDg/QpSuU2zfZ3rzwY8ukEnaXw=="], + "to-regex-range": ["to-regex-range@5.0.1", "", { "dependencies": { "is-number": "^7.0.0" } }, "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ=="], "toad-cache": ["toad-cache@3.7.4", "", {}, "sha512-m1TdR/rvT7kgGJZhspNtXdsdYk0fddFpJJFlG5s+UkPFo6lkLoZ3YLOaovPYjq1R75NP5JfeTlSHaOsE09peCg=="], @@ -2808,6 +2904,8 @@ "unzip-stream": ["unzip-stream@0.3.4", "", { "dependencies": { "binary": "^0.3.0", "mkdirp": "^0.5.1" } }, "sha512-PyofABPVv+d7fL7GOpusx7eRT9YETY2X04PhwbSipdj6bMxVCFJrr+nm0Mxqbf9hUiTin/UsnuFWBXlDZFy0Cw=="], + "unzipper": ["unzipper@0.10.14", "", { "dependencies": { "big-integer": "^1.6.17", "binary": "~0.3.0", "bluebird": "~3.4.1", "buffer-indexof-polyfill": "~1.0.0", "duplexer2": "~0.1.4", "fstream": "^1.0.12", "graceful-fs": "^4.2.2", "listenercount": "~1.0.1", "readable-stream": "~2.3.6", "setimmediate": "~1.0.4" } }, "sha512-ti4wZj+0bQTiX2KmKWuwj7lhV+2n//uXEotUmGuQqrbVZSEGFMbI68+c6JCQ8aAmUWYvtHEz2A8K6wXvueR/6g=="], + "update-browserslist-db": ["update-browserslist-db@1.3.1", "", { "dependencies": { "escalade": "^3.2.0", "picocolors": "^1.1.1" }, "peerDependencies": { "browserslist": ">= 4.21.0" }, "bin": { "update-browserslist-db": "cli.js" } }, "sha512-ZZ61DsRsOnakl74HAmp3oSN4aXUmEWXf+i/yv0h7tIBfICc3VdrFErQKUUKPgu3AMsTUMbcongALEN4l6GSUrQ=="], "util-deprecate": ["util-deprecate@1.0.2", "", {}, "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw=="], @@ -2866,8 +2964,14 @@ "xdg-basedir": ["xdg-basedir@5.1.0", "", {}, "sha512-GCPAHLvrIH13+c0SuacwvRYj2SxJXQ4kaVTT5xgL3kPrz56XxkF21IGhjSE1+W0aw7gpBWRGXLCPnPby6lSpmQ=="], + "xml": ["xml@1.0.1", "", {}, "sha512-huCv9IH9Tcf95zuYCsQraZtWnJvBtLVE0QHMOs8bWyZAFZNDcYjsPq1nEx8jKA9y+Beo9v+7OBPRisQTjinQMw=="], + + "xml-js": ["xml-js@1.6.11", "", { "dependencies": { "sax": "^1.2.4" }, "bin": { "xml-js": "./bin/cli.js" } }, "sha512-7rVi2KMfwfWFl+GpPg6m80IVMWXLRjO+PxTq7V2CDhoGak0wzYzFgUY2m4XJ47OGdXd8eLE8EmwfAmdjw7lC1g=="], + "xml-naming": ["xml-naming@0.3.0", "", {}, "sha512-ghig2TBE/H11aOVgmahA3MhimvkBr6JIYknH/Dhdk10nXwdbIqBJsbfMxpvFPG8bAw77gN29aQWvKpmVoPlvPQ=="], + "xmlchars": ["xmlchars@2.2.0", "", {}, "sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw=="], + "xmlhttprequest-ssl": ["xmlhttprequest-ssl@2.1.2", "", {}, "sha512-TEU+nJVUUnA4CYJFLvK5X9AOeH4KvDvhIfm0vV1GaQRtchnG0hgK5p8hw/xjv8cunWYCsiPCSDzObPyhEwq3KQ=="], "y18n": ["y18n@5.0.8", "", {}, "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA=="], @@ -3200,10 +3304,14 @@ "ai-gateway-provider/@openrouter/ai-sdk-provider": ["@openrouter/ai-sdk-provider@2.10.0", "", { "peerDependencies": { "ai": "^6.0.0", "zod": "^3.25.0 || ^4.0.0" } }, "sha512-FMsAEjLUt5pWuRE2LDC/LCvVrFjLlrEzUITH5+5SZtfq7KZ2wrOHjQVxzz92sju8S9ltpzW87CLW8/b0oBXVCw=="], + "archiver/readable-stream": ["readable-stream@4.7.0", "", { "dependencies": { "abort-controller": "^3.0.0", "buffer": "^6.0.3", "events": "^3.3.0", "process": "^0.11.10", "string_decoder": "^1.3.0" } }, "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg=="], + "archiver-utils/glob": ["glob@10.5.0", "", { "dependencies": { "foreground-child": "^3.1.0", "jackspeak": "^3.1.2", "minimatch": "^9.0.4", "minipass": "^7.1.2", "package-json-from-dist": "^1.0.0", "path-scurry": "^1.11.1" }, "bin": { "glob": "dist/esm/bin.mjs" } }, "sha512-DfXN8DfhJ7NH3Oe7cFmu3NCu1wKbkReJ8TorzSAFbSKrlNaQSKfIzqYqVY8zlbs2NLBbWpRiU52GX2PbaBVNkg=="], "archiver-utils/is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="], + "archiver-utils/readable-stream": ["readable-stream@4.7.0", "", { "dependencies": { "abort-controller": "^3.0.0", "buffer": "^6.0.3", "events": "^3.3.0", "process": "^0.11.10", "string_decoder": "^1.3.0" } }, "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg=="], + "axios/form-data": ["form-data@4.0.6", "", { "dependencies": { "asynckit": "^0.4.0", "combined-stream": "^1.0.8", "es-set-tostringtag": "^2.1.0", "hasown": "^2.0.4", "mime-types": "^2.1.35" } }, "sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ=="], "axios/https-proxy-agent": ["https-proxy-agent@5.0.1", "", { "dependencies": { "agent-base": "6", "debug": "4" } }, "sha512-dFcAjpTQFgoLMzC2VwU+C/CbS7uRL0lWmxDITmqm7C+7F0Odmj6s9l6alZc6AELXhrnggM2CeWSXHGOdX2YtwA=="], @@ -3212,22 +3320,36 @@ "babel-plugin-module-resolver/glob": ["glob@9.3.5", "", { "dependencies": { "fs.realpath": "^1.0.0", "minimatch": "^8.0.2", "minipass": "^4.2.4", "path-scurry": "^1.6.1" } }, "sha512-e1LleDykUz2Iu+MTYdkSsuWX8lvAjAcs0Xef0lNIu0S2wOAzuTxCJtcd9S3cijlwYF18EsU3rzb8jPVobxDh9Q=="], + "bl/buffer": ["buffer@5.7.1", "", { "dependencies": { "base64-js": "^1.3.1", "ieee754": "^1.1.13" } }, "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ=="], + "body-parser/debug": ["debug@2.6.9", "", { "dependencies": { "ms": "2.0.0" } }, "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA=="], "c12/chokidar": ["chokidar@5.0.0", "", { "dependencies": { "readdirp": "^5.0.0" } }, "sha512-TQMmc3w+5AxjpL8iIiwebF73dRDF4fBIieAqGn9RGCWaEVwQ6Fb2cGe31Yns0RRIzii5goJ1Y7xbMwo1TxMplw=="], "compress-commons/is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="], + "compress-commons/readable-stream": ["readable-stream@4.7.0", "", { "dependencies": { "abort-controller": "^3.0.0", "buffer": "^6.0.3", "events": "^3.3.0", "process": "^0.11.10", "string_decoder": "^1.3.0" } }, "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg=="], + + "crc32-stream/readable-stream": ["readable-stream@4.7.0", "", { "dependencies": { "abort-controller": "^3.0.0", "buffer": "^6.0.3", "events": "^3.3.0", "process": "^0.11.10", "string_decoder": "^1.3.0" } }, "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg=="], + "cross-fetch/node-fetch": ["node-fetch@2.7.0", "", { "dependencies": { "whatwg-url": "^5.0.0" }, "peerDependencies": { "encoding": "^0.1.0" }, "optionalPeers": ["encoding"] }, "sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A=="], "cross-spawn/which": ["which@2.0.2", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "node-which": "./bin/node-which" } }, "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA=="], "dom-serializer/entities": ["entities@4.5.0", "", {}, "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw=="], + "duplexer2/readable-stream": ["readable-stream@2.3.8", "", { "dependencies": { "core-util-is": "~1.0.0", "inherits": "~2.0.3", "isarray": "~1.0.0", "process-nextick-args": "~2.0.0", "safe-buffer": "~5.1.1", "string_decoder": "~1.1.1", "util-deprecate": "~1.0.1" } }, "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA=="], + "effect/@standard-schema/spec": ["@standard-schema/spec@1.1.0", "", {}, "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=="], "engine.io-client/ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="], + "es-get-iterator/isarray": ["isarray@2.0.5", "", {}, "sha512-xHjhDr3cNBK0BzdUJSPXZntQUx/mwMS5Rw4A7lPJ90XGAO6ISP/ePDNuo0vhqOZU+UD5JoodwCAAoZQd3FeAKw=="], + + "exceljs/archiver": ["archiver@5.3.2", "", { "dependencies": { "archiver-utils": "^2.1.0", "async": "^3.2.4", "buffer-crc32": "^0.2.1", "readable-stream": "^3.6.0", "readdir-glob": "^1.1.2", "tar-stream": "^2.2.0", "zip-stream": "^4.1.0" } }, "sha512-+25nxyyznAXF7Nef3y0EbBeqmGZgeN/BxHX29Rs39djAfaFalmQ89SE6CWyDCHzGL0yt/ycBtNOmGTW0FyGWNw=="], + + "exceljs/uuid": ["uuid@8.3.2", "", { "bin": { "uuid": "dist/bin/uuid" } }, "sha512-+NYs2QeMWy+GWFOEm9xnn6HCDp0l7QBD7ml8zLUmJ+93Q5NF0NocErnwkTkXVFNiX3/fpC6afS8Dhb/gz7R7eg=="], + "execa/is-stream": ["is-stream@3.0.0", "", {}, "sha512-LnQR4bZ9IADDRSkvpqMGvt/tEJWclzklNgSw48V5EAaAeDd6qGvN8ei6k5p0tvxSR171VmGyHuTiAOfxAbr8kA=="], "express/debug": ["debug@2.6.9", "", { "dependencies": { "ms": "2.0.0" } }, "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA=="], @@ -3242,6 +3364,8 @@ "form-data/mime-types": ["mime-types@2.1.35", "", { "dependencies": { "mime-db": "1.52.0" } }, "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw=="], + "fstream/rimraf": ["rimraf@2.7.1", "", { "dependencies": { "glob": "^7.1.3" }, "bin": { "rimraf": "./bin.js" } }, "sha512-uWjbaKIK3T1OSVptzX7Nl6PvQ3qAGtKEtVRjRuazjfL3Bx5eI409VZSqgND+4UNnmzLVdPj9FqFJNPqBZFve4w=="], + "gcp-metadata/gaxios": ["gaxios@7.1.3", "", { "dependencies": { "extend": "^3.0.2", "https-proxy-agent": "^7.0.1", "node-fetch": "^3.3.2", "rimraf": "^5.0.1" } }, "sha512-YGGyuEdVIjqxkxVH1pUTMY/XtmmsApXrCVv5EU25iX6inEPbV+VakJfLealkBtJN69AQmh1eGOdCl9Sm1UP6XQ=="], "gcp-metadata/google-logging-utils": ["google-logging-utils@1.1.3", "", {}, "sha512-eAmLkjDjAFCVXg7A1unxHsLf961m6y17QFqXqAXGj/gVkKFrEICfStRfwUlGNfeCEjNRa32JEWOUTlYXPyyKvA=="], @@ -3250,6 +3374,8 @@ "htmlparser2/entities": ["entities@4.5.0", "", {}, "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw=="], + "jszip/readable-stream": ["readable-stream@2.3.8", "", { "dependencies": { "core-util-is": "~1.0.0", "inherits": "~2.0.3", "isarray": "~1.0.0", "process-nextick-args": "~2.0.0", "safe-buffer": "~5.1.1", "string_decoder": "~1.1.1", "util-deprecate": "~1.0.1" } }, "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA=="], + "lazystream/readable-stream": ["readable-stream@2.3.8", "", { "dependencies": { "core-util-is": "~1.0.0", "inherits": "~2.0.3", "isarray": "~1.0.0", "process-nextick-args": "~2.0.0", "safe-buffer": "~5.1.1", "string_decoder": "~1.1.1", "util-deprecate": "~1.0.1" } }, "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA=="], "light-my-request/cookie": ["cookie@1.1.1", "", {}, "sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ=="], @@ -3284,6 +3410,8 @@ "parse5/entities": ["entities@6.0.1", "", {}, "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g=="], + "pdf-lib/tslib": ["tslib@1.14.1", "", {}, "sha512-Xni35NKzjgMrwevysHTCArtLDpPvye8zV/0E4EyYn43P7/7qvQwPh9BGkHewbMulVntbigmcT7rdX3BNo9wRJg=="], + "readdir-glob/minimatch": ["minimatch@5.1.9", "", { "dependencies": { "brace-expansion": "^2.0.1" } }, "sha512-7o1wEA2RyMP7Iu7GNba9vc0RWWGACJOCZBJX2GJWip0ikV+wcOsgVuY9uE8CPiyQhkGFSlhuSkZPavN7u1c2Fw=="], "redrob/@ai-sdk/cerebras": ["@ai-sdk/cerebras@2.0.60", "", { "dependencies": { "@ai-sdk/openai-compatible": "2.0.54", "@ai-sdk/provider": "3.0.12", "@ai-sdk/provider-utils": "4.0.33" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-Rnok3cThg6awBwaDSyiZpgRpbV7pqxGYrA89LODCo5cuEHeP2h0AM0lLHP7zIkclAdXfOm4wldKi/S2T/DGCOw=="], @@ -3292,6 +3420,10 @@ "rimraf/glob": ["glob@10.5.0", "", { "dependencies": { "foreground-child": "^3.1.0", "jackspeak": "^3.1.2", "minimatch": "^9.0.4", "minipass": "^7.1.2", "package-json-from-dist": "^1.0.0", "path-scurry": "^1.11.1" }, "bin": { "glob": "dist/esm/bin.mjs" } }, "sha512-DfXN8DfhJ7NH3Oe7cFmu3NCu1wKbkReJ8TorzSAFbSKrlNaQSKfIzqYqVY8zlbs2NLBbWpRiU52GX2PbaBVNkg=="], + "safe-array-concat/isarray": ["isarray@2.0.5", "", {}, "sha512-xHjhDr3cNBK0BzdUJSPXZntQUx/mwMS5Rw4A7lPJ90XGAO6ISP/ePDNuo0vhqOZU+UD5JoodwCAAoZQd3FeAKw=="], + + "safe-push-apply/isarray": ["isarray@2.0.5", "", {}, "sha512-xHjhDr3cNBK0BzdUJSPXZntQUx/mwMS5Rw4A7lPJ90XGAO6ISP/ePDNuo0vhqOZU+UD5JoodwCAAoZQd3FeAKw=="], + "send/debug": ["debug@2.6.9", "", { "dependencies": { "ms": "2.0.0" } }, "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA=="], "send/mime": ["mime@1.6.0", "", { "bin": { "mime": "cli.js" } }, "sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg=="], @@ -3318,18 +3450,24 @@ "type-is/mime-types": ["mime-types@2.1.35", "", { "dependencies": { "mime-db": "1.52.0" } }, "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw=="], + "unzipper/readable-stream": ["readable-stream@2.3.8", "", { "dependencies": { "core-util-is": "~1.0.0", "inherits": "~2.0.3", "isarray": "~1.0.0", "process-nextick-args": "~2.0.0", "safe-buffer": "~5.1.1", "string_decoder": "~1.1.1", "util-deprecate": "~1.0.1" } }, "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA=="], + "venice-ai-sdk-provider/@ai-sdk/openai-compatible": ["@ai-sdk/openai-compatible@2.0.67", "", { "dependencies": { "@ai-sdk/provider": "3.0.15", "@ai-sdk/provider-utils": "4.0.45" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-glcEJC2mBXJKj7joFI0fRhcbdDYKTBgXMPcT6Vcnlym67tTzuNG9pFx3zblxVv8TdOxhojJja5zGG19yeGJxuA=="], "venice-ai-sdk-provider/@ai-sdk/provider": ["@ai-sdk/provider@3.0.15", "", { "dependencies": { "json-schema": "^0.4.0" } }, "sha512-XeZW1CcDF2GMbH4wejW6xBRI2QCOgnkVYUnxoeDadB1mf85riL2bMUeDoh+6gJ/r4mjNfzUPW8OjLjvwTP0u1Q=="], "venice-ai-sdk-provider/@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.46", "", { "dependencies": { "@ai-sdk/provider": "3.0.15", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.8", "undici": "^6.28.0" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-tEtld97plCFiYevsJuOkGkeuhQndeMWFBVrJS4AjnbD5AqrNSXRCe0p+BZ3Cju/sxDeeZ9ym3q9YUV8fASA7aQ=="], + "which-builtin-type/isarray": ["isarray@2.0.5", "", {}, "sha512-xHjhDr3cNBK0BzdUJSPXZntQUx/mwMS5Rw4A7lPJ90XGAO6ISP/ePDNuo0vhqOZU+UD5JoodwCAAoZQd3FeAKw=="], + "wrap-ansi-cjs/ansi-styles": ["ansi-styles@4.3.0", "", { "dependencies": { "color-convert": "^2.0.1" } }, "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg=="], "wrap-ansi-cjs/string-width": ["string-width@4.2.3", "", { "dependencies": { "emoji-regex": "^8.0.0", "is-fullwidth-code-point": "^3.0.0", "strip-ansi": "^6.0.1" } }, "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g=="], "wrap-ansi-cjs/strip-ansi": ["strip-ansi@6.0.1", "", { "dependencies": { "ansi-regex": "^5.0.1" } }, "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A=="], + "zip-stream/readable-stream": ["readable-stream@4.7.0", "", { "dependencies": { "abort-controller": "^3.0.0", "buffer": "^6.0.3", "events": "^3.3.0", "process": "^0.11.10", "string_decoder": "^1.3.0" } }, "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg=="], + "@actions/artifact/@actions/core/@actions/exec": ["@actions/exec@2.0.0", "", { "dependencies": { "@actions/io": "^2.0.0" } }, "sha512-k8ngrX2voJ/RIN6r9xB82NVqKpnMRtxDoiO+g3olkIUpQNqjArXrCQceduQZCQj3P3xm32pChRLqRrtXTlqhIw=="], "@actions/core/@actions/http-client/undici": ["undici@5.29.0", "", { "dependencies": { "@fastify/busboy": "^2.0.0" } }, "sha512-raqeBD6NQK4SkWhQzeYKd1KmIG6dllBOTt55Rmkt4HtI9mwdWtJljnrXjAFUBLTSN67HWrOIZ3EPF4kjUw80Bg=="], @@ -3562,13 +3700,29 @@ "cross-spawn/which/isexe": ["isexe@2.0.0", "", {}, "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw=="], + "duplexer2/readable-stream/safe-buffer": ["safe-buffer@5.1.2", "", {}, "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g=="], + + "duplexer2/readable-stream/string_decoder": ["string_decoder@1.1.1", "", { "dependencies": { "safe-buffer": "~5.1.0" } }, "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg=="], + + "exceljs/archiver/archiver-utils": ["archiver-utils@2.1.0", "", { "dependencies": { "glob": "^7.1.4", "graceful-fs": "^4.2.0", "lazystream": "^1.0.0", "lodash.defaults": "^4.2.0", "lodash.difference": "^4.5.0", "lodash.flatten": "^4.4.0", "lodash.isplainobject": "^4.0.6", "lodash.union": "^4.6.0", "normalize-path": "^3.0.0", "readable-stream": "^2.0.0" } }, "sha512-bEL/yUb/fNNiNTuUz979Z0Yg5L+LzLxGJz8x79lYmR54fmTIb6ob/hNQgkQnIUDWIFjZVQwl9Xs356I6BAMHfw=="], + + "exceljs/archiver/buffer-crc32": ["buffer-crc32@0.2.13", "", {}, "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ=="], + + "exceljs/archiver/tar-stream": ["tar-stream@2.2.0", "", { "dependencies": { "bl": "^4.0.3", "end-of-stream": "^1.4.1", "fs-constants": "^1.0.0", "inherits": "^2.0.3", "readable-stream": "^3.1.1" } }, "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ=="], + + "exceljs/archiver/zip-stream": ["zip-stream@4.1.1", "", { "dependencies": { "archiver-utils": "^3.0.4", "compress-commons": "^4.1.2", "readable-stream": "^3.6.0" } }, "sha512-9qv4rlDiopXg4E69k+vMHjNN63YFMe9sZMrdlvKnCjlCRWeCBswPPMPUfx+ipsAWq1LXHe70RcbaHdJJpS6hyQ=="], + "express/debug/ms": ["ms@2.0.0", "", {}, "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A=="], "finalhandler/debug/ms": ["ms@2.0.0", "", {}, "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A=="], "form-data/mime-types/mime-db": ["mime-db@1.52.0", "", {}, "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg=="], - "lazystream/readable-stream/isarray": ["isarray@1.0.0", "", {}, "sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ=="], + "fstream/rimraf/glob": ["glob@7.2.3", "", { "dependencies": { "fs.realpath": "^1.0.0", "inflight": "^1.0.4", "inherits": "2", "minimatch": "^3.1.1", "once": "^1.3.0", "path-is-absolute": "^1.0.0" } }, "sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q=="], + + "jszip/readable-stream/safe-buffer": ["safe-buffer@5.1.2", "", {}, "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g=="], + + "jszip/readable-stream/string_decoder": ["string_decoder@1.1.1", "", { "dependencies": { "safe-buffer": "~5.1.0" } }, "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg=="], "lazystream/readable-stream/safe-buffer": ["safe-buffer@5.1.2", "", {}, "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g=="], @@ -3596,6 +3750,10 @@ "type-is/mime-types/mime-db": ["mime-db@1.52.0", "", {}, "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg=="], + "unzipper/readable-stream/safe-buffer": ["safe-buffer@5.1.2", "", {}, "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g=="], + + "unzipper/readable-stream/string_decoder": ["string_decoder@1.1.1", "", { "dependencies": { "safe-buffer": "~5.1.0" } }, "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg=="], + "venice-ai-sdk-provider/@ai-sdk/openai-compatible/@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.45", "", { "dependencies": { "@ai-sdk/provider": "3.0.15", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.8", "undici": "^5.29.0" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-7u5B/E2uZmU65SlJhhQGFHZwRCN0xOz4HHtFc4sEGV9PHbX3fGiEiZBpc/SABay1dGeJgK3VD60rvLGoWdWPXA=="], "venice-ai-sdk-provider/@ai-sdk/provider-utils/@standard-schema/spec": ["@standard-schema/spec@1.1.0", "", {}, "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=="], @@ -3672,6 +3830,16 @@ "babel-plugin-module-resolver/glob/path-scurry/minipass": ["minipass@7.1.3", "", {}, "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A=="], + "exceljs/archiver/archiver-utils/glob": ["glob@7.2.3", "", { "dependencies": { "fs.realpath": "^1.0.0", "inflight": "^1.0.4", "inherits": "2", "minimatch": "^3.1.1", "once": "^1.3.0", "path-is-absolute": "^1.0.0" } }, "sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q=="], + + "exceljs/archiver/archiver-utils/readable-stream": ["readable-stream@2.3.8", "", { "dependencies": { "core-util-is": "~1.0.0", "inherits": "~2.0.3", "isarray": "~1.0.0", "process-nextick-args": "~2.0.0", "safe-buffer": "~5.1.1", "string_decoder": "~1.1.1", "util-deprecate": "~1.0.1" } }, "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA=="], + + "exceljs/archiver/zip-stream/archiver-utils": ["archiver-utils@3.0.4", "", { "dependencies": { "glob": "^7.2.3", "graceful-fs": "^4.2.0", "lazystream": "^1.0.0", "lodash.defaults": "^4.2.0", "lodash.difference": "^4.5.0", "lodash.flatten": "^4.4.0", "lodash.isplainobject": "^4.0.6", "lodash.union": "^4.6.0", "normalize-path": "^3.0.0", "readable-stream": "^3.6.0" } }, "sha512-KVgf4XQVrTjhyWmx6cte4RxonPLR9onExufI1jhvw/MQ4BB6IsZD5gT8Lq+u/+pRkWna/6JoHpiQioaqFP5Rzw=="], + + "exceljs/archiver/zip-stream/compress-commons": ["compress-commons@4.1.2", "", { "dependencies": { "buffer-crc32": "^0.2.13", "crc32-stream": "^4.0.2", "normalize-path": "^3.0.0", "readable-stream": "^3.6.0" } }, "sha512-D3uMHtGc/fcO1Gt1/L7i1e33VOvD4A9hfQLP+6ewd+BvG/gQ84Yh4oftEhAdjSMgBgwGL+jsppT7JYNpo6MHHg=="], + + "fstream/rimraf/glob/minimatch": ["minimatch@3.1.5", "", { "dependencies": { "brace-expansion": "^1.1.7" } }, "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w=="], + "readdir-glob/minimatch/brace-expansion/balanced-match": ["balanced-match@1.0.2", "", {}, "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="], "redrob/@ai-sdk/cerebras/@ai-sdk/provider-utils/@standard-schema/spec": ["@standard-schema/spec@1.1.0", "", {}, "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=="], @@ -3688,6 +3856,30 @@ "babel-plugin-module-resolver/glob/minimatch/brace-expansion/balanced-match": ["balanced-match@1.0.2", "", {}, "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="], + "exceljs/archiver/archiver-utils/glob/minimatch": ["minimatch@3.1.5", "", { "dependencies": { "brace-expansion": "^1.1.7" } }, "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w=="], + + "exceljs/archiver/archiver-utils/readable-stream/safe-buffer": ["safe-buffer@5.1.2", "", {}, "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g=="], + + "exceljs/archiver/archiver-utils/readable-stream/string_decoder": ["string_decoder@1.1.1", "", { "dependencies": { "safe-buffer": "~5.1.0" } }, "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg=="], + + "exceljs/archiver/zip-stream/archiver-utils/glob": ["glob@7.2.3", "", { "dependencies": { "fs.realpath": "^1.0.0", "inflight": "^1.0.4", "inherits": "2", "minimatch": "^3.1.1", "once": "^1.3.0", "path-is-absolute": "^1.0.0" } }, "sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q=="], + + "exceljs/archiver/zip-stream/compress-commons/crc32-stream": ["crc32-stream@4.0.3", "", { "dependencies": { "crc-32": "^1.2.0", "readable-stream": "^3.4.0" } }, "sha512-NT7w2JVU7DFroFdYkeq8cywxrgjPHWkdX1wjpRQXPX5Asews3tA+Ght6lddQO5Mkumffp3X7GEqku3epj2toIw=="], + + "fstream/rimraf/glob/minimatch/brace-expansion": ["brace-expansion@1.1.21", "", { "dependencies": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" } }, "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw=="], + "rimraf/glob/minimatch/brace-expansion/balanced-match": ["balanced-match@1.0.2", "", {}, "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="], + + "exceljs/archiver/archiver-utils/glob/minimatch/brace-expansion": ["brace-expansion@1.1.21", "", { "dependencies": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" } }, "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw=="], + + "exceljs/archiver/zip-stream/archiver-utils/glob/minimatch": ["minimatch@3.1.5", "", { "dependencies": { "brace-expansion": "^1.1.7" } }, "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w=="], + + "fstream/rimraf/glob/minimatch/brace-expansion/balanced-match": ["balanced-match@1.0.2", "", {}, "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="], + + "exceljs/archiver/archiver-utils/glob/minimatch/brace-expansion/balanced-match": ["balanced-match@1.0.2", "", {}, "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="], + + "exceljs/archiver/zip-stream/archiver-utils/glob/minimatch/brace-expansion": ["brace-expansion@1.1.21", "", { "dependencies": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" } }, "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw=="], + + "exceljs/archiver/zip-stream/archiver-utils/glob/minimatch/brace-expansion/balanced-match": ["balanced-match@1.0.2", "", {}, "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="], } } diff --git a/packages/core/src/plugin/skill.ts b/packages/core/src/plugin/skill.ts index 5295d349d0..fcf4a78e5d 100644 --- a/packages/core/src/plugin/skill.ts +++ b/packages/core/src/plugin/skill.ts @@ -7,25 +7,66 @@ import { Effect } from "effect" import { AbsolutePath } from "../schema" import { SkillV2 } from "../skill" import customizeRedrobContent from "./skill/customize-redrob.md" with { type: "text" } +import documentToolchainContent from "./skill/document-toolchain.md" with { type: "text" } +import skillWriterContent from "./skill/skill-writer.md" with { type: "text" } export const CustomizeRedrobContent = customizeRedrobContent +/** K-3's skill document. Exported so a test can assert its claims against the real tool tree. */ +export const DocumentToolchainContent = documentToolchainContent + +/** K-5's skill document. Exported so a test can round-trip the template it prescribes. */ +export const SkillWriterContent = skillWriterContent + +/** + * The skills the engine ships with. Exported as data rather than built inline in the plugin + * effect so a test can decode every entry through `SkillV2.Info` — `Info.make` does not run + * the decoder, so a field in a shape the schema rejects would otherwise reach a session. + */ +export const BuiltinSkills: ReadonlyArray = [ + SkillV2.Info.make({ + name: "customize-redrob", + description: + "Use ONLY when the user is editing or creating redrob's own configuration: redrob.json, redrob.jsonc, files under .redrob/, or files under ~/.config/redrob/. Also use when creating or fixing redrob agents, subagents, commands, skills, plugins, MCP servers, or permission rules. Do not use for the user's own application code, or for any project that is not configuring redrob itself.", + location: AbsolutePath.make("/builtin/customize-redrob.md"), + content: CustomizeRedrobContent, + }), + // K-3. The keywords are the words a request for this work actually contains; the URL globs + // are the web editors where a document is the thing on screen. + SkillV2.Info.make({ + name: "documents", + description: + "Use when reading or writing a Word, Excel, PowerPoint or PDF file in code mode: the `docx`, `xlsx`, `pptx` and `pdf` globals, their exact read and write shapes, and which parts of a PDF genuinely cannot be read. Use it before writing any program that touches a document file. Do not use for plain text, markdown, CSV or JSON, which need no document toolchain.", + location: AbsolutePath.make("/builtin/document-toolchain.md"), + autoInject: SkillV2.AutoInject.make({ + keywords: ["docx", "xlsx", "pptx", "pdf", "spreadsheet", "powerpoint", "word document", "slide deck"], + url: [ + "docs.google.com/document/**", + "docs.google.com/spreadsheets/**", + "docs.google.com/presentation/**", + "*.officeapps.live.com/**", + ], + }), + content: DocumentToolchainContent, + }), + // K-5. No `autoInject`: writing a skill is something a person asks for by name, and a skill + // that armed on the word "skill" would be in the prompt of every session that mentions one. + SkillV2.Info.make({ + name: "skill-writer", + description: + "Use when creating a new redrob skill, or fixing one that does not load: where SKILL.md must sit for the loader to find it, every frontmatter key the loader actually reads, the auto-arming block, and the template to copy. Do not use for agents, commands or plugins, which are different files with different shapes.", + location: AbsolutePath.make("/builtin/skill-writer.md"), + content: SkillWriterContent, + }), +] + export const Plugin = define({ id: "skill", effect: Effect.fn(function* (ctx) { yield* ctx.skill.transform((draft) => { - draft.source( - SkillV2.EmbeddedSource.make({ - type: "embedded", - skill: SkillV2.Info.make({ - name: "customize-redrob", - description: - "Use ONLY when the user is editing or creating redrob's own configuration: redrob.json, redrob.jsonc, files under .redrob/, or files under ~/.config/redrob/. Also use when creating or fixing redrob agents, subagents, commands, skills, plugins, MCP servers, or permission rules. Do not use for the user's own application code, or for any project that is not configuring redrob itself.", - location: AbsolutePath.make("/builtin/customize-redrob.md"), - content: CustomizeRedrobContent, - }), - }), - ) + for (const skill of BuiltinSkills) { + draft.source(SkillV2.EmbeddedSource.make({ type: "embedded", skill })) + } }) }), }) diff --git a/packages/core/src/plugin/skill/document-toolchain.md b/packages/core/src/plugin/skill/document-toolchain.md new file mode 100644 index 0000000000..30f2cdf539 --- /dev/null +++ b/packages/core/src/plugin/skill/document-toolchain.md @@ -0,0 +1,125 @@ + + +# Reading and writing documents in code mode + +Code mode binds four document objects as bare globals: `docx`, `xlsx`, `pptx` and `pdf`. +Each has exactly two methods, `read` and `write`. They are part of the language, not an MCP +server, so they are there whether or not anything is connected. + +**A `write` produces a real file.** It serialises through a format library, lands bytes on +disk, and returns `{ path, bytes }` where `bytes` is the size the filesystem reports for the +file that is now there. There is no mode in which a write returns a summary of a document it +did not produce. If a write cannot be done it throws and says why. + +Paths may be relative — they resolve against the instance directory — or absolute. Writing +outside the instance directory asks the user for permission first, and a refusal throws +rather than writing somewhere else. + +## docx — Word + +```js +const back = await docx.read({ path: "report.docx" }) +// => { paragraphs: string[], text: string } + +await docx.write({ + path: "report.docx", + title: "Quarterly report", + paragraphs: [ + { text: "Revenue grew.", heading: 1 }, + { text: "Costs held flat.", bold: true }, + { text: "Headcount unchanged." }, + ], +}) +// => { path: "/abs/report.docx", bytes: 9445 } +``` + +`read` returns the body's paragraph text in document order, and `text` is those paragraphs +joined with newlines. It does not reconstruct styling, and it does not descend into headers, +footers or footnotes — so a paragraph that only exists in a header will not appear. + +`heading` is a level from 1 to 6. `title` is optional and renders as the document title. + +## xlsx — Excel + +```js +const book = await xlsx.read({ path: "numbers.xlsx" }) +// => { sheets: [{ name: "Totals", rows: [["region", "units"], ["EMEA", 1200]] }] } + +const one = await xlsx.read({ path: "numbers.xlsx", sheet: "Totals" }) + +await xlsx.write({ + path: "numbers.xlsx", + sheets: [{ name: "Totals", rows: [["region", "units"], ["EMEA", 1200]] }], +}) +``` + +Cells come back as strings, numbers, booleans or `null`. A date becomes an ISO 8601 string. +A formula cell gives its cached result, and `null` when the file carries no cached result — +never the formula text, which is not the cell's value. Naming a `sheet` that does not exist +throws instead of returning an empty list. + +## pptx — PowerPoint + +```js +const deck = await pptx.read({ path: "deck.pptx" }) +// => { slides: [{ index: 0, text: ["Slide 1", "point 1a"] }] } + +await pptx.write({ + path: "deck.pptx", + slides: [{ title: "Findings", bullets: ["One", "Two"], notes: "say it slowly" }], +}) +``` + +`read` gives the text of each slide in deck order, `index` counting from zero. It reads the +slides themselves, not layouts or masters, so boilerplate that lives on a master is absent. + +## pdf — PDF + +```js +const doc = await pdf.read({ path: "brief.pdf" }) +// => { pageCount, title?, author?, pages: [{ index, text, textComplete, reason? }] } + +await pdf.write({ + path: "brief.pdf", + title: "Engine brief", + fontSize: 12, + pages: [{ lines: ["First line", "Second line"] }], +}) +``` + +`pdf.write` works: it produces a real PDF, one page per entry, lines laid out in Helvetica, +and a page that runs out of room continues onto a new one rather than dropping text. A +character outside WinAnsi is written as `?`, because the standard fonts cannot encode it. + +**`pdf.read` extracts text from content streams; it does not render the page.** Read +`textComplete` before you trust `text`: + +- `textComplete: true` — the page's text was recovered as characters. +- `textComplete: false` with a `reason` — some or all of it could not be. The three reasons + are a page whose text is drawn with a composite (`Type0`) font, where the string bytes are + glyph identifiers rather than characters and are therefore skipped; a page with no readable + content stream, which is a scan or an image and there is no OCR in the engine; and a page + that draws only graphics. + +A page reported `textComplete: false` has not been read. Say so to the user instead of +treating its `text` as the page's content. + +## When a document call fails + +Both failure kinds throw with a sentence naming the object and the operation: + +- `` `docx.read` failed: no file at /abs/absent.docx `` — the call reached the format layer + and that layer refused. Wrong path, wrong container, a library rejection. +- `` The `pdf` domain object is not available in this session: … `` — the capability itself is + absent. This is what a catalog preview returns, because a preview has no directory to + resolve a path against and is not a live execution. + +Neither is recoverable by retrying the same call. Report the sentence; do not substitute a +plausible value for a document you could not read. diff --git a/packages/core/src/plugin/skill/skill-writer.md b/packages/core/src/plugin/skill/skill-writer.md new file mode 100644 index 0000000000..40ee892f4b --- /dev/null +++ b/packages/core/src/plugin/skill/skill-writer.md @@ -0,0 +1,104 @@ + + +# Writing a skill + +A skill is one markdown file: a YAML frontmatter block that says what the skill is, and a +body that says what to do. Nothing registers it — the loader finds it by where it sits. + +## Where the file goes + +The loader globs `{*.md,**/SKILL.md}` under each skill directory, so exactly two layouts are +found and no others: + +| layout | path | name comes from | +| --- | --- | --- | +| directory (prefer this) | `.redrob/skill//SKILL.md` | frontmatter `name`, else nothing — a `SKILL.md` with no `name` is dropped | +| single file | `.redrob/skill/.md` | frontmatter `name`, else the filename without `.md` | + +- Project scope: `.redrob/skill/` or `.redrob/skills/`. +- Global scope: `~/.config/redrob/skill/` or `~/.config/redrob/skills/`. + +Use the directory layout and always write a `name`. A skill in a subdirectory whose +frontmatter has no `name` is dropped by the loader with an `unnamed:` reason, because the +path gives it no fallback — that is the single most common way a new skill silently fails to +appear. + +Two skills with the same `name` collide and the later source wins. Check the existing names +before choosing one. + +## The frontmatter + +Only these keys are read. Anything else is ignored, so a misspelled key is not an error — +it is a skill that quietly does not do the thing you meant. + +| key | type | effect | +| --- | --- | --- | +| `name` | string | the skill's identity. Write it. | +| `description` | string | when to use the skill. This is what the model reads to decide. | +| `slash` | boolean | expose it as a slash command. | +| `icon` | string | icon URL, for surfaces that list skills. | +| `autoInject.keywords` | string array | case-insensitive words matched against the prompt on word boundaries. | +| `autoInject.url` | string array | globs matched against the current tab URL. | + +`autoInject` is what arms the skill without the user naming it. A skill with **no** +`autoInject` block never auto-arms and stays explicitly loadable — which is the right choice +for most skills. `autoInject: {}` with both lists absent also never arms: an empty list +matches nothing, deliberately, because a skill that arms on everything is a skill that is +always in the prompt. + +A malformed `autoInject` costs only the arming. The skill still loads, and a warning names +the file and the reason. A malformed frontmatter block drops the whole document. + +## The template + + + +```markdown +--- +name: release-notes +description: Use when drafting release notes from a tag range, or when the user asks what changed between two versions. +slash: true +icon: https://code.redrob.ai/icon/release-notes.svg +autoInject: + keywords: + - release notes + - changelog + url: + - github.com/*/releases/** +--- + +# Drafting release notes + +State what the skill should do, in the order it should be done, and what to refuse. +Write for an agent that has the repository in front of it and no other context. +``` + +Copy that block, replace every value, delete the keys you do not need — `name` and +`description` are the two worth always keeping — and write it to the path chosen above. + +## Writing the body + +The body is the instruction, and it is read by an agent that has nothing else. So: + +- Name the concrete surface: the file, the command, the global, the endpoint. Not "the + relevant config". +- Say what to refuse and what to ask about, not only what to do. +- Every claim about a tool, a global or a flag must be true. A skill that describes a method + that does not exist produces an agent that calls it and fails. +- Keep it short enough to be read whole. A skill is a page, not a manual. + +## After writing + +Skills are loaded when redrob starts and are not hot-reloaded. Tell the user to restart +redrob, then confirm the skill appears. If it does not, the frontmatter was rejected or the +file is outside a skill directory — the startup log names the file and the reason in both +cases. diff --git a/packages/core/test/skill/skill-writer.test.ts b/packages/core/test/skill/skill-writer.test.ts new file mode 100644 index 0000000000..65c97f024e --- /dev/null +++ b/packages/core/test/skill/skill-writer.test.ts @@ -0,0 +1,172 @@ +/** + * K-5: the skill that writes skills, verified by taking the frontmatter template out of the + * skill document itself and putting it through the REAL loader. + * + * The template is not retyped here and it is not asserted by eye. This test reads + * `SkillWriterContent` — the exact string the skill hands to an agent — extracts the fenced + * block the document marks as the template, writes it where the loader globs, and asserts the + * decoded `SkillV2.Info`. So a key the loader does not read, a shape its decoder rejects, or a + * typo in the document's own template fails here. + */ +import fs from "fs/promises" +import path from "path" +import { describe, expect } from "bun:test" +import { Effect, Layer, Schema } from "effect" +import { AgentV2 } from "@redrob-code/core/agent" +import { AppNodeBuilder } from "@redrob-code/core/effect/app-node-builder" +import { LayerNode } from "@redrob-code/core/effect/layer-node" +import { AbsolutePath } from "@redrob-code/core/schema" +import { SkillV2 } from "@redrob-code/core/skill" +import { SkillDiscovery } from "@redrob-code/core/skill/discovery" +import { SkillPlugin } from "@redrob-code/core/plugin/skill" +import { tmpdir } from "../fixture/tmpdir" +import { testEffect } from "../lib/effect" + +const discovery = Layer.succeed( + SkillDiscovery.Service, + SkillDiscovery.Service.of({ pull: () => Effect.succeed([]) }), +) +const it = testEffect( + AppNodeBuilder.build(LayerNode.group([SkillV2.node, AgentV2.node]), [[SkillDiscovery.node, discovery]]), +) + +/** + * The template the document tells an agent to copy. Anchored on the document's own + * `` marker rather than on "the first code fence", so adding an earlier + * example to the document cannot silently repoint this at the wrong block. + * + * `\r` is stripped and the fence match tolerates CRLF: git checks this file out with native + * line endings, so on Windows the document arrives CRLF and an `\n`-only pattern finds no fence + * at all. That failed the Windows core job, which is the only place it could show. + */ +const template = (() => { + const marker = "" + const content = SkillPlugin.SkillWriterContent.replaceAll("\r\n", "\n") + const start = content.indexOf(marker) + if (start < 0) throw new Error("the skill-writer document no longer carries its template marker") + const fence = /```markdown\n([\s\S]*?)```/.exec(content.slice(start)) + if (!fence) throw new Error("no ```markdown fence follows the template marker") + return fence[1]! +})() + +const withTmpdir = (use: (directory: string) => Effect.Effect) => + Effect.acquireRelease( + Effect.promise(() => tmpdir()), + (tmp) => Effect.promise(() => tmp[Symbol.asyncDispose]()), + ).pipe(Effect.flatMap((tmp) => use(tmp.path))) + +/** Writes one document where the loader globs it and returns what the loader made of it. */ +const load = (directory: string, name: string, content: string) => + Effect.gen(function* () { + yield* Effect.promise(async () => { + await fs.mkdir(path.join(directory, name), { recursive: true }) + await fs.writeFile(path.join(directory, name, "SKILL.md"), content) + }) + const skill = yield* SkillV2.Service + yield* skill.transform((editor) => editor.source({ type: "directory", path: AbsolutePath.make(directory) })) + return yield* skill.list() + }) + +describe("K-5 the skill that writes skills", () => { + it.live("the template the document prescribes round-trips through the real loader", () => + withTmpdir((directory) => + Effect.gen(function* () { + const skills = yield* load(directory, "release-notes", template) + expect(skills).toHaveLength(1) + const skill = skills[0]! + + // Every frontmatter key the template declares, decoded by the loader — not read off + // the YAML by this test. + expect(skill.name).toBe("release-notes") + expect(skill.description).toBe( + "Use when drafting release notes from a tag range, or when the user asks what changed between two versions.", + ) + expect(skill.slash).toBe(true) + expect(skill.icon).toBe("https://code.redrob.ai/icon/release-notes.svg") + expect(skill.autoInject).toBeDefined() + expect(skill.autoInject!.keywords).toEqual(["release notes", "changelog"]) + expect(skill.autoInject!.url).toEqual(["github.com/*/releases/**"]) + expect(skill.location).toBe(AbsolutePath.make(path.join(directory, "release-notes", "SKILL.md"))) + expect(skill.content).toContain("# Drafting release notes") + }), + ), + ) + + it.live("the loaded skill re-decodes through the Info schema, so nothing passed by luck", () => + withTmpdir((directory) => + Effect.gen(function* () { + const skills = yield* load(directory, "release-notes", template) + // The loader builds `Info` with `make`, which does not run the decoder over the whole + // value. Putting the loaded skill back through `decodeUnknownSync` does, so a field + // the loader happened to accept in a shape the schema rejects fails here. + const again = Schema.decodeUnknownSync(SkillV2.Info)(JSON.parse(JSON.stringify(skills[0]!))) + expect(again.name).toBe("release-notes") + expect(again.autoInject).toEqual({ keywords: ["release notes", "changelog"], url: ["github.com/*/releases/**"] }) + }), + ), + ) + + it.live("a document missing the loader's one required key is DROPPED, which is what the skill warns about", () => + withTmpdir((directory) => + Effect.gen(function* () { + // The template's own claim under test: a SKILL.md in a subdirectory with no `name` has + // no fallback and does not become a skill. + const skills = yield* load( + directory, + "nameless", + template.replace("name: release-notes\n", ""), + ) + expect(skills).toHaveLength(0) + }), + ), + ) + + it.live("a malformed autoInject costs only the arming, exactly as the document says", () => + withTmpdir((directory) => + Effect.gen(function* () { + const broken = template.replace("autoInject:\n keywords:", "autoInject:\n keywords: not-a-list\n ignored:") + const skills = yield* load(directory, "release-notes", broken) + expect(skills).toHaveLength(1) + expect(skills[0]!.name).toBe("release-notes") + expect(skills[0]!.autoInject).toBeUndefined() + }), + ), + ) +}) + +describe("K-5 the skill document itself", () => { + it.live("every builtin skill decodes through the Info schema the loader uses", () => + Effect.gen(function* () { + expect(SkillPlugin.BuiltinSkills.map((skill) => skill.name)).toStrictEqual([ + "customize-redrob", + "documents", + "skill-writer", + ]) + for (const skill of SkillPlugin.BuiltinSkills) { + // `Info.make` does not decode, so a field in a shape the schema rejects would reach a + // session unnoticed. This runs the real decoder over each shipped skill. + const decoded = Schema.decodeUnknownSync(SkillV2.Info)(JSON.parse(JSON.stringify(skill))) + expect(decoded.name).toBe(skill.name) + expect(decoded.description).toBeTruthy() + expect(decoded.content.length).toBeGreaterThan(200) + } + const documents = SkillPlugin.BuiltinSkills.find((skill) => skill.name === "documents")! + expect(documents.autoInject?.keywords).toContain("docx") + expect(documents.autoInject?.url).toContain("docs.google.com/document/**") + // K-5 deliberately does not auto-arm. + expect(SkillPlugin.BuiltinSkills.find((skill) => skill.name === "skill-writer")!.autoInject).toBeUndefined() + }), + ) + + it.live("is registered as a builtin skill with the keys the loader reads", () => + Effect.gen(function* () { + expect(SkillPlugin.SkillWriterContent).toContain("# Writing a skill") + // The document claims these are the only frontmatter keys read. They are the keys the + // loader's own decoder declares, so a key added to one and not the other is caught. + for (const key of ["name", "description", "slash", "icon", "autoInject"]) { + expect(SkillPlugin.SkillWriterContent).toContain(`\`${key}\``) + } + expect(SkillPlugin.SkillWriterContent).toContain("{*.md,**/SKILL.md}") + }), + ) +}) diff --git a/packages/redrob/package.json b/packages/redrob/package.json index 077caf56eb..1e4b2ea726 100644 --- a/packages/redrob/package.json +++ b/packages/redrob/package.json @@ -117,8 +117,10 @@ "cross-spawn": "catalog:", "decimal.js": "10.5.0", "diff": "catalog:", + "docx": "9.8.1", "drizzle-orm": "catalog:", "effect": "catalog:", + "exceljs": "4.4.0", "fuzzysort": "3.1.0", "gitlab-ai-provider": "6.12.1", "glob": "13.0.5", @@ -136,6 +138,8 @@ "opencode-poe-auth": "0.0.1", "opentui-spinner": "catalog:", "partial-json": "0.1.7", + "pdf-lib": "1.17.1", + "pptxgenjs": "4.0.1", "remeda": "catalog:", "semver": "^7.6.3", "solid-js": "catalog:", diff --git a/packages/redrob/src/tool/code-mode.ts b/packages/redrob/src/tool/code-mode.ts index 162a45b9c9..c922818e19 100644 --- a/packages/redrob/src/tool/code-mode.ts +++ b/packages/redrob/src/tool/code-mode.ts @@ -9,6 +9,9 @@ import { Session } from "@/session/session" import { Permission } from "@/permission" import { Plugin } from "@/plugin" import { DOMAIN_GLOBALS, domainTools, sessionChannel, unavailableChannel, unavailablePage } from "./domain" +import * as Documents from "./document" +import { InstanceRef } from "@/effect/instance-ref" +import { assertExternalDirectoryEffect } from "./external-directory" export const CODE_MODE_TOOL = "execute" @@ -225,6 +228,34 @@ export const CodeModeTool = Tool.define( // refuses by name rather than returning a plausible value. const page = unavailablePage() const channel = sessionChannel({ sessions, sessionID: ctx.sessionID, messageID: ctx.messageID }) + // K-3 document objects. Unlike `page`, these DO work in the engine process: a write + // lands a real file. The guard is the same external-directory check the `write` tool + // uses, so a program cannot drop a document outside the instance directory without + // the user being asked. + // + // `InstanceRef` is read directly rather than through `InstanceState.context`, which + // DIES when no instance is bound. A code-mode execution with no instance is a real + // configuration — the tool is reachable from harnesses that bind none — and the right + // answer there is to resolve relative paths against the process directory, not to + // kill the tool over a document global the program may never touch. + const instance = yield* InstanceRef + const documents = Documents.documents({ + directory: instance?.directory ?? process.cwd(), + guard: (filepath) => + assertExternalDirectoryEffect(ctx, filepath).pipe( + Effect.asVoid, + Effect.catchCause((cause) => { + const error = Cause.squash(cause) + return Effect.fail( + new Documents.DocumentFailureError({ + object: "document", + operation: "write", + reason: `writing ${filepath} was not permitted: ${error instanceof Error ? error.message : String(error)}`, + }), + ) + }), + ), + }) const publish = () => ctx.metadata({ title: CODE_MODE_TOOL, metadata: { toolCalls: calls.map((c) => ({ ...c })) } }) @@ -252,7 +283,7 @@ export const CodeModeTool = Tool.define( // Domain namespaces are spread last deliberately: a connected MCP server named // `page` must not shadow the typed domain object a skill document is written // against. The domain objects are part of the language; the MCP catalog is not. - tools: { ...toolTree(catalog, callTool), ...domainTools({ page, channel }) }, + tools: { ...toolTree(catalog, callTool), ...domainTools({ page, channel, documents }) }, globals: DOMAIN_GLOBALS, onToolCallStart: ({ index, name, input }) => Effect.suspend(() => { diff --git a/packages/redrob/src/tool/document.ts b/packages/redrob/src/tool/document.ts new file mode 100644 index 0000000000..563d3e4dd6 --- /dev/null +++ b/packages/redrob/src/tool/document.ts @@ -0,0 +1,930 @@ +/** + * K-3: a real document toolchain in code mode — `docx`, `xlsx`, `pptx` and `pdf` as typed + * globals in the same shape as K-1's `page`, each with a read half and a write half. + * + * The rule this module exists to keep: **a write produces a real file or it fails.** Every + * `write` below serialises through a format library and lands bytes on disk at the path the + * program asked for, and the result carries the byte count read back from that file. None of + * them returns a stringified summary, and none of them reports success for a file it did not + * write. Where a capability is genuinely absent the implementation THROWS and names what is + * missing, exactly as `unavailablePage` does — see `unavailableDocuments`. + * + * Honest limits, stated here because they are properties of the formats and not bugs: + * + * - **`pdf.read` text is extracted from content streams, not rendered.** A page whose text is + * drawn with a composite (`Type0`) font carries glyph identifiers rather than characters, so + * that page's runs are SKIPPED and the page is returned with `textComplete: false` and a + * reason naming the font. A scanned page has no text operators at all and is reported the + * same way — there is no OCR in the engine process. Returning glyph soup as if it were text + * would be the fake-success this module refuses. + * - **Simple-font text is decoded as Latin-1**, which equals WinAnsi for every codepoint the + * standard fonts place below 0x80 and differs only in the 0x80–0x9F band. + * - OOXML reads (`docx`, `pptx`) take the text of the document part. They do not reconstruct + * styling, and `docx.read` does not descend into headers, footers or footnotes. + */ +import * as path from "path" +import * as fs from "node:fs/promises" +import { inflateSync } from "node:zlib" +import { Effect, Schema } from "effect" +import * as zip from "@zip.js/zip.js" +import { Tool as SandboxTool, toolError } from "@redrob-code/codemode" +import { DomainUnavailableError } from "./domain-error" + +/** + * A document operation that reached the format layer and failed there: a missing file, a file + * that is not the container it claims to be, a library rejection. Distinct from + * `DomainUnavailableError`, which says the capability is not present at all. + */ +export class DocumentFailureError extends Schema.TaggedErrorClass()( + "CodeModeDocumentFailure", + { object: Schema.String, operation: Schema.String, reason: Schema.String }, +) { + override get message() { + return `\`${this.object}.${this.operation}\` failed: ${this.reason}` + } +} + +/** Every document call fails with one of these two, and with nothing else. */ +export type DocumentError = DocumentFailureError | DomainUnavailableError + +const fail = (object: string, operation: string, reason: string) => + Effect.fail(new DocumentFailureError({ object, operation, reason })) + +const attempt = (object: string, operation: string, run: () => Promise | A) => + Effect.tryPromise({ + try: async () => await run(), + catch: (cause) => + new DocumentFailureError({ + object, + operation, + reason: cause instanceof Error ? cause.message : String(cause), + }), + }) + +/* ------------------------------------------------------------------------------------------ */ +/* The host: where a path resolves to and who is allowed to be asked about it */ +/* ------------------------------------------------------------------------------------------ */ + +/** + * What the four objects need from the process they run in. Injected rather than imported so a + * test drives real files in a temp directory and the preview path can supply nothing at all. + * + * `guard` is the permission hook: the engine wires it to the same external-directory check the + * `write` tool uses, so a code-mode program cannot write a document outside the instance + * directory without the user being asked. + */ +export interface DocumentHost { + /** Directory a relative path is resolved against. */ + readonly directory: string + /** Called with the resolved absolute path before any write. */ + readonly guard?: (filepath: string) => Effect.Effect +} + +const resolveRead = (host: DocumentHost, object: string, operation: string, filepath: string) => + Effect.gen(function* () { + if (filepath.trim().length === 0) return yield* fail(object, operation, "an empty path is not a file") + return path.isAbsolute(filepath) ? filepath : path.join(host.directory, filepath) + }) + +const resolveWrite = (host: DocumentHost, object: string, operation: string, filepath: string) => + Effect.gen(function* () { + const absolute = yield* resolveRead(host, object, operation, filepath) + if (host.guard) yield* host.guard(absolute) + return absolute + }) + +const readBytes = (object: string, operation: string, absolute: string) => + Effect.gen(function* () { + const stat = yield* Effect.tryPromise({ + try: () => fs.stat(absolute), + catch: () => new DocumentFailureError({ object, operation, reason: `no file at ${absolute}` }), + }) + if (!stat.isFile()) return yield* fail(object, operation, `${absolute} is not a file`) + const buffer = yield* attempt(object, operation, () => fs.readFile(absolute)) + return new Uint8Array(buffer) + }) + +/** + * Writes the bytes, then STATS THE FILE and returns the size the filesystem reports. The + * returned byte count is therefore a reading of the artefact, not of the buffer we hoped to + * write — which is the difference between a verified write and a claimed one. + */ +const writeBytes = (object: string, operation: string, absolute: string, bytes: Uint8Array) => + Effect.gen(function* () { + yield* attempt(object, operation, () => fs.mkdir(path.dirname(absolute), { recursive: true })) + yield* attempt(object, operation, () => fs.writeFile(absolute, bytes)) + const stat = yield* attempt(object, operation, () => fs.stat(absolute)) + if (stat.size === 0) return yield* fail(object, operation, `wrote ${absolute} but it is empty`) + return { path: absolute, bytes: stat.size } + }) + +/* ------------------------------------------------------------------------------------------ */ +/* OOXML: one zip reader for docx and pptx */ +/* ------------------------------------------------------------------------------------------ */ + +// Web workers would outlive the test process and keep it alive; the archives here are small +// enough that inline inflation costs nothing. +zip.configure({ useWebWorkers: false }) + +/** Reads the named parts out of an OOXML package. Returns only the parts that exist. */ +const readParts = (object: string, operation: string, bytes: Uint8Array, wanted: (name: string) => boolean) => + attempt(object, operation, async () => { + const reader = new zip.ZipReader(new zip.Uint8ArrayReader(bytes)) + try { + const entries = await reader.getEntries() + if (entries.length === 0) throw new Error("not an OOXML package: the archive has no entries") + const parts: Array<{ name: string; xml: string }> = [] + for (const entry of entries) { + if (entry.directory || !wanted(entry.filename) || !entry.getData) continue + parts.push({ name: entry.filename, xml: await entry.getData(new zip.TextWriter()) }) + } + return parts + } finally { + await reader.close() + } + }) + +const XML_ENTITIES: Record = { amp: "&", lt: "<", gt: ">", quot: '"', apos: "'" } + +/** Decodes the XML entities an OOXML text node can carry, including numeric references. */ +const decodeEntities = (text: string) => + text.replace(/&(#x?[0-9a-fA-F]+|[a-zA-Z]+);/g, (whole, body: string) => { + if (body.startsWith("#x") || body.startsWith("#X")) { + const code = Number.parseInt(body.slice(2), 16) + return Number.isFinite(code) ? String.fromCodePoint(code) : whole + } + if (body.startsWith("#")) { + const code = Number.parseInt(body.slice(1), 10) + return Number.isFinite(code) ? String.fromCodePoint(code) : whole + } + return XML_ENTITIES[body] ?? whole + }) + +/** The text of one OOXML run container, in document order, with tabs and breaks honoured. */ +const runText = (fragment: string, textTag: string) => { + let out = "" + const pattern = new RegExp(`<${textTag}(?:\\s[^>]*)?>([\\s\\S]*?)|<(?:w|a):(tab|br)\\s*/>`, "g") + for (const match of fragment.matchAll(pattern)) { + if (match[1] !== undefined) out += decodeEntities(match[1]) + else if (match[2] === "tab") out += "\t" + else out += "\n" + } + return out +} + +/* ------------------------------------------------------------------------------------------ */ +/* The four interfaces */ +/* ------------------------------------------------------------------------------------------ */ + +/** One paragraph a `docx.write` lays down. `heading` 1–6 renders as that heading level. */ +export interface DocxParagraph { + readonly text: string + readonly heading?: number + readonly bold?: boolean +} + +export interface DocxResult { + readonly path: string + readonly bytes: number +} + +/** Word documents. */ +export interface Docx { + /** Paragraph text of the document body, in document order. */ + readonly read: (input: { + readonly path: string + }) => Effect.Effect<{ readonly paragraphs: ReadonlyArray; readonly text: string }, DocumentError> + /** Writes a real `.docx` package and returns the path and the size on disk. */ + readonly write: (input: { + readonly path: string + readonly title?: string + readonly paragraphs: ReadonlyArray + }) => Effect.Effect +} + +/** A spreadsheet cell as it crosses the sandbox boundary. */ +export type CellValue = string | number | boolean | null + +export interface XlsxSheet { + readonly name: string + readonly rows: ReadonlyArray> +} + +/** Excel workbooks. */ +export interface Xlsx { + /** Every sheet, or one named sheet, as rows of plain JSON cell values. */ + readonly read: (input: { + readonly path: string + readonly sheet?: string + }) => Effect.Effect<{ readonly sheets: ReadonlyArray }, DocumentError> + /** Writes a real `.xlsx` workbook and returns the path and the size on disk. */ + readonly write: (input: { + readonly path: string + readonly sheets: ReadonlyArray + }) => Effect.Effect +} + +export interface PptxSlideRead { + readonly index: number + readonly text: ReadonlyArray +} + +export interface PptxSlideWrite { + readonly title?: string + readonly bullets?: ReadonlyArray + readonly notes?: string +} + +/** PowerPoint decks. */ +export interface Pptx { + /** The text of every slide, slides in deck order. */ + readonly read: (input: { + readonly path: string + }) => Effect.Effect<{ readonly slides: ReadonlyArray }, DocumentError> + /** Writes a real `.pptx` package and returns the path and the size on disk. */ + readonly write: (input: { + readonly path: string + readonly slides: ReadonlyArray + }) => Effect.Effect +} + +/** + * One page of a `pdf.read`. `textComplete` is the honest field: false means some or all of the + * page's text could not be recovered as characters, and `reason` says why. + */ +export interface PdfPageRead { + readonly index: number + readonly text: string + readonly textComplete: boolean + readonly reason?: string +} + +/** PDF documents. */ +export interface Pdf { + /** Page count, document metadata, and per-page text with its completeness stated. */ + readonly read: (input: { readonly path: string }) => Effect.Effect< + { + readonly pageCount: number + readonly title?: string + readonly author?: string + readonly pages: ReadonlyArray + }, + DocumentError + > + /** Writes a real `.pdf` and returns the path and the size on disk. */ + readonly write: (input: { + readonly path: string + readonly title?: string + readonly fontSize?: number + readonly pages: ReadonlyArray<{ readonly lines: ReadonlyArray }> + }) => Effect.Effect +} + +/** The four objects as one bundle, so the globals and the implementations cannot drift. */ +export interface Documents { + readonly docx: Docx + readonly xlsx: Xlsx + readonly pptx: Pptx + readonly pdf: Pdf +} + +/* ------------------------------------------------------------------------------------------ */ +/* docx */ +/* ------------------------------------------------------------------------------------------ */ + +const makeDocx = (host: DocumentHost): Docx => ({ + read: ({ path: filepath }) => + Effect.gen(function* () { + const absolute = yield* resolveRead(host, "docx", "read", filepath) + const bytes = yield* readBytes("docx", "read", absolute) + const parts = yield* readParts("docx", "read", bytes, (name) => name === "word/document.xml") + const body = parts[0] + if (!body) + return yield* fail("docx", "read", `${absolute} has no word/document.xml, so it is not a Word document`) + const paragraphs: string[] = [] + for (const match of body.xml.matchAll(/]*)?>([\s\S]*?)<\/w:p>/g)) { + paragraphs.push(runText(match[1] ?? "", "w:t")) + } + return { paragraphs, text: paragraphs.join("\n") } + }), + write: ({ path: filepath, title, paragraphs }) => + Effect.gen(function* () { + const absolute = yield* resolveWrite(host, "docx", "write", filepath) + if (paragraphs.length === 0 && title === undefined) + return yield* fail("docx", "write", "nothing to write: give at least a title or one paragraph") + const bytes = yield* attempt("docx", "write", async () => { + const { Document, HeadingLevel, Packer, Paragraph, TextRun } = await import("docx") + const levels = [ + HeadingLevel.HEADING_1, + HeadingLevel.HEADING_2, + HeadingLevel.HEADING_3, + HeadingLevel.HEADING_4, + HeadingLevel.HEADING_5, + HeadingLevel.HEADING_6, + ] + const children = [ + ...(title === undefined ? [] : [new Paragraph({ text: title, heading: HeadingLevel.TITLE })]), + ...paragraphs.map((paragraph) => { + const level = paragraph.heading === undefined ? undefined : levels[Math.min(6, Math.max(1, paragraph.heading)) - 1] + return new Paragraph({ + children: [new TextRun({ text: paragraph.text, bold: paragraph.bold === true })], + ...(level ? { heading: level } : {}), + }) + }), + ] + const document = new Document({ sections: [{ children }] }) + return new Uint8Array(await Packer.toBuffer(document)) + }) + return yield* writeBytes("docx", "write", absolute, bytes) + }), +}) + +/* ------------------------------------------------------------------------------------------ */ +/* xlsx */ +/* ------------------------------------------------------------------------------------------ */ + +/** Projects one ExcelJS cell value onto the JSON shape the sandbox boundary allows. */ +const cellValue = (value: unknown): CellValue => { + if (value === null || value === undefined) return null + if (typeof value === "string" || typeof value === "number" || typeof value === "boolean") return value + if (value instanceof Date) return value.toISOString() + const record = value as Record + // A formula cell carries its cached result; a cell with no cached result is honestly null + // rather than the formula text, which is not the cell's value. + if ("result" in record) return cellValue(record["result"]) + if ("text" in record && typeof record["text"] === "string") return record["text"] + if ("richText" in record && Array.isArray(record["richText"])) + return (record["richText"] as Array<{ text?: string }>).map((run) => run.text ?? "").join("") + if ("error" in record) return String(record["error"]) + if ("hyperlink" in record && typeof record["text"] === "string") return record["text"] + return null +} + +const makeXlsx = (host: DocumentHost): Xlsx => ({ + read: ({ path: filepath, sheet }) => + Effect.gen(function* () { + const absolute = yield* resolveRead(host, "xlsx", "read", filepath) + const bytes = yield* readBytes("xlsx", "read", absolute) + const sheets = yield* attempt("xlsx", "read", async () => { + const { default: ExcelJS } = await import("exceljs") + const workbook = new ExcelJS.Workbook() + await workbook.xlsx.load(bytes as unknown as ArrayBuffer) + const out: XlsxSheet[] = [] + workbook.eachSheet((worksheet) => { + if (sheet !== undefined && worksheet.name !== sheet) return + const rows: CellValue[][] = [] + worksheet.eachRow({ includeEmpty: true }, (row) => { + // `row.values` is 1-indexed with a hole at 0; drop it rather than emitting a null + // column nobody asked for. + const values = Array.isArray(row.values) ? row.values.slice(1) : [] + rows.push(values.map(cellValue)) + }) + out.push({ name: worksheet.name, rows }) + }) + return out + }) + if (sheet !== undefined && sheets.length === 0) + return yield* fail("xlsx", "read", `${absolute} has no sheet named ${sheet}`) + return { sheets } + }), + write: ({ path: filepath, sheets }) => + Effect.gen(function* () { + const absolute = yield* resolveWrite(host, "xlsx", "write", filepath) + if (sheets.length === 0) return yield* fail("xlsx", "write", "a workbook needs at least one sheet") + const bytes = yield* attempt("xlsx", "write", async () => { + const { default: ExcelJS } = await import("exceljs") + const workbook = new ExcelJS.Workbook() + for (const sheet of sheets) { + const worksheet = workbook.addWorksheet(sheet.name) + for (const row of sheet.rows) worksheet.addRow([...row]) + } + const written = await workbook.xlsx.writeBuffer() + return new Uint8Array(written as ArrayBuffer) + }) + return yield* writeBytes("xlsx", "write", absolute, bytes) + }), +}) + +/* ------------------------------------------------------------------------------------------ */ +/* pptx */ +/* ------------------------------------------------------------------------------------------ */ + +const SLIDE_PART = /^ppt\/slides\/slide(\d+)\.xml$/ + +const makePptx = (host: DocumentHost): Pptx => ({ + read: ({ path: filepath }) => + Effect.gen(function* () { + const absolute = yield* resolveRead(host, "pptx", "read", filepath) + const bytes = yield* readBytes("pptx", "read", absolute) + const parts = yield* readParts("pptx", "read", bytes, (name) => SLIDE_PART.test(name)) + if (parts.length === 0) + return yield* fail("pptx", "read", `${absolute} has no ppt/slides parts, so it is not a PowerPoint deck`) + // Deck order is numeric, not lexical: slide10 comes after slide9, which a string sort + // gets wrong for any deck past nine slides. + const ordered = parts + .map((part) => ({ part, number: Number.parseInt(SLIDE_PART.exec(part.name)![1]!, 10) })) + .sort((a, b) => a.number - b.number) + return { + slides: ordered.map(({ part }, index) => { + const text: string[] = [] + for (const match of part.xml.matchAll(/]*)?>([\s\S]*?)<\/a:p>/g)) { + const line = runText(match[1] ?? "", "a:t") + if (line.length > 0) text.push(line) + } + return { index, text } + }), + } + }), + write: ({ path: filepath, slides }) => + Effect.gen(function* () { + const absolute = yield* resolveWrite(host, "pptx", "write", filepath) + if (slides.length === 0) return yield* fail("pptx", "write", "a deck needs at least one slide") + const bytes = yield* attempt("pptx", "write", async () => { + const { default: PptxGenJS } = await import("pptxgenjs") + const deck = new PptxGenJS() + for (const input of slides) { + const slide = deck.addSlide() + if (input.title !== undefined) + slide.addText(input.title, { x: 0.5, y: 0.4, w: 9, h: 0.8, fontSize: 28, bold: true }) + const bullets = input.bullets ?? [] + if (bullets.length > 0) + slide.addText( + bullets.map((text) => ({ text, options: { bullet: true, fontSize: 16, breakLine: true } })), + { x: 0.7, y: 1.5, w: 8.6, h: 4.5 }, + ) + if (input.notes !== undefined) slide.addNotes(input.notes) + } + const written = await deck.write({ outputType: "nodebuffer" }) + return new Uint8Array(written as ArrayBuffer) + }) + return yield* writeBytes("pptx", "write", absolute, bytes) + }), +}) + +/* ------------------------------------------------------------------------------------------ */ +/* pdf */ +/* ------------------------------------------------------------------------------------------ */ + +/** A PDF literal or hex string, read from a content stream as raw byte codes. */ +const readPdfString = (source: string, start: number): { value: string; next: number } | undefined => { + if (source[start] === "(") { + let depth = 1 + let index = start + 1 + let out = "" + while (index < source.length) { + const char = source[index]! + if (char === "\\") { + const escaped = source[index + 1] + index += 2 + switch (escaped) { + case "n": + out += "\n" + break + case "r": + out += "\r" + break + case "t": + out += "\t" + break + case "b": + case "f": + break + case "\n": + break + default: + if (escaped !== undefined && escaped >= "0" && escaped <= "7") { + let octal = escaped + while (octal.length < 3 && source[index] !== undefined && source[index]! >= "0" && source[index]! <= "7") { + octal += source[index]! + index += 1 + } + out += String.fromCharCode(Number.parseInt(octal, 8)) + } else if (escaped !== undefined) out += escaped + } + continue + } + if (char === "(") depth += 1 + if (char === ")") { + depth -= 1 + if (depth === 0) return { value: out, next: index + 1 } + } + out += char + index += 1 + } + return undefined + } + if (source[start] === "<" && source[start + 1] !== "<") { + const end = source.indexOf(">", start + 1) + if (end < 0) return undefined + const hex = source.slice(start + 1, end).replace(/[^0-9a-fA-F]/g, "") + let out = "" + for (let index = 0; index + 1 < hex.length; index += 2) + out += String.fromCharCode(Number.parseInt(hex.slice(index, index + 2), 16)) + return { value: out, next: end + 1 } + } + return undefined +} + +type Operand = { readonly kind: "string"; readonly value: string } | { readonly kind: "name"; readonly value: string } | { readonly kind: "other" } + +/** + * Walks a decoded content stream and returns the text it shows, plus the names of any fonts + * whose runs were skipped because their codes are not characters. + * + * The scan is an operand stack cleared at every operator, which is what a PDF content stream + * is: postfix operands followed by an operator. `Tf` sets the current font, and a run drawn in + * a font this function cannot decode is dropped and recorded rather than guessed at. + */ +export const extractPdfText = ( + content: string, + decodable: (font: string) => boolean, +): { text: string; skipped: ReadonlyArray } => { + const lines: string[] = [] + let current = "" + let font: string | undefined + const skipped = new Set() + let operands: Operand[] = [] + let index = 0 + + const breakLine = () => { + lines.push(current) + current = "" + } + const show = (value: string) => { + if (font !== undefined && !decodable(font)) { + skipped.add(font) + return + } + current += value + } + + while (index < content.length) { + const char = content[index]! + if (char === "(" || (char === "<" && content[index + 1] !== "<")) { + const read = readPdfString(content, index) + if (!read) break + operands.push({ kind: "string", value: read.value }) + index = read.next + continue + } + if (char === "<" || char === ">") { + index += content[index + 1] === char ? 2 : 1 + continue + } + if (char === "/") { + const match = /^\/([^\s/[\]<>(){}]*)/.exec(content.slice(index))! + operands.push({ kind: "name", value: match[1] ?? "" }) + index += match[0].length + continue + } + if (char === "[" || char === "]") { + index += 1 + continue + } + if (/\s/.test(char)) { + index += 1 + continue + } + const token = /^[^\s/[\]<>(){}]+/.exec(content.slice(index))?.[0] + if (!token) { + index += 1 + continue + } + index += token.length + if (/^[-+.\d]/.test(token)) { + operands.push({ kind: "other" }) + continue + } + switch (token) { + case "Tf": { + const name = operands.findLast((operand) => operand.kind === "name") + if (name && name.kind === "name") font = name.value + break + } + case "Tj": + case "TJ": { + for (const operand of operands) if (operand.kind === "string") show(operand.value) + break + } + case "'": + case '"': { + breakLine() + for (const operand of operands) if (operand.kind === "string") show(operand.value) + break + } + case "Td": + case "TD": + case "T*": + case "TL": + case "Tm": + case "ET": + if (current.length > 0) breakLine() + break + default: + break + } + operands = [] + } + if (current.length > 0) lines.push(current) + return { text: lines.join("\n"), skipped: [...skipped] } +} + +const makePdf = (host: DocumentHost): Pdf => ({ + read: ({ path: filepath }) => + Effect.gen(function* () { + const absolute = yield* resolveRead(host, "pdf", "read", filepath) + const bytes = yield* readBytes("pdf", "read", absolute) + return yield* attempt("pdf", "read", async () => { + const { PDFArray, PDFDict, PDFDocument, PDFName, PDFRawStream } = await import("pdf-lib") + const document = await PDFDocument.load(bytes, { updateMetadata: false }) + const pages: PdfPageRead[] = [] + document.getPages().forEach((page, index) => { + // A composite font's string bytes are glyph ids in the font's own space, so they are + // not text and are not treated as text. Everything else is a simple font whose codes + // are character codes. + const composite = new Set() + const resources = page.node.get(PDFName.of("Resources")) + const fonts = + resources instanceof PDFDict ? document.context.lookupMaybe(resources.get(PDFName.of("Font")), PDFDict) : undefined + if (fonts) + for (const [name, ref] of fonts.entries()) { + const font = document.context.lookupMaybe(ref, PDFDict) + const subtype = font?.get(PDFName.of("Subtype")) + if (subtype !== undefined && String(subtype) === "/Type0") composite.add(name.asString().slice(1)) + } + + const contents = page.node.get(PDFName.of("Contents")) + const refs = + contents instanceof PDFArray + ? contents.asArray() + : contents === undefined + ? [] + : [contents] + let decoded = "" + for (const ref of refs) { + const looked = document.context.lookup(ref) + if (!(looked instanceof PDFRawStream)) continue + const stream = looked + const raw = Buffer.from(stream.getContents()) + const filter = stream.dict.get(PDFName.of("Filter")) + const flate = filter !== undefined && String(filter).includes("FlateDecode") + decoded += flate ? inflateSync(raw).toString("latin1") : raw.toString("latin1") + } + if (decoded.length === 0) { + pages.push({ + index, + text: "", + textComplete: false, + reason: + "the page has no readable content stream: it is an image-only or scanned page, and the engine process has no OCR", + }) + return + } + const { text, skipped } = extractPdfText(decoded, (font) => !composite.has(font)) + if (skipped.length > 0) { + pages.push({ + index, + text, + textComplete: false, + reason: `text drawn with composite (Type0) font${skipped.length === 1 ? "" : "s"} ${skipped.join(", ")} was skipped: those strings are glyph identifiers, not characters, and no ToUnicode mapping is applied`, + }) + return + } + if (text.length === 0) { + pages.push({ + index, + text: "", + textComplete: false, + reason: "the content stream shows no text: the page draws only graphics", + }) + return + } + pages.push({ index, text, textComplete: true }) + }) + const title = document.getTitle() + const author = document.getAuthor() + return { + pageCount: document.getPageCount(), + ...(title ? { title } : {}), + ...(author ? { author } : {}), + pages, + } + }) + }), + write: ({ path: filepath, title, fontSize, pages }) => + Effect.gen(function* () { + const absolute = yield* resolveWrite(host, "pdf", "write", filepath) + if (pages.length === 0) return yield* fail("pdf", "write", "a PDF needs at least one page") + const size = fontSize ?? 12 + if (!Number.isFinite(size) || size <= 0) return yield* fail("pdf", "write", `fontSize ${fontSize} is not a size`) + const bytes = yield* attempt("pdf", "write", async () => { + const { PDFDocument, StandardFonts } = await import("pdf-lib") + const document = await PDFDocument.create() + if (title !== undefined) document.setTitle(title) + const font = await document.embedFont(StandardFonts.Helvetica) + const [width, height] = [595.28, 841.89] // A4 at 72dpi, the size a reader expects + const margin = 56 + const leading = size * 1.45 + for (const input of pages) { + let page = document.addPage([width, height]) + let cursor = height - margin + for (const line of input.lines) { + // A page that runs out of room continues onto a new one rather than silently + // dropping the rest of the text. + if (cursor < margin) { + page = document.addPage([width, height]) + cursor = height - margin + } + // WinAnsi is what the standard 14 fonts encode; a character outside it would make + // pdf-lib throw, so it is replaced rather than failing the whole write. + const drawable = [...line].map((char) => (char.codePointAt(0)! > 0xff ? "?" : char)).join("") + page.drawText(drawable, { x: margin, y: cursor, size, font }) + cursor -= leading + } + } + return new Uint8Array(await document.save()) + }) + return yield* writeBytes("pdf", "write", absolute, bytes) + }), +}) + +/* ------------------------------------------------------------------------------------------ */ +/* The two implementations: the real one, and the one that refuses */ +/* ------------------------------------------------------------------------------------------ */ + +/** The working implementation, bound to a directory and an optional permission guard. */ +export const documents = (host: DocumentHost): Documents => ({ + docx: makeDocx(host), + xlsx: makeXlsx(host), + pptx: makePptx(host), + pdf: makePdf(host), +}) + +/** + * The refusing implementation, for the catalog preview — which describes the globals without + * being a live execution and so has no directory to resolve a path against. Same discipline as + * `unavailablePage`: every method fails with the reason named, and none of them returns a + * plausible value or a fake success. + */ +export const unavailableDocuments = (missing = "this is a catalog preview, not a live execution"): Documents => { + const refuse = (object: string) => + Effect.fail(new DomainUnavailableError({ object, missing })) as Effect.Effect + return { + docx: { read: () => refuse("docx"), write: () => refuse("docx") }, + xlsx: { read: () => refuse("xlsx"), write: () => refuse("xlsx") }, + pptx: { read: () => refuse("pptx"), write: () => refuse("pptx") }, + pdf: { read: () => refuse("pdf"), write: () => refuse("pdf") }, + } +} + +/* ------------------------------------------------------------------------------------------ */ +/* Code Mode namespaces */ +/* ------------------------------------------------------------------------------------------ */ + +const PathInput = Schema.String.annotate({ + description: "Path to the file. Relative paths resolve against the instance directory.", +}) + +const WriteResult = Schema.Struct({ + path: Schema.String, + bytes: Schema.Number, +}).annotate({ description: "The absolute path written and the size the filesystem reports for it." }) + +const CellSchema = Schema.Union([Schema.String, Schema.Number, Schema.Boolean, Schema.Null]) + +const SheetSchema = Schema.Struct({ + name: Schema.String.annotate({ description: "Sheet name." }), + rows: Schema.Array(Schema.Array(CellSchema)).annotate({ description: "Rows of cell values, top-left first." }), +}) + +/** + * Lifts a document call into a Code Mode tool. Both error kinds carry a sentence, so an + * unavailable object and a format failure read as precise refusals in the program rather than + * as an opaque execution failure. + */ +const lift = + (call: (input: I) => Effect.Effect) => + (input: I) => + call(input).pipe(Effect.mapError((error) => toolError(error.message, error))) + +export const docxTools = (docx: Docx) => ({ + read: SandboxTool.make({ + description: "Paragraph text of a Word document, in document order.", + input: Schema.Struct({ path: PathInput }), + output: Schema.Struct({ paragraphs: Schema.Array(Schema.String), text: Schema.String }), + run: lift((input: { path: string }) => docx.read(input)), + }), + write: SandboxTool.make({ + description: "Write a real .docx file. Returns the path written and its size on disk.", + input: Schema.Struct({ + path: PathInput, + title: Schema.optionalKey(Schema.String.annotate({ description: "Document title paragraph." })), + paragraphs: Schema.Array( + Schema.Struct({ + text: Schema.String, + heading: Schema.optionalKey(Schema.Number.annotate({ description: "Heading level 1-6." })), + bold: Schema.optionalKey(Schema.Boolean), + }), + ), + }), + output: WriteResult, + run: lift( + (input: { path: string; title?: string; paragraphs: ReadonlyArray }) => docx.write(input), + ), + }), +}) + +export const xlsxTools = (xlsx: Xlsx) => ({ + read: SandboxTool.make({ + description: "Rows of an Excel workbook as plain cell values, every sheet or one named sheet.", + input: Schema.Struct({ + path: PathInput, + sheet: Schema.optionalKey(Schema.String.annotate({ description: "Read only this sheet." })), + }), + output: Schema.Struct({ sheets: Schema.Array(SheetSchema) }), + run: lift((input: { path: string; sheet?: string }) => xlsx.read(input)), + }), + write: SandboxTool.make({ + description: "Write a real .xlsx workbook. Returns the path written and its size on disk.", + input: Schema.Struct({ path: PathInput, sheets: Schema.Array(SheetSchema) }), + output: WriteResult, + run: lift((input: { path: string; sheets: ReadonlyArray }) => xlsx.write(input)), + }), +}) + +export const pptxTools = (pptx: Pptx) => ({ + read: SandboxTool.make({ + description: "The text of every slide in a PowerPoint deck, in deck order.", + input: Schema.Struct({ path: PathInput }), + output: Schema.Struct({ + slides: Schema.Array(Schema.Struct({ index: Schema.Number, text: Schema.Array(Schema.String) })), + }), + run: lift((input: { path: string }) => pptx.read(input)), + }), + write: SandboxTool.make({ + description: "Write a real .pptx deck. Returns the path written and its size on disk.", + input: Schema.Struct({ + path: PathInput, + slides: Schema.Array( + Schema.Struct({ + title: Schema.optionalKey(Schema.String), + bullets: Schema.optionalKey(Schema.Array(Schema.String)), + notes: Schema.optionalKey(Schema.String.annotate({ description: "Speaker notes." })), + }), + ), + }), + output: WriteResult, + run: lift((input: { path: string; slides: ReadonlyArray }) => pptx.write(input)), + }), +}) + +export const pdfTools = (pdf: Pdf) => ({ + read: SandboxTool.make({ + description: + "Page count, metadata and per-page text of a PDF. `textComplete` is false with a reason where text could not be recovered as characters.", + input: Schema.Struct({ path: PathInput }), + output: Schema.Struct({ + pageCount: Schema.Number, + title: Schema.optionalKey(Schema.String), + author: Schema.optionalKey(Schema.String), + pages: Schema.Array( + Schema.Struct({ + index: Schema.Number, + text: Schema.String, + textComplete: Schema.Boolean, + reason: Schema.optionalKey(Schema.String), + }), + ), + }), + run: lift((input: { path: string }) => pdf.read(input)), + }), + write: SandboxTool.make({ + description: "Write a real .pdf. Lines are laid out in Helvetica and overflow onto new pages.", + input: Schema.Struct({ + path: PathInput, + title: Schema.optionalKey(Schema.String), + fontSize: Schema.optionalKey(Schema.Number.annotate({ description: "Point size, default 12." })), + pages: Schema.Array(Schema.Struct({ lines: Schema.Array(Schema.String) })), + }), + output: WriteResult, + run: lift( + (input: { + path: string + title?: string + fontSize?: number + pages: ReadonlyArray<{ lines: ReadonlyArray }> + }) => pdf.write(input), + ), + }), +}) + +/** The global names K-3 adds, in the order they are declared. */ +export const DOCUMENT_GLOBALS = ["docx", "pdf", "pptx", "xlsx"] as const + +/** The four namespaces keyed by the global each is bound to. */ +export const documentTools = (input: Documents) => ({ + docx: docxTools(input.docx), + pdf: pdfTools(input.pdf), + pptx: pptxTools(input.pptx), + xlsx: xlsxTools(input.xlsx), +}) diff --git a/packages/redrob/src/tool/domain-error.ts b/packages/redrob/src/tool/domain-error.ts new file mode 100644 index 0000000000..43a2fd3274 --- /dev/null +++ b/packages/redrob/src/tool/domain-error.ts @@ -0,0 +1,23 @@ +/** + * The one error both K-1's domain objects and K-3's document objects raise when the capability + * itself is absent. + * + * It lives in its own module ON PURPOSE. `domain.ts` binds K-3's globals into `DOMAIN_GLOBALS` + * at module scope, and `document.ts` needs this class — so importing it from `domain.ts` would + * make the two modules a cycle whose evaluation order decides whether `DOCUMENT_GLOBALS` is + * defined when `domain.ts` reads it. A leaf module has no such order to get wrong. + */ +import { Schema } from "effect" + +/** + * A domain object the running engine cannot reach. Carries the missing capability by name + * so the model, the logs, and a person reading a transcript all see the same reason. + */ +export class DomainUnavailableError extends Schema.TaggedErrorClass()( + "CodeModeDomainUnavailable", + { object: Schema.String, missing: Schema.String }, +) { + override get message() { + return `The \`${this.object}\` domain object is not available in this session: ${this.missing}` + } +} diff --git a/packages/redrob/src/tool/domain.ts b/packages/redrob/src/tool/domain.ts index 165a9aa9a8..409b72e96e 100644 --- a/packages/redrob/src/tool/domain.ts +++ b/packages/redrob/src/tool/domain.ts @@ -22,6 +22,8 @@ import { Effect, Schema } from "effect" import { Tool as SandboxTool, toolError } from "@redrob-code/codemode" import { PartID, type MessageID, type SessionID } from "../session/schema" import type { Session } from "@/session/session" +import { DOCUMENT_GLOBALS, documentTools, unavailableDocuments, type Documents } from "./document" +import { DomainUnavailableError } from "./domain-error" /** One node returned by a page query. Kept to what a skill can act on without a handle. */ export interface PageNode { @@ -33,15 +35,12 @@ export interface PageNode { /** * A domain object the running engine cannot reach. Carries the missing capability by name * so the model, the logs, and a person reading a transcript all see the same reason. + * + * Defined in `./domain-error` and re-exported here, so `document.ts` can raise it without + * this module and that one forming an evaluation cycle. Importers keep importing it from + * here. */ -export class DomainUnavailableError extends Schema.TaggedErrorClass()( - "CodeModeDomainUnavailable", - { object: Schema.String, missing: Schema.String }, -) { - override get message() { - return `The \`${this.object}\` domain object is not available in this session: ${this.missing}` - } -} +export { DomainUnavailableError } from "./domain-error" /** * The browser page the agent is acting in. @@ -220,13 +219,22 @@ export const channelTools = (channel: Channel) => ({ }) /** The names bound as bare globals in the interpreter scope, in the order they are declared. */ -export const DOMAIN_GLOBALS = ["channel", "page"] as const +export const DOMAIN_GLOBALS = ["channel", "page", ...DOCUMENT_GLOBALS] as const /** * The domain namespaces for one execution, keyed by the global name each is bound to. * Returned as one object so the tool tree and `DOMAIN_GLOBALS` cannot drift apart. + * + * `documents` defaults to the refusing implementation: a caller that has no directory to + * resolve a path against — the catalog preview — still advertises the same globals a live + * execution binds, and every call on them fails by name instead of pretending. */ -export const domainTools = (input: { readonly page: Page; readonly channel: Channel }) => ({ +export const domainTools = (input: { + readonly page: Page + readonly channel: Channel + readonly documents?: Documents +}) => ({ channel: channelTools(input.channel), page: pageTools(input.page), + ...documentTools(input.documents ?? unavailableDocuments()), }) diff --git a/packages/redrob/test/tool/code-mode.test.ts b/packages/redrob/test/tool/code-mode.test.ts index 053cad8cc3..fde63971d7 100644 --- a/packages/redrob/test/tool/code-mode.test.ts +++ b/packages/redrob/test/tool/code-mode.test.ts @@ -267,11 +267,12 @@ describe("code mode execute", () => { ctx, ), ) - // The K-1 domain namespaces (`channel`, `page`) are part of the language, so they are - // always present alongside whatever MCP servers are connected. + // The K-1 domain namespaces (`channel`, `page`) and K-3's document namespaces (`docx`, + // `pdf`, `pptx`, `xlsx`) are part of the language, so they are always present alongside + // whatever MCP servers are connected. expect(JSON.parse(output.output)).toEqual({ - namespaces: ["github", "linear", "channel", "page", "$codemode"], - count: 5, + namespaces: ["github", "linear", "channel", "page", "docx", "pdf", "pptx", "xlsx", "$codemode"], + count: 9, }) }) diff --git a/packages/redrob/test/tool/document-skill.test.ts b/packages/redrob/test/tool/document-skill.test.ts new file mode 100644 index 0000000000..5afee604c5 --- /dev/null +++ b/packages/redrob/test/tool/document-skill.test.ts @@ -0,0 +1,101 @@ +/** + * K-3's skill document, held to the code it describes. + * + * A skill is only cheap because the objects exist; a skill that describes a method that does + * not exist produces an agent that calls it and fails. So this test reads the document the + * engine actually ships — `SkillPlugin.DocumentToolchainContent` — and checks it BOTH ways: + * every `.` the document names is a real tool, and every real tool is named by + * the document. The field names it promises are compared against the keys real calls return, + * not against a list retyped here. + */ +import { afterAll, beforeAll, describe, expect, test } from "bun:test" +import * as fs from "node:fs/promises" +import * as os from "node:os" +import * as path from "node:path" +import { Effect } from "effect" +import { SkillPlugin } from "@redrob-code/core/plugin/skill" +import { DOCUMENT_GLOBALS, documentTools, documents, unavailableDocuments } from "@/tool/document" + +const DOC = SkillPlugin.DocumentToolchainContent + +let directory: string +beforeAll(async () => { + directory = await fs.mkdtemp(path.join(os.tmpdir(), "redrob-k3-skill-")) +}) +afterAll(async () => { + await fs.rm(directory, { recursive: true, force: true }) +}) + +const run = (effect: Effect.Effect) => Effect.runPromise(effect) + +/** Every `.` the document mentions, deduplicated. */ +const mentioned = new Set( + [...DOC.matchAll(/\b(docx|xlsx|pptx|pdf)\.([A-Za-z]+)\b/g)].map((match) => `${match[1]}.${match[2]}`), +) + +/** Every `.` the code actually binds. */ +const real = new Set( + Object.entries(documentTools(unavailableDocuments())).flatMap(([global, namespace]) => + Object.keys(namespace).map((method) => `${global}.${method}`), + ), +) + +describe("the K-3 skill document describes the code that exists", () => { + test("it names every global the engine binds", () => { + for (const name of DOCUMENT_GLOBALS) expect(DOC).toContain(`\`${name}\``) + }) + + test("every method the document names is a real tool", () => { + expect([...mentioned].filter((entry) => !real.has(entry))).toStrictEqual([]) + }) + + test("every real tool is documented, so nothing ships undescribed", () => { + expect([...real].filter((entry) => !mentioned.has(entry))).toStrictEqual([]) + }) + + test("the result fields it promises are the fields a real call returns", async () => { + const docs = documents({ directory }) + + const docxWrite = await run(docs.docx.write({ path: path.join(directory, "a.docx"), paragraphs: [{ text: "x" }] })) + for (const key of Object.keys(docxWrite)) expect(DOC).toContain(key) + + const docxRead = await run(docs.docx.read({ path: path.join(directory, "a.docx") })) + for (const key of Object.keys(docxRead)) expect(DOC).toContain(key) + + await run( + docs.xlsx.write({ path: path.join(directory, "a.xlsx"), sheets: [{ name: "S", rows: [["h"]] }] }), + ) + const xlsxRead = await run(docs.xlsx.read({ path: path.join(directory, "a.xlsx") })) + for (const key of Object.keys(xlsxRead)) expect(DOC).toContain(key) + for (const key of Object.keys(xlsxRead.sheets[0]!)) expect(DOC).toContain(key) + + await run(docs.pptx.write({ path: path.join(directory, "a.pptx"), slides: [{ title: "T" }] })) + const pptxRead = await run(docs.pptx.read({ path: path.join(directory, "a.pptx") })) + for (const key of Object.keys(pptxRead)) expect(DOC).toContain(key) + for (const key of Object.keys(pptxRead.slides[0]!)) expect(DOC).toContain(key) + + await run(docs.pdf.write({ path: path.join(directory, "a.pdf"), title: "T", pages: [{ lines: ["l"] }] })) + const pdfRead = await run(docs.pdf.read({ path: path.join(directory, "a.pdf") })) + for (const key of Object.keys(pdfRead)) expect(DOC).toContain(key) + for (const key of Object.keys(pdfRead.pages[0]!)) expect(DOC).toContain(key) + // `reason` only appears on an incomplete page, so it is asserted from one. + const blank = await run(docs.pdf.write({ path: path.join(directory, "b.pdf"), pages: [{ lines: [] }] })) + expect(blank.bytes).toBeGreaterThan(0) + const blankRead = await run(docs.pdf.read({ path: path.join(directory, "b.pdf") })) + for (const key of Object.keys(blankRead.pages[0]!)) expect(DOC).toContain(key) + }) + + test("it states the limit that makes pdf.read honest, and does not promise OCR", () => { + expect(DOC).toContain("textComplete") + expect(DOC).toContain("Type0") + expect(DOC).toContain("there is no OCR in the engine") + expect(DOC).not.toContain("Aside") + }) + + test("the registered skill's own description and arming match the document it carries", async () => { + // The skill is registered as an embedded source in packages/core; this asserts the body + // the plugin hands over is this document and not a stale copy. + expect(DOC).toContain("# Reading and writing documents in code mode") + expect(DOC.startsWith("