diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b09c43cc3..c6dd99e24 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -11,6 +11,8 @@ jobs: steps: - name: Checkout uses: actions/checkout@v4 + with: + fetch-depth: 0 - name: Setup Node uses: actions/setup-node@v4 @@ -28,6 +30,11 @@ jobs: SDP_CORPUS_COMMIT: ${{ github.sha }} run: npm run check + - name: Check article evidence + shell: bash + run: | + npm run check:articles -- "$GITHUB_WORKSPACE" + - name: Upload run evidence if: always() uses: actions/upload-artifact@v4 diff --git a/.prettierignore b/.prettierignore index 313fb66bc..127c29003 100644 --- a/.prettierignore +++ b/.prettierignore @@ -2,6 +2,9 @@ AGENTS.md CLAUDE.md CONTEXT.md docs/** +!docs/articles/ +!docs/articles/sync-evidence.mjs +!docs/articles/sync-evidence.d.mts jtbd-stories/** plans/** .sisyphus/** diff --git a/AGENTS.md b/AGENTS.md index 1af0c26dd..82db9e227 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -63,7 +63,7 @@ Progressive disclosure — start at the top, follow the pointers down. | `src/` | **the engine** — `model` (Spec/descriptors/pack/anchors) · `extract` · `graph` · `validate` · `reader` (agent surface) · `projections` (Design Review, census, Mermaid, Gherkin and Studio) · `cli` (`sdp build` · `validate` · `view` · `census` · `mermaid` · `gherkin` · `studio` · `vault` · `stamp` · `generate` · `import` · `new spec` · `q`) · `import` (carrier conversion and import analysis) · `testing` (the registrar runtime exported through `./testing`) · `runner` / `codegen` / `notation` / `adapters` | when implementing or verifying **current engine** behavior | | `examples/checkout-v1` | **the worked MVP example** (Markdown carriers; originally the TS DSL tracer bullet) — specs, anchors, untracked `generated/` (regenerated in-pipeline); walkthrough in its README | when proving the loop end-to-end | | `explorations/` | **evidence only** (carrier exhibits, executable-example findings) — mapping evidence for design; **never promote spike code into product** | when judging design evidence; not a source tree to ship | -| `docs/lineage/` | restored historical design documents — the v0 concept set, verbatim with lineage headers; evidence for design reviews, never intended truth | when a review needs superseded design on the table rather than in git history | +| `docs/lineage/` | historical design evidence — the v0 concept set, restored verbatim with lineage headers, plus two authored annotation notes: the short annotation context (`annotation-context.md`) and the dated research scout, kept temporarily under an `archive-` name; each note states its own status; all of it is evidence for design reviews, never intended truth | when a review needs superseded design, or the background to the annotation decisions, on the table rather than in git history | | `pack:roadmap` + recipe 53 | **the next work** — the owner's advisory priority over upcoming capabilities, each entry placed by the graph as to design, to state, to build or built; the open arc's record is its draft pull request | before writing code | | `npm run check` | **the green gate** — `check:temporal` → `lint` → `format:check` → `build` → `generate:self-hosting` → `generate:example` → `typecheck` → `typecheck:examples` → `test` → `check:self-hosting-gates` → `check:self-hosting` → `check:example` → `preflight` | before claiming green / after engine edits | | `plans/`, `reviews/` | **frozen history** of earlier arcs: plans, session reviews, prompts and assets; read for provenance; nothing is added | rarely | diff --git a/README.md b/README.md index 5514d6bf8..05e983c85 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,26 @@ Libar Software Delivery Protocol is a typed, executable, self-validating meta-mo delivery: author `Spec` documents in the repository, derive one graph, and check conformance and honesty. +## Read the product story + +The [article series and publication plan](docs/articles/README.md) are the guided introduction to +the Protocol, its human review experience and its AI direction. The series opened in +[PR #34](https://github.com/libar-dev/software-delivery-protocol/pull/34) and is maintained on +`main` through focused refresh PRs. + +Start with [the spec is the prompt is the design](docs/articles/01-the-spec-is-the-prompt-is-the-design.md), +then read [one graph, many views](docs/articles/02-one-graph-many-views.md) for the projection +family and the decision boundaries around changing it. +The [hands-on walkthrough](docs/articles/03-walkthrough-intent-to-verified-code.md) follows intent +into verified code. [The agent-surface article](docs/articles/04-stop-building-verb-walls-for-coding-agents.md) +explains `sdp q`, and [the AI direction](docs/articles/05-ai-can-shape-the-reading.md) separates +current agent workflows from future presentation assistance. + +The [pre-code seed design](docs/lineage/v0-design/README.md) carries the original product +experience. The [short annotation context](docs/lineage/annotation-context.md) explains why source +bindings are critical to useful projections. For the first UX/UI design pass, use the compact +article packet, its designer's brief and a Studio page set generated from the same source pin. + **Carrier rule:** Specs and Packs default to Markdown; the TS DSL survives as import source and a lawful per-ID option, while behavior and example Specs may use Gherkin canonically per ID (the carrier ruling, MD-18, the Pack syntax ruling, MD-25, and the Gherkin carrier option, MD-27). diff --git a/docs/articles/01-the-spec-is-the-prompt-is-the-design.md b/docs/articles/01-the-spec-is-the-prompt-is-the-design.md new file mode 100644 index 000000000..b4ad8c16f --- /dev/null +++ b/docs/articles/01-the-spec-is-the-prompt-is-the-design.md @@ -0,0 +1,184 @@ +# The spec is the prompt is the design: why AI-native delivery needs one graph + +*Part 1 in a series introducing the Libar Software Delivery Protocol. Links and measurements are pinned to one commit; see the footer.* + +At the start of a project, intent is sharp. You write an architecture document, record a few decisions, file tickets and write tests. Then a customer edge case forces a design change during a pull request. The code and test change. The architecture document, decision record and ticket do not. + +An AI coding agent inherits that fragmentation. It can write a function quickly, but it cannot tell an early idea from a settled contract, an implementation shortcut from a ratified decision, or a verifier binding from a passing test. It reads raw files and stale prose, then guesses at the missing structure. + +[Libar Software Delivery Protocol](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753) (`@libar-dev/software-delivery-protocol`, CLI `sdp`) addresses that missing structure. It is an executable, self-validating meta-model of the software-delivery process. + +Its premise has four parts: + +> Specs are code; the graph is derived; the `claim` stays honest; git is the event log. + +This article explains those parts and what they make possible. + +## One Spec, enriched in place + +Delivery tools often make an idea change artifact types as it matures. A note becomes a requirement, then a design document, then tickets, then acceptance criteria. Each migration loses context and breaks traceability. + +The Protocol has one authored truth-primitive, `Spec`. A Spec keeps its stable, namespaced identity, such as `spec:orders.create-order`, from its first capture through implementation. You enrich it with typed sections or refine it into child Specs. You do not convert it into a different artifact ([`spec:model.core-model`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/model/core-model.sdp.md)). + +Three independent descriptors position every Spec: + +- `kind` names the category of truth: `behavior`, `workflow`, `example`, `rule`, `constraint`, `model`, `decision` or `contract`. +- `altitude` names size and scope: `epic`, `feature` or `story`. +- `readiness` is the author's stated design maturity: `idea`, `scoped`, `defined` or `ready`. + +Use Case, NFR, Decision Record, Epic, Story and Scenario are named coordinates on this one shape, not separate schemas. An Executable Spec is an `example` with a verifier, not another type or a readiness rung. + +A Pack groups Specs for reading and review without stating system truth of its own. An anchor binds a code unit, test or expected-outcome oracle into the graph. Anchors carry identity and optional structure, never behavior, rationale or readiness. + +### Why the Spec is the prompt + +A Spec can carry the intent, rules, examples, constraints, decisions, design entries and open questions needed to implement it. Its graph context also includes relations, source bindings, verifier bindings and the structural floor behind its stated readiness: + +```sh +npm run --silent sdp:q -- 'return g.specContext("spec:consumers.reader")' --json +``` + +"Implement `spec:…`" can therefore be a precise instruction. The agent reads the same authored intent and bindings that a reviewer sees. The work shifts from rewriting briefs for different tools to improving the Spec itself. + +That only works when authors avoid padding. The Protocol's readiness floor is a minimum structural requirement, not a quota of sections. A familiar CRUD endpoint may need little detail. A novel business rule may need much more. Checks can tell whether a stated rung clears its floor. They cannot decide whether the design is good. + +## The graph is derived + +Nobody authors the graph. `sdp build` reads the carriers and source anchors and derives one flat, typed, deterministic document: nodes, edges and a location table. Validators, recipes and projections consume that one read model rather than parsing source into private alternatives ([`spec:extraction.derive-graph`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/extraction/derive-graph.sdp.md)). + +Edges that touch the same Spec can carry different claims: + +- `refines` from one Spec to another is authored intent with a `declared` claim. +- `satisfies` from a code unit to a Spec comes from an anchor and has an `anchored` claim. +- `belongsTo` from a Spec to its Pack is computed from the Pack manifest and inherits the manifest's `declared` claim. + +The distinction is not bookkeeping. It lets a consumer answer "what did the authors intend?" separately from "what source did they bind?" and "what did static analysis infer?" + +The graph holds current state at one commit. Git holds prior states, deleted Specs and the answers to "who changed what?" and "when?" A decision may declare `supersedes` while the older record still explains something a reader needs. That is current intent, not a second history store. + +## The claim stays honest + +Every node and edge has one of three claims: + +| Claim | Source | Authority | +|---|---|---| +| `declared` | Human-authored Spec or Pack content | Authoritative intent | +| `anchored` | Human-authored source or test anchor | Authoritative binding, no intent | +| `inferred` | Mechanical analysis | Advisory structure, never silently promoted | + +The current curated graph contains declared and anchored content. File-level imports supplied through `sdp build --imports` remain separate inferred input for curation advice. They do not mint curated edges or delivery facts ([`spec:extraction.claim-taxonomy`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/extraction/claim-taxonomy.sdp.md)). + +Readiness and realization stay independent too: + +- Stated readiness is an author's position, checked against a structural floor. +- Derived readiness is the highest rung whose floor clauses pass. It appears beside the stated rung and never replaces it. +- `implemented` means at least one resolving `satisfies` edge binds code directly to the Spec. +- `has-verifier` means a resolving verifier binding exists. It does not mean the test passed. +- `observed` would record runtime liveness. It is designed for and not derived today. + +Two useful queries fall out of this separation. + +The operational build backlog finds ready Specs that no code realizes, excluding examples and decision records for their audited realization postures: + +```js +g.specs().filter((s) => + s.statedReadiness === "ready" && + !s.deliveryFacts.includes("implemented") && + s.specKind !== "example" && + s.specKind !== "decision") +``` + +The drift alarm finds code ahead of stated design maturity: + +```js +g.specs().filter((s) => + s.deliveryFacts.includes("implemented") && + s.statedReadiness !== "ready") +``` + +Neither query is a CI failure. They are standing questions over current repository state. + +## Checks police conformance and honesty + +`sdp validate` runs two check families over the one graph. + +Conformance checks ask whether the model is well-formed. They catch unresolved relations, duplicate IDs, invalid claims and malformed structural bindings. Honesty checks ask whether the model is pretending. They reject hand-authored delivery facts and readiness statements that do not clear their floors. Informative gaps, such as a ready Spec without a verifier, remain visible without becoming mandatory workflow gates. + +A team can add corpus rules through `sdp validate --rules`. Those findings remain distinct from the Protocol's two families. + +The permanent guardrail is worth stating exactly: checks police conformance and honesty, never content quality and never workflow. The strongest claim is "deterministically validated," never "provably correct." + +## One graph feeds different consumers + +A graph is useful only if people and agents can read it in forms that fit their work. + +Agents script the graph through `sdp q` and the exported typed reader. People can read Markdown Design Reviews, census reports, Mermaid diagrams, Gherkin-shaped views, the static HTML Spec Studio or generated Obsidian notes. These are projections, disposable and regenerable rather than rival stores of delivery status. + +The Spec Studio makes the human side concrete. A reviewer can explore a Pack, inspect open questions and architecture, compare a design with a base, read supplied run evidence and gather requests as scoped intent. An agent applies accepted changes as ordinary source edits. The view never becomes canonical. + +[Part 2](./02-one-graph-many-views.md) develops that human-projection story and its implications for UX/UI design. [Part 4](./04-stop-building-verb-walls-for-coding-agents.md) takes the agent side. + +## The Protocol measures itself + +The Protocol has no separate PRD. It authors its own rules as Specs, binds its engine and tests to them, derives its own graph and checks that graph in CI ([`spec:protocol.self-hosting`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/protocol/self-hosting.sdp.md)). + + +Measured at commit `b5e0a8bea1e3bedd48c616f96c9928e955792753`: + +| Measure | Value | +|---|---| +| Authored Specs | 456: 447 `ready`, 4 `scoped`, 5 `idea` | +| Packs | 9 | +| Source and test bindings | 241 CodeNodes and 366 Anchors | +| Graph | 1,072 nodes, 2,978 edges, 3,120 locations | +| Delivery facts | 166 `implemented`, 397 `has-verifier`, 0 `observed` | +| Operational build backlog | 2 | +| Drift alarm | 0 | +| Runnable recipes | 55 | + + +The [generated baseline metrics](./README.md#step-3-re-derive-corpus-metrics) report self-hosting validation errors and warnings by rule. A `honesty/gaps` warning names a ready Spec without a resolving verifier. Re-run the commands below rather than carrying these numbers to another commit. + +The first adopter has also used the Protocol to design a much larger platform ahead of implementation. That story, including context bundles in parts, design slices and corpus rules, belongs to Part 9. + +## What the Protocol does not try to be + +The Protocol is not a ticket state machine. It adopts delivery nouns such as phase, iteration and release as descriptive vocabulary and projections, not mandatory gates. + +It is not an embedded permissions system. Approval belongs in git and CI. + +It does not infer intent. Mechanical structure can assist curation, but a person authors intent and bindings. + +The aim is to connect adjacent tools rather than replace them. Today the Studio reads supplied CTRF test reports as run evidence. An interface and API reference remains an [idea-stage capture](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/reference-projection.sdp.md). Dedicated integrations with issue trackers and design tools are not designed yet; generic Design and UI prose can mention them without providing that experience. + +At the pinned commit, installation is from a reviewed git commit rather than an npm release. The package version is `0.0.0`; `sdp --version` names the build commit. + +## Try it + +For this source checkout, use Node `^20.19.0 || ^22.12.0 || >=24.0.0`, where the ranges declared by the installed test toolchain (Vitest, Vite and Rolldown) overlap. The installed CLI separately declares Node `>=20`: + +```sh +git clone https://github.com/libar-dev/software-delivery-protocol.git +cd software-delivery-protocol +git checkout b5e0a8bea1e3bedd48c616f96c9928e955792753 +npm ci + +npm run --silent sdp -- validate . \ + --exclude explorations --exclude examples --exclude test/fixtures/import/parity + +npm run --silent sdp:q -- "$(cat dist/recipes/01-the-build-backlog.js)" --json +npm run --silent sdp:q -- "$(cat dist/recipes/02-the-drift-alarm.js)" --json + +npm run --silent sdp -- studio . \ + --exclude explorations --exclude examples --exclude test/fixtures/import/parity +``` + +Open `generated/spec-studio/index.html`. The `sdp:q` script supplies the repository's three exclusions itself; the explicit CLI commands include them. + +## Up next + +[Part 2: One graph, many views](./02-one-graph-many-views.md) follows a reviewer through the Studio and asks the question that matters for the first UX/UI design pass: what can change freely when every interface is a projection? + +--- + +*About this article: Part 1 in a series introducing the [Libar Software Delivery Protocol](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753). Repository links and measurements are pinned to commit `b5e0a8be`; re-run the commands rather than quoting the numbers at a later commit.* diff --git a/docs/articles/02-one-graph-many-views.md b/docs/articles/02-one-graph-many-views.md new file mode 100644 index 000000000..cb7c541e3 --- /dev/null +++ b/docs/articles/02-one-graph-many-views.md @@ -0,0 +1,153 @@ +# One graph, many views: the interface is allowed to change + +*Part 2 in a series introducing the Libar Software Delivery Protocol. Links are pinned to one commit; see the footer.* + +A product manager opens a pull-request preview on a phone. The design under review has no implementation yet. They read the Pack's framing, inspect a blocking question, and open the Spec that carries it. A dependency points to an existing component, so they follow it to see what the new design would touch. A base graph shows what changed in the design against the pull request's base commit. They want a different example and a clearer constraint, so they gather both requests as scoped intent and hand the resulting proposal to an agent. + +The agent edits the carriers. Git records the change. The conformance and honesty checks run. A new graph produces a new page set. + +Nothing in that workflow requires the interface to be a database, an editor, or the one canonical way of reading a design. The interface is a projection. We can replace the Studio interface under its carrying Specs without replacing the truth it presents. + +That freedom is the subject of this article. + +## The seed was a workbench, not a documentation page + +The original design, written before the implementation, described a generated HTML Spec Studio where people could explore Specs, architecture, tests and evidence. Its central scenario was not "read a requirement online." It was a person understanding a system, finding gaps, proposing a change and returning that change to the repository ([the seed vision](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/v0-design/00-vision-and-product.md), [the seed Studio design](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/v0-design/07-spec-studio-and-projections.md)). + +Some representations in that design have since been rejected. TypeScript is no longer the default carrier. JSON patches and codemods have given way to scoped intent and ordinary source edits. Test verdicts no longer enter the graph as readiness. The vocabulary has been ratified and narrowed. + +The product idea survived those changes: one delivery model should support reading and review experiences that raw carriers and flat documents cannot provide. + +The current Studio is an early realization of that idea, not its final visual design. Its pages, diagrams and interaction patterns give us working material to learn from. They are not a template that the first UX/UI designer must preserve. + +## One record, several readings + +The repository holds authored Specs, Packs and source anchors. The extractor derives one graph. Validators and consumers read that graph. A projection turns it into an artifact for a particular consumer ([`spec:consumers.projections-model`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/projections-model.sdp.md)). + +At the pinned commit, the package publishes five projection roots: + +| Projection | Consumer and reading | +|---|---| +| Design Review | A Spec or Pack rendered in context as Markdown, with the relations and bindings needed for a readiness decision | +| Census | Taxonomy, readiness, bindings and findings as a compact report | +| Mermaid | Relationship and architecture diagrams with their source available for inspection | +| Gherkin | A familiar behavioral reading of the graph | +| Spec Studio | A static HTML workbench for exploration, review and composing intent | + +The graph also serves `sdp q`, CLI answers and typed contracts. An Obsidian vault provides generated notes and Bases beside a person's own notes. These are different readings, not competing repositories of delivery status. + +This breadth is intentional, but not every form is open. The [shipped-projections decision](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/decisions/shipped-projections-frozen.sdp.md) (MD-32) settles Design Review, census, Mermaid and Gherkin as read models. The Studio and any new reading are where we learn which forms earn their place. + +The article's diagram evidence comes from the same mechanism. This Mermaid source is emitted by `sdp mermaid` from the checkout roadmap Pack, not redrawn for publication: + + +```mermaid +flowchart LR + n_007000610063006b003a0063006800650063006b006f00750074002d0072006f00610064006d00610070["Checkout roadmap pack:checkout-roadmap Pack"] + n_0073007000650063003a006f00720064006500720073002e006f0072006400650072002d006d006f00640065006c["Order-management domain vocabulary spec:orders.order-model Primitive"] + n_0073007000650063003a006f00720064006500720073002e006f0072006400650072002d0070006c006100630065006d0065006e0074002d0066006c006f0077["Order placement flow spec:orders.order-placement-flow Primitive"] + n_0073007000650063003a006f00720064006500720073002e006f0072006400650072002d006d006f00640065006c -->|"belongsTo [declared]"| n_007000610063006b003a0063006800650063006b006f00750074002d0072006f00610064006d00610070 + n_0073007000650063003a006f00720064006500720073002e006f0072006400650072002d0070006c006100630065006d0065006e0074002d0066006c006f0077 -->|"belongsTo [declared]"| n_007000610063006b003a0063006800650063006b006f00750074002d0072006f00610064006d00610070 +``` + + +The rule that matters is not "everything must look the same." It is "no view may quietly become another source of truth." + +## What the current Studio already does + +`sdp studio` publishes a static page set that works from `file://` or a static host. It needs no application server. Every join, delivery fact, readiness floor and finding is computed during the build. The browser does not assemble a second graph ([`spec:consumers.spec-studio`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/spec-studio.sdp.md)). + +A reader can move through five lenses: Packs, capabilities, architecture, tests and evidence. Other pages collect the roadmap, the owner queue, open questions, decisions, declarations, the design backlog and the glossary. A build can also include a design diff from a supplied base graph and run evidence from supplied CTRF reports. A Pack can be exported as one self-contained HTML file. + +For a pull-request reviewer, the important detail is the stamp. Publishing metadata names the corpus commit and engine after the page set is built. The page set itself remains a deterministic function of its graph and named inputs ([`spec:consumers.spec-studio.sharing`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/spec-studio.sharing.sdp.md)). + +This is already enough to test a real review workflow. It is not evidence that five lenses, the current navigation or today's information density are the right long-term answers. Those are UX questions. + +## Honesty has to be visible + +A flexible interface still has obligations to the model. The important ones are not visual conventions. They are rules against misleading the reader. + +**Intent, bindings and mechanical structure need different meanings.** A `declared` relation is authored intent. An `anchored` relation is a human binding from code or tests. `inferred` structure is machine-derived and advisory. A view may use words, placement or typography to make that difference legible. It must not merge the three into an undifferentiated web of connections. + +**A binding is not a verdict.** `implemented` means a resolving `satisfies` binding exists. `has-verifier` means a resolving verifier exists. Neither means the code works or a test passed. Supplied run records can appear beside those bindings, labeled with their input and commit. They remain external run evidence, not new delivery facts. + +**Readiness and realization are independent.** An author states readiness. The graph derives the highest structural floor reached and the delivery facts. A useful review page lets a person see all three without turning them into a single progress score. + +**Absence needs words.** An example-space value no example binds, a slot whose coverage is not measurable, a missing verifier and an evidence lens with no tracked observations say different things. `coverage-unknown` names a separate file-impact blind spot: a changed file the graph records nothing at. An empty panel with no explanation forces the reader to guess which absence the system means. + +**A drawing needs a text equivalent.** Studio diagrams retain words that state every box and edge. When a drawing exceeds its bound, it folds and the text remains. This is an accessibility rule and an honesty rule: visual compression must not remove meaning. + +These constraints leave many visual choices open. They rule out interface shortcuts that would invent certainty. + +## The view composes intent, not edits + +The seed design used a patch panel. The current design has a composing panel instead. + +A person can gather requests while reading, scoped to a Spec, an entry, a Pack or an open question. The panel produces a change proposal and its Markdown memo when it has a base and a reason. Anything the proposal contract cannot carry stays as scoped-intent text. The page sends nothing over the network and opens no pull request ([`spec:consumers.spec-studio.intent-panel`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/spec-studio.intent-panel.sdp.md)). + +An agent receives that hand-off and edits ordinary source. Git records the result. The same checks apply to that edit as to one typed directly into a carrier. + +The distinction matters for UX. The person is not manipulating a shadow copy of the graph. They are explaining what they want changed, with enough scope for another reader, human or agent, to act on it. + +## A compact brief for the first UX/UI design pass + +Use the current Studio as a working prototype and the seed design as product context. Do not start by restyling today's pages. Start with the review jobs. + +The fixed ground: + +- The repository is canonical and the graph is the sole read model. +- A projection is disposable and regenerable. No interface state becomes delivery truth. +- The [shipped-projections decision](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/decisions/shipped-projections-frozen.sdp.md) (MD-32) freezes Design Review, census, Mermaid and Gherkin. Reshaping one needs a refining or superseding decision. The Studio can evolve under its own Specs. +- Claims, stated readiness, derived readiness and delivery facts stay distinct. +- Outside inputs and their commits stay named wherever they appear. +- Unknowns and missing evidence remain visible. +- Diagrams keep complete text equivalents. +- Changes leave as scoped intent. Source edits and git history happen outside the view. +- Conformance and honesty checks never judge visual quality or design quality. + +The open design space: + +- Navigation, page structure, visual hierarchy, typography and interaction patterns. +- Which Studio pages deserve to survive, merge or disappear. +- How a person moves between a Pack overview, one Spec, an entry and its source. +- How desktop and phone review differ. +- How much information belongs in the first reading and what belongs behind disclosure. +- How a future AI assistant can steer a reading without obscuring its basis. + +Start from the Studio's [manual acceptance tasks](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/spec-studio.sdp.md#verification--manual) and test prototypes against them. These are specified acceptance criteria, not evidence that a person has completed them. + +The first design pass should answer these questions with prototypes: + +1. What can a reviewer accomplish from a stamped link without opening the repository? +2. What is the fastest route from "what changed?" to "is this design ready?" for one Pack? +3. How should a reader distinguish intent, code bindings, inferred structure and supplied run evidence without learning the implementation schema first? +4. How do missing verification, missing observations, unmeasurable slot coverage and example-space values with no example look different? +5. When is a diagram better than prose, and how does its text equivalent read on a phone? +6. How does a person gather several related requests and understand exactly what will be handed to an agent? +7. Which current pages solve real review jobs, and which only expose data because it is available? + +The deliverable should be a small set of task-based experiences, not a comprehensive design system for every page the generator can emit. + +## Try the material + +From a checkout at the pinned commit: + +```sh +git clone https://github.com/libar-dev/software-delivery-protocol.git +cd software-delivery-protocol +git checkout b5e0a8bea1e3bedd48c616f96c9928e955792753 +npm ci + +npm run --silent sdp -- generate . \ + --exclude explorations --exclude examples --exclude test/fixtures/import/parity +``` + +Open `generated/spec-studio/index.html`. Compare it with `generated/design-review/`, `generated/census/`, `generated/mermaid/` and `generated/gherkin/`. They are different answers over one graph, not five versions of the truth to reconcile. + +## Up next + +Part 3, [From rough idea to verified code](./03-walkthrough-intent-to-verified-code.md), leaves the reading surfaces and follows one Spec through authoring, source bindings, generated contracts and executable verification. + +--- + +*About this article: Part 2 in a series introducing the [Libar Software Delivery Protocol](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753). Repository links are pinned to commit `b5e0a8be`. The seed documents explain the original product experience; current Specs and source determine what is built now.* diff --git a/docs/articles/03-walkthrough-intent-to-verified-code.md b/docs/articles/03-walkthrough-intent-to-verified-code.md new file mode 100644 index 000000000..faddd2efd --- /dev/null +++ b/docs/articles/03-walkthrough-intent-to-verified-code.md @@ -0,0 +1,296 @@ +# From rough idea to verified code: a hands-on tour of the Protocol + +*Part 3 in a series introducing the Libar Software Delivery Protocol. Links and measurements are pinned to one commit; see the footer.* + +[Part 1](./01-the-spec-is-the-prompt-is-the-design.md) introduced one enrichable `Spec`, source anchors and the derived graph. [Part 2](./02-one-graph-many-views.md) showed how people read that graph through regenerable projections. + +This article follows the engineering loop in a text editor and a terminal. We will use Order Management in [`examples/checkout-v1`](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753/examples/checkout-v1), then inspect a comment-form binding in the Protocol's own engine. + +## Setup + +Every command runs from a checkout at the pinned commit. Use Node `^20.19.0 || ^22.12.0 || >=24.0.0`, where the ranges declared by the installed test toolchain (Vitest, Vite and Rolldown) overlap: + +```sh +git clone https://github.com/libar-dev/software-delivery-protocol.git +cd software-delivery-protocol +git checkout b5e0a8bea1e3bedd48c616f96c9928e955792753 +npm ci +``` + +Inside the Protocol's own checkout, run the CLI as `npm run --silent sdp -- `. Do not invoke a bare `sdp`, which can select macOS's unrelated binary. An adopter that installed the package uses `npx sdp` or `pnpm exec sdp` instead. + +`examples/checkout-v1` is its own extraction root, so these commands need none of the exclusions used for the Protocol's self-hosting corpus. + +## The worked example + +The example models Order Management with all eight Spec kinds and all four readiness rungs. The generated build output below carries its counts at the pin; the files are arranged as follows: + +```text +examples/checkout-v1/ + specs/ + checkout.pack.sdp.md # pack:checkout-v1 + roadmap.pack.sdp.md # pack:checkout-roadmap + orders/order-management.sdp.md # behavior · epic · defined + orders/order-placement-flow.sdp.md # workflow · feature · scoped + orders/create-order.sdp.md # behavior · feature · defined + orders/create-order-valid-cart.sdp.md # example · story · ready + orders/create-order-invalid-cart.sdp.md # example · story · defined + orders/create-order-api-contract.sdp.md # contract · story · idea + orders/order-total-rule.sdp.md # rule · story · defined + orders/order-inventory-rule.sdp.md # rule · story · defined + orders/order-latency-constraint.sdp.md # constraint · story · defined + orders/order-model.sdp.md # model · story · defined + decisions/order-lifecycle.sdp.md # decision · feature · defined + src/orders/ + create-order.use-case.ts # two implementation anchors + create-order.route.ts # one API anchor + test/orders/ + create-order.valid-cart.test.ts # one test anchor + create-order.oracle.ts # one oracle anchor + orders.create-order.valid-cart.test.generated.ts +``` + +Build its graph: + +```sh +npm run --silent sdp -- build examples/checkout-v1 +``` + + +```text +11 specs · 2 packs · 5 anchors → 18 nodes · 34 edges (0 errors, 0 warnings) +Wrote /examples/checkout-v1/generated/graph.json +Wrote /examples/checkout-v1/generated/contracts (3 modules) +``` + + +Before opening a file, ask which Specs have code ahead of their stated design maturity. Recipe 2 answers the drift alarm, `implemented ∧ ¬ready`: + +```sh +npm run --silent sdp -- q "$(cat dist/recipes/02-the-drift-alarm.js)" \ + --root examples/checkout-v1 --json +``` + + +```json +{ + "total": 2, + "alarms": [ + { + "id": "spec:orders.create-order", + "statedReadiness": "defined", + "floorReached": "ready", + "firstUnmetClause": null, + "implementationBindings": 2 + }, + { + "id": "spec:orders.order-total-rule", + "statedReadiness": "defined", + "floorReached": "ready", + "firstUnmetClause": null, + "implementationBindings": 1 + } + ] +} +``` + + +The Specs listed by the generated drift answer have implementation bindings and state only `defined`. Their structural floors reach `ready`, so they wait only for an author's readiness statement. The build output above counts extraction and contract-generation diagnostics; `sdp validate examples/checkout-v1` runs the conformance and honesty checks. The drift alarm is a query, not a workflow gate. + +## Author the parent and its example space + +Open [`create-order.sdp.md`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/examples/checkout-v1/specs/orders/create-order.sdp.md): + + +````markdown +--- +id: spec:orders.create-order +kind: behavior +altitude: feature +readiness: defined +relations: + refines: spec:orders.order-management + constrainedBy: spec:orders.order-latency-constraint + decidedBy: spec:decisions.order-lifecycle +--- +# Customer creates an order + +## Intent + +- actor: customer +- outcome: Turn a valid cart into an order. +- value: Customers can complete purchases without the example modeling the rest of checkout. + +## Example space + +```gwt-vocabulary +Given a customer has a cart with {n:number} line items +And every line item has quantity {q:number} and unit price {price:number} +And every cart item is {availability:"in stock"|"out of stock"} +When the customer submits the cart for order creation +Then an order is created with total {total:number} +And the order contains the original cart lines +And order creation is rejected because {reason:"empty cart"|"out of stock"} +``` +```` + + +The YAML frontmatter is the closed envelope. It carries identity, the three descriptors and relations. A hand-authored `implemented` property or a derived `satisfies` relation is refused. + +Sections carry content and relations carry linkage. The parent names its own refinement, constraint and shaping decision in the envelope rather than hiding those relations in prose. + +The `gwt-vocabulary` fence declares the typed example space its child examples share. Each `{slot:type}` placeholder becomes part of the generated contract. + +### Bind one point + +The child example [`create-order-valid-cart.sdp.md`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/examples/checkout-v1/specs/orders/create-order-valid-cart.sdp.md) refines and verifies its parent: + + +````markdown +--- +id: spec:orders.create-order.valid-cart +kind: example +altitude: story +readiness: ready +relations: + refines: spec:orders.create-order + verifies: spec:orders.create-order +--- +# Valid cart creates an order + +## Intent + +- outcome: Show that a valid cart can become an order. +- value: The authored example demonstrates the happy path for create-order. + +```gwt +Given a customer has a cart with {n: 2} line items +And every line item has quantity {q: 1} and unit price {price: 50} +And every cart item is {availability: "in stock"} +When the customer submits the cart for order creation +Then an order is created with total {total: 100} +And the order contains the original cart lines +``` + +## Verification — executable + +- The order result contains a stable id. +- The returned total matches the authored total — the spec's value, never the test's. +```` + + +Under the point-per-example law, one example binds one concrete point. Under the concreteness law, leaving a used slot unbound, such as `{n}` instead of `{n: 2}`, caps the example below `defined` and withholds its step contract and registrar. + +A behavior or example Spec may use `.sdp.gherkin` instead of Markdown as its one canonical carrier. Markdown remains the default and supports authoring features the Gherkin carrier does not, including open questions and relation reasons. [Part 8's outline](./README.md#article-08-executable-specifications-without-cucumber-glue) takes the carrier and executable-contract design further. + +## Bind source code + +An anchor is a human-authored binding in a `.ts` or `.tsx` file. It carries no Spec prose, readiness or delivery status. Two representations feed the same closed envelope. + +The constant form works when source may import the package or its zero-dependency `/anchors` subpath: + + +```ts +import { codeAnchor, codeAnchorId, ref } from "@libar-dev/software-delivery-protocol"; + +// … + +export const createOrderUseCaseAnchor = codeAnchor({ + id: codeAnchorId("impl:orders.create-order-use-case"), + label: "createOrderFromCart", + satisfies: ref("spec:orders.create-order"), +}); +``` + + +The comment form needs no runtime import. This anchor comes from the Protocol's own [`src/projections/census.ts`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/src/projections/census.ts): + + +```ts +/** + * @sdpAnchor impl:protocol.census-page + * @sdpLabel renders the derived census and runtime taxonomy projection + * @sdpSatisfies spec:consumers.census-page, spec:model.structural-patterns + * @sdpComponent component:protocol.projections + * @sdpReferences spec:decisions.architectural-annotation + * @sdpRole projection + */ +``` + + +`satisfies` and `references` have different meanings: + +- `satisfies` says this unit realizes the Spec's promises together with any other satisfying units. A resolving edge confers `implemented`. +- `references` says this unit answers to a design without claiming to realize it. It supports traceability and impact queries, confers no delivery fact and does not trip the drift alarm. + +A stub or a unit following a decision can therefore name that design without pretending to implement it. + +## Generate the executable half + +`sdp build` emits a step contract for each example and a space contract for the parent. It also emits a registrar beside the test. The authored test imports and activates that registrar, so the adopted registrar is committed and byte-checked. Generated code never imports authored semantics. + +Three pieces meet at test time. + +**The space contract** types the example space. For create-order it includes `{ n: number; q: number; price: number; availability: "in stock" | "out of stock" }`, every child's bound point and the outcome union derived from the parent's Then vocabulary. + +**The expected-outcome oracle** is authored beside the tests and bound with `specOracle`. It returns `unspecified` where the Spec states no outcome: + + +```ts +export function expected(conditions: Partial): CreateOrderOutcome { + if ( + conditions.n === undefined || + conditions.q === undefined || + conditions.price === undefined || + conditions.availability === undefined + ) { + return unspecified; + } + + if (conditions.n === 0) { + return { kind: "order creation is rejected because {reason}", reason: "empty cart" }; + } + + if (conditions.availability === "out of stock") { + return { kind: "order creation is rejected because {reason}", reason: "out of stock" }; + } + + if (conditions.q <= 0 || conditions.price <= 0) { + return unspecified; + } + + return { + kind: "an order is created with total {total}", + total: conditions.n * conditions.q * conditions.price, + }; +} +``` + + +**The bound test** anchors `test:orders.create-order.valid-cart` and activates the registrar with `createWorld`, `invoke`, `observe`, `expected` and optional `assertions` adapters. + +At the matching Then step, the registrar checks both the Spec's authored values and the observed outcome against the oracle. If the Markdown changes `{total: 100}` to `{total: 120}` while the oracle and code still say 100, the test fails on the changed Spec value. Agreement between code and oracle alone is not enough. + +The second example, `invalid-cart`, declares `verifies` but has no test anchor. Recipe 10 shows it as declared but not enabled. Its `defined` rung makes that an honest incomplete trace, not a warning. An unbound example stating `ready` would draw a verifies-linkage warning. + +## Break it on purpose + +Run each experiment in a disposable worktree. If an experiment file already has local changes, do not edit it in your checkout. Otherwise, restore only the files changed by that experiment and rerun `npm run generate:example`. + +| Experiment | Edit | Run | Expected result | +|---|---|---|---| +| Misspell a section heading | Change `## Rule` to `## Rul` in `order-total-rule.sdp.md` | `npm run --silent sdp -- validate examples/checkout-v1` | Exits 1 with `extract/unrecognized-heading` and follow-on structure errors. The Spec drops out of the graph and `graph.json` is not written. | +| Dangle a relation | Misspell `spec:orders.order-management` in `create-order.sdp.md` | The same validate command | Exits 1 with `conformance/referential-integrity` and a nearest-ID suggestion. | +| Keep an unearned rung | Add a blocking open question to the stated-`defined` create-order Spec | The same validate command | Exits 1 with `honesty/readiness-floor`, naming `no-blocking-open-questions`. | +| Leave a slot unbound | Change `{n: 2}` to `{n}` in the valid-cart example | The same validate command | Exits 1 with `honesty/readiness-floor`; the example's contract and registrar are withheld. | +| Drift a Spec value | Change `{total: 100}` to `{total: 120}` | `npm run generate:example && npx vitest --run examples/checkout-v1/test` | The bound test fails because the Spec's value no longer matches the oracle. | + +These experiments show the boundary between extraction, conformance, honesty and executable checks. They also show why none of them needs to judge sprint state or design taste. + +## Up next + +[Part 4: Stop building 30-verb CLI walls for coding agents](./04-stop-building-verb-walls-for-coding-agents.md) follows the graph downstream: the demand map, `sdp q`, recipes and the evidence from agents using that surface. + +--- + +*About this article: Part 3 in a series introducing the [Libar Software Delivery Protocol](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753). Repository links and command outputs are pinned to commit `b5e0a8be`; re-run the commands rather than quoting the outputs at a later commit.* diff --git a/docs/articles/04-stop-building-verb-walls-for-coding-agents.md b/docs/articles/04-stop-building-verb-walls-for-coding-agents.md new file mode 100644 index 000000000..cd83ffebb --- /dev/null +++ b/docs/articles/04-stop-building-verb-walls-for-coding-agents.md @@ -0,0 +1,148 @@ +# Stop building 30-verb CLI walls for coding agents + +*Part 4 in a series introducing the Libar Software Delivery Protocol. Links and measurements are pinned to one commit; see the footer.* + +A CLI gains `list-specs`. Then it gains a readiness filter, a command for verifiers, another for relations and flags for every join. The agent still receives a question no command answers: "show every `defined` Spec in this Pack with no blocking questions and the files that reference it." + +The agent makes a sequence of calls, receives more output than its context can hold and joins the results in scratch space. The tool has exposed many operations and little of the underlying model. + +The Libar Software Delivery Protocol takes another path ([the agent front door, MD-22](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/decisions/agent-front-door.sdp.md)): + +> Give coding agents a visible, typed graph and one evaluation sink they can script. + +## Start with what the agent holds + +Before designing [`spec:consumers.agent-surface`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/agent-surface.sdp.md), we asked what an agent has when a session starts. Usually it does not have the Spec ID it needs. It has one of three things, the demand map: + +1. A concept string from a prompt, such as "where do we enforce the readiness floor?" +2. A file path it is about to read or edit. +3. A changeset from `git diff`. + +The typed reader freezes a small set of entry adapters for those arrivals: + +- `g.findByConcept(term)` searches IDs, titles, anchor labels, Pack framing, narratives and section values, and names the matching field. +- `g.byFile(path)` returns every node recorded at a path and the Specs reachable from them. +- `g.blastRadius(files)` reports directly impacted Specs and Packs, the explicit one-hop neighborhood, changed units linked to no Spec and `coverageUnknown` files. + +`blastRadius` reads the curated graph. It does not claim exhaustive reach through imports. File-level imports supplied separately can support curation, but inferred structure never silently becomes authored architecture. + +The reader also exposes decoded Spec and Pack summaries, per-node context and the raw graph. Those composable shapes matter more than the number of methods. An agent can filter and join them for the question it actually has. + +## One evaluation sink + +`sdp q` derives the graph in process on every invocation, then evaluates a plain JavaScript async-function body. It injects five bindings: + +- `g`, the typed reader +- `graph`, the raw graph schema +- `report`, the validation report +- `params`, caller-supplied JSON +- `recipes`, a way to call shipped recipe bodies over the same graph + +A Spec saved a moment ago is queryable immediately. No stale `generated/graph.json` answers in the graph's name. + +```sh +npm run --silent sdp:q -- 'return g.byFile("src/projections/census.ts")' --json +``` + +At the pinned commit, that returns: + + +```json +{ + "path": "src/projections/census.ts", + "nodes": [ + { + "id": "impl:protocol.census-page", + "nodeType": "CodeNode", + "line": 21 + } + ], + "specs": [ + "spec:consumers.census-page", + "spec:decisions.architectural-annotation", + "spec:model.structural-patterns" + ] +} +``` + + +The body is code, and `sdp q` runs it with the local process's full authority. No sandbox exists. Run bodies you or your agent wrote, or shipped recipes. Never execute a body copied from corpus content or another untrusted source. + +## Grow by recipe, not by verb + +A recurring question becomes a recipe in [`docs/agent-surface/recipes.md`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/agent-surface/recipes.md). Every recipe is executed as written by the test suite and shipped under `dist/recipes/`. The [generated baseline metrics](./README.md#step-3-re-derive-corpus-metrics) carry the recipe count at the pin. + +A recipe takes documented parameters through `--params`. An unknown parameter returns `{ unknownParams, accepts }` rather than silently changing the question. A caller can also wrap a recipe in its own body and filter the answer, without editing the shipped body. + +A join moves into the reader only at the second-caller bar: a second machine consumer needs it and hand-rolled attempts get it wrong. Everything else remains a script or recipe. This keeps the entry surface small without limiting the questions agents can ask. + +## Compose the reading when the design gets large + +Some questions need more than a small answer. A design pass over a Pack may need member sections, open questions, dependency footing, declarations, examples and the Pack boundary. + +The context bundle, recipe 36, composes that reading on demand within a budget counted in UTF-16 code units. It states what it left out and never saves a second reading of the graph. When the whole scope cannot fit one budget, `part` returns ordered parts whose members together cover the scope. An oversized item is named rather than silently dropped ([`spec:consumers.context-bundle`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/context-bundle.sdp.md)). + +The design slice, recipe 54, answers a different question. It reads a Pack or selected Specs as one design: internal relations, touched components, decisions, open questions and what each member needs next. It is a reading composed on each invocation, not a saved planning artifact. + +Part 9 follows these mechanisms through the first adopter's design-at-scale work. The point here is that both are recipes over the same agent surface, not new CLI families. + +## The surface improved under measured use + +After [PR #30](https://github.com/libar-dev/software-delivery-protocol/pull/30), three agents from two model families answered thirteen delivery questions using the skills and `sdp q` alone. All three answered every question correctly. They also exposed recurring costs: oversized answers, unexplained misses, two meanings of "next rung" and rules without addresses or source lines. + +[PR #31](https://github.com/libar-dev/software-delivery-protocol/pull/31) added summary-first answers, explained misses, both next rungs, rule keys and recorded lines. Its acceptance re-run on `gpt-6.1-sol` again answered 13 of 13 but took 23 calls against the first trial's 20 and overflowed one batched answer. The response, in [PR #32](https://github.com/libar-dev/software-delivery-protocol/pull/32), was not another verb. It was a task-to-recipe sheet in the `sdp-agent-surface` skill and better answer shapes, such as the impacted Packs in recipe 4's summary. + +That is the useful feedback loop. The graph remains stable while the on-ramps and recipes improve from observed use. + +## Repository-owned skills teach the workflow + +A capable surface still needs an on-ramp. The package ships repository-owned skills, counted in the [generated baseline metrics](./README.md#step-3-re-derive-corpus-metrics), under [`.agents/skills/`](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753/.agents/skills): + +- `sdp-agent-surface` teaches graph reading and the task-to-recipe map. +- `sdp-authoring` teaches the carriers, readiness floors, relations and bindings. +- `sdp-sessions` routes advisory delivery work shapes without making them workflow gates. +- `sdp-studio` covers publishing, reading and sharing the Studio and receiving composed intent. +- `sdp-vault` covers the generated Obsidian notes and the hand-off back to source. + +Their standing rule is the anti-anecdote rule: the derived graph outranks a skill, a cached summary or a paraphrase. The skill teaches how to ask. The graph answers. + +## People read the same graph differently + +This article is about the agent surface, but the Protocol does not require people to consume JSON. The Studio, Design Review, census, Mermaid and Gherkin projections read the same graph for human tasks. + +The Studio's browser code does not build a second graph. Its joins and derived values come from the reader during publication. Its composing panel hands wanted changes back as scoped intent rather than mutating source. + +[Part 2](./02-one-graph-many-views.md) gives that side of the system its own article. The two surfaces matter together: an agent can inspect the same Spec and bindings a reviewer just saw, without translating between two delivery databases. + +## Try the agent surface + +From a checkout at the pinned commit: + +```sh +git clone https://github.com/libar-dev/software-delivery-protocol.git +cd software-delivery-protocol +git checkout b5e0a8bea1e3bedd48c616f96c9928e955792753 +npm ci + +# Find a concept. +npm run --silent sdp:q -- "$(cat dist/recipes/06-where-is-this-concept.js)" \ + --params '{"term":"readiness floor"}' --json + +# Read the owner's advisory roadmap. +npm run --silent sdp:q -- "$(cat dist/recipes/53-the-roadmap.js)" \ + --params '{"pack":"pack:roadmap"}' --json + +# Read one Pack as a design, with summary rows. +npm run --silent sdp:q -- "$(cat dist/recipes/54-the-design-slice.js)" \ + --params '{"pack":"pack:spec-studio-v1"}' --json +``` + +In this source checkout, `npm run --silent sdp:q --` supplies the corpus's fixture exclusions. An adopter invokes the installed `sdp q` with its own root and exclusions. + +## Up next + +[Part 5: AI can shape the reading, but it cannot rewrite the record](./05-ai-can-shape-the-reading.md) separates what agents already do from a future AI presentation layer, including the reproducibility question that layer must answer. + +--- + +*About this article: Part 4 in a series introducing the [Libar Software Delivery Protocol](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753). Repository links and command outputs are pinned to commit `b5e0a8be`; re-run the commands rather than quoting them at a later commit.* diff --git a/docs/articles/05-ai-can-shape-the-reading.md b/docs/articles/05-ai-can-shape-the-reading.md new file mode 100644 index 000000000..5a4812891 --- /dev/null +++ b/docs/articles/05-ai-can-shape-the-reading.md @@ -0,0 +1,82 @@ +# AI can shape the reading, but it cannot rewrite the record + +*Part 5 in a series introducing the Libar Software Delivery Protocol. This is a direction article, not a description of shipped AI presentation features. Links are pinned to one commit; see the footer.* + +The Protocol already uses AI agents in two distinct roles. An agent can read the delivery graph and answer questions. It can also receive scoped intent and edit source. A third role belongs to the product's longer-term direction: AI can help decide how a person reads the graph. + +That might mean selecting a useful projection, narrowing a Pack to one concern, explaining a difficult relation, or adapting a review to the reader's context. The appeal is not a universal interface generated on every visit. It is presentation that fits the question while the underlying record stays unchanged. + +This role needs the same honesty discipline as the other two. It also introduces a real design tension: today's projections are deterministic, while language-model output can vary. + +The series should discuss that tension directly. + +## Three roles, three different trust boundaries + +**AI as a reader is built.** Through `sdp q`, an agent scripts the typed graph. Recipes answer recurring questions. A context bundle composes the reading for a design pass within a stated budget, in parts when the scope needs them. Nothing in this path asks the model to invent the model. + +**AI as an editor is built as a hand-off.** A person or agent composes scoped intent. The Studio's composing panel can produce a change proposal and memo. An agent checks the proposal against its base and applies accepted changes as ordinary source edits. Git and the conformance and honesty checks see the same edit they would see from a person. + +**AI as a presentation assistant is a future direction.** The current package does not use an LLM to generate Studio pages, select a lens for a reader, summarize a Pack or rearrange a review. The MCP surface for user-facing apps remains deferred. Runtime observation also remains deferred, and supplied run evidence is not runtime observation. + +Keeping the roles separate prevents a useful reading aid from acquiring authority it never earned. + +## What AI may shape + +A presentation assistant could work at several levels. + +It could choose among deterministic artifacts. Given a reviewer's question, it might open a design slice, a capability map, a Mermaid diagram or a Spec's source context. The assistant chooses the reading; the existing projection supplies the content. + +It could compose a new deterministic projection through code. An agent proposes a projection and its checks. Once reviewed and built, that projection reads the graph like any other. AI helped write the generator, not the answer on each render. + +It could produce explanatory prose or a bespoke arrangement beside a deterministic projection. A summary might explain why two Specs conflict or translate a readiness floor failure into plainer language. This is where the fine-tuning and presentation finesse become valuable, and where the current guarantees need careful extension. + +All three patterns are options to explore. None is a ratified product design today. + +## The determinism question cannot be waved away + +Today, `graph = f(repo)` and a shipped projection is a pure, disposable, regenerable function of the graph. The same graph and named inputs produce the same bytes. That property makes staleness falsifiable and lets the repository's checks compare regenerated output. + +An LLM-written summary does not automatically inherit that property. Model versions, decoding choices and prompt context can change the words even when the graph has not changed. + +A mature design must decide where that variation belongs. It could cache generated prose as a named artifact, record its model and source commit, and present it beside the deterministic view. It could restrict AI to selecting deterministic content. It could define a separate explanatory projection with a different reproducibility contract. These are design questions, not implementation details to hide. + +The wrong answer is to call every generated sentence a projection and assume the existing guarantee still applies. + +The same caution applies to layout. An adaptive interface may reorder information, but it must not drop a blocking question because the summary seemed complete, turn an inferred edge into an authoritative one, or hide supplied evidence whose commit does not match the page's stamp. + +## The record keeps its boundaries + +Whatever presentation mechanism emerges, several boundaries remain useful tests for it. + +- A model may explain a `declared` relation, but it may not present its own inference as `declared` intent. +- A model may summarize a binding, but `implemented` still means a resolving `satisfies` edge and `has-verifier` still means a resolving verifier exists. +- A model may discuss supplied run results, but it may not turn them into `observed` or imply that a verifier passed when no run record says so. +- A model may recommend a readiness statement. Only an author states readiness, and the structural floor still checks that statement. +- A model may propose a source edit. A view does not become canonical by displaying that proposal. +- A model may omit detail for readability. It must preserve the scope and what was left out, as a context bundle does. + +These rules do not guarantee that a summary is good. They make its claims inspectable. + +## A better goal than "AI-generated UI" + +The seed design treated AI context as a first-class projection and the Studio as a place to explore and refine a living model ([the seed projection design](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/v0-design/07-spec-studio-and-projections.md)). The current system has made the graph, agent surface, Studio and intent hand-off real. + +The next useful step is not a button that generates an entire dashboard. It is a set of measured experiments: + +1. Can an assistant select the right existing reading for a review task more reliably than a reader navigating alone? +2. Does a generated explanation help a reviewer understand a Pack without making them less likely to inspect its Specs? +3. Can a compact, reader-specific projection preserve every claim and omission needed for an honest decision? +4. Which generated artifacts need a model, prompt and source-commit record to remain reviewable? +5. When does adaptive presentation save effort, and when does a stable page structure make review easier? + +The first UX/UI design pass can explore these questions without committing the product to AI-rendered pages. The deterministic Studio remains a baseline and a fallback. + +The strategic value is flexibility within explicit contracts. We can test additional readings and explore AI assistance while preserving the current deterministic surfaces. Reshaping Design Review, census, Mermaid or Gherkin requires a decision that refines or supersedes the [shipped-projections decision](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/decisions/shipped-projections-frozen.sdp.md) (MD-32); Studio changes follow its own carrying Specs. The interface can keep changing. The record does not change with it. + +## Up next + +Part 6, *Shared language is a performance feature*, returns to the construction of the Protocol: why precise terms reduced ambiguity for people and agents, and how the carrier competition chose Markdown without closing every authoring option. + +--- + +*About this article: Part 5 in a series introducing the [Libar Software Delivery Protocol](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753). Repository links are pinned to commit `b5e0a8be`. Current capabilities are distinguished from design options throughout; the AI presentation direction is not shipped functionality.* diff --git a/docs/articles/README.md b/docs/articles/README.md new file mode 100644 index 000000000..468400891 --- /dev/null +++ b/docs/articles/README.md @@ -0,0 +1,260 @@ +# Media plan and living series playbook + +> **Pinned source baseline:** commit [`b5e0a8be`](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753), the merge of [PR #36](https://github.com/libar-dev/software-delivery-protocol/pull/36). +> **Posture:** This directory is maintained on `main` through focused refresh PRs as the Protocol advances. The opening arc used a standing PR for review, but no permanent media diff is needed. Current behavior, seed-design intent and future directions must read as three different things. Re-run Protocol-corpus measurements at a new pin rather than inheriting them; adopter and historical counts keep their cited measurement sources. Evidence verification always uses a separate checkout of the exact source pin, so manuscript commits cannot change the answers being verified. The sync creates and removes that disposable checkout in both local and CI runs; its writes are described in Step 4. + +## 1. The series argument + +Software teams keep intent in tickets and product documents, architecture in diagrams and wikis, decisions in ADRs, acceptance criteria in test files and run results in CI. The links between them are usually prose. AI coding agents inherit that fragmentation and must guess at structure. + +Libar Software Delivery Protocol (`@libar-dev/software-delivery-protocol`, CLI `sdp`) models delivery as typed repository content, derives one graph and generates consumer-specific readings from it. Its founding slogan is: + +> Specs are code; the graph is derived; the `claim` stays honest; git is the event log. + +The series has to carry four consequences, not just the engine behind them: + +1. **One Spec, enriched in place.** Eight kinds, three altitudes and four stated readiness rungs cover delivery intent without migrating an idea through artifact types. +2. **Authored intent and derived facts stay separate.** `declared`, `anchored` and `inferred` claims are never collapsed. `implemented` and `has-verifier` are derived from bindings; `observed` remains deferred. +3. **Projections are regenerable readings.** Markdown, Mermaid, Gherkin, HTML, CLI answers and generated notes are readings, not competing stores. Reshaping Design Review, census, Mermaid or Gherkin requires a decision that refines or supersedes the [shipped-projections decision](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/decisions/shipped-projections-frozen.sdp.md) (MD-32). The Studio can evolve under its own Specs and remains working material for UX learning. +4. **AI has explicit trust boundaries.** Agents already read the graph and apply scoped intent. AI-assisted presentation is a future direction that must preserve those boundaries and address non-determinism honestly. + +The pre-code seed design in [`docs/lineage/v0-design/`](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/v0-design) is primary context for the product experience. It is historical evidence, not current law. Current Specs decide what is built and ratified. + +## 2. Audiences and distribution + +| Audience | Problem to lead with | Protocol story | Main articles | +|---|---|---|---| +| Staff engineers and architects | Design and code drift apart; impact and ownership are hard to reconstruct | Stable Spec identities, source anchors, claim-aware graph, architecture and design slices | 1, 3, 7, 9 | +| AI and coding-agent builders | Verb walls and raw-file search waste calls and context | `sdp q`, typed reader, recipes, context bundles and explicit edit hand-offs | 1, 4, 5, 9 | +| Engineering leaders and technical founders | Process theater and hand-authored status create false confidence | Readiness floors instead of quotas, live backlog and drift queries, evidence with named limits | 1, 2, 5, 6 | +| Product, QA and design reviewers | The repository holds the design but the useful review job is hard to perform | Stamped Studio previews, task-based projections, open questions, run evidence and scoped intent | 2, 3, 5, 9 | +| UX/UI designers | The product has many possible readings and no settled visual language | One graph, disposable projections, visible honesty, current Studio as prototype | 2 and 5, with 1 as orientation | +| BDD practitioners | Scenario text and step regexes drift; non-behavioral truth fits Gherkin poorly | Typed example spaces, generated contracts, oracles and lawful carriers | 3 and 8 | + +### Channels + +- Publish long-form manuscripts from `docs/articles/` to the project blog, dev.to or another canonical home. Keep commit-pinned repository links and runnable try-it blocks. +- Convert Markdown tables to lists or accessible images on platforms that do not render tables. +- Lead technical launches with Articles 1, 4 and 7. Lead design and product discussions with Article 2. +- Generate the articles' Markdown tables, source excerpts, CLI output and Mermaid blocks at the publication pin. Screenshots are optional UX review material, not the primary evidence and never a substitute for regenerable content. +- Use compact versions of Articles 1, 2 and 5 as the first UX/UI design packet, together with the Studio page set generated at the source pin. Article 2 already contains the compact design brief. The generated pages let the designer inspect the current interface without relying on screenshots. The packet does not need to wait for public serialization. +- For audio or NotebookLM, supply the five full manuscripts and the reading path below, not the whole repository. + +## 3. Source reading path + +### Current vocabulary and jobs + +1. [`CONTEXT.md`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/CONTEXT.md), the ratified language. +2. [`docs/concept/00`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/concept/00-vision-scope-and-mvp-boundary.md) and [`01`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/concept/01-founding-principles-and-invariants.md), scope and principles. +3. [`jtbd-stories/05`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/jtbd-stories/05-see-and-share.md) and [`06`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/jtbd-stories/06-edit-through-the-lens.md), the human reading and edit jobs. + +### Seed design and annotation evidence + +4. [`docs/lineage/README.md`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/README.md), the lineage guide and annotation-scout pointer. +5. Seed [`00`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/v0-design/00-vision-and-product.md), [`01`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/v0-design/01-core-primitives.md) and [`02`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/v0-design/02-system-architecture.md), the original model and canonical/derived split. +6. Seed [`07`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/v0-design/07-spec-studio-and-projections.md) and [`08`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/lineage/v0-design/08-delivery-evidence-and-tooling.md), the original human experience, AI slices and evidence ambition. +7. The [short annotation context](../lineage/annotation-context.md), with the archived scout behind it and the rulings carried by [PR #29](https://github.com/libar-dev/software-delivery-protocol/pull/29). It explains why complete annotation exclusion was an overcorrection and why source bindings are critical to projections. + +### Current carrying Specs and on-ramps + +8. [`spec:model.core-model`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/model/core-model.sdp.md) and [`spec:consumers.projections-model`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/projections-model.sdp.md). +9. [`spec:consumers.spec-studio`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/spec-studio.sdp.md), its lenses, sharing and intent-panel Specs in the same directory. +10. [`spec:consumers.agent-surface`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/agent-surface.sdp.md), [`spec:consumers.context-bundle`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/context-bundle.sdp.md) and [`spec:consumers.design-slice`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/consumers/design-slice.sdp.md). +11. The repository-owned skills under [`.agents/skills/`](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753/.agents/skills) and the [recipe catalog](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/docs/agent-surface/recipes.md). +12. [`examples/checkout-v1`](https://github.com/libar-dev/software-delivery-protocol/tree/b5e0a8bea1e3bedd48c616f96c9928e955792753/examples/checkout-v1), the worked engineering loop. + +For any corpus fact, query the graph before reading a carrier. The source list is reading material for prose and design context, not permission to bypass the graph for measurements. + +## 4. The ten-part series + +| # | Title and file | Status | Job in the series | +|---|---|---|---| +| 01 | [The spec is the prompt is the design](./01-the-spec-is-the-prompt-is-the-design.md) | Full draft | The problem, one Spec, one graph, claim separation and the two check families | +| 02 | [One graph, many views: the interface is allowed to change](./02-one-graph-many-views.md) | Full draft and UX/UI input | The seed's human scenario updated to current mechanics, the projection family, Studio honesty and a compact designer's brief | +| 03 | [From rough idea to verified code](./03-walkthrough-intent-to-verified-code.md) | Full draft | The engineering loop through the worked example, bindings, contracts, oracles and deliberate failures | +| 04 | [Stop building 30-verb CLI walls for coding agents](./04-stop-building-verb-walls-for-coding-agents.md) | Full draft | Demand map, `sdp q`, recipes, bundles in parts, design slices and measured agent use | +| 05 | [AI can shape the reading, but it cannot rewrite the record](./05-ai-can-shape-the-reading.md) | Full direction draft | Built AI roles versus future presentation assistance, including the deterministic-projection tension | +| 06 | Shared language is a performance feature | Detailed outline | Vocabulary decisions, readiness wording and the four-way carrier competition | +| 07 | Excluding annotations was the mistake: what three generations taught us | Detailed outline | Gen 1, the pre-code seed and the current Protocol; the return of source bindings and what projections need from them | +| 08 | Executable specifications without Cucumber glue | Detailed outline | The design behind typed example spaces, contracts and expected-outcome oracles, without repeating Part 3's tutorial | +| 09 | Designing a large platform ahead of its code | Detailed outline | The first adopter, entry addresses, bundles in parts, design slices, corpus rules and roadmap readings | +| 10 | Auditing your own corpus with cross-family AI reviewers | Detailed outline | A transferable review method, with the historical whole-corpus audit as evidence rather than the headline | + +### Article 06: *Shared language is a performance feature* + +- Open on the cost of ambiguous words such as `status`, `abstraction` and `provenance` for people and agents. +- Explain the governing rubric in `CONTEXT.md`: adopt established delivery nouns, reject process gates, coin a term only for a real difference. +- Use the seed's "readiness is a claim" as the concrete collision. Today readiness is stated, while `claim` names epistemic status. The wording change records a model decision. +- Cover the rename ledger and why a rejected name may return with a narrower job, as `marker` did for carrier syntax. +- Tell the four-way carrier competition through its exhibits and probes. Explain why Markdown won the default, why TypeScript remains lawful per ID and why graph-aware Gherkin returned for behavior and example Specs. +- Keep one canonical carrier per ID as the closing rule. + +### Article 07: *Excluding annotations was the mistake* + +- Start with the apparent contradiction: Gen 1's tag sprawl was a real failure, but removing source annotations altogether cut off the bindings that make architecture, traceability and impact projections valuable. +- Use three generations rather than a simple old-versus-new story. Gen 1 proved production value and taxonomy drift. The pre-code seed restored structured source markers to the design. The early current Protocol overcorrected by refusing JSDoc identity tags and restricting bindings. +- Let the annotation scout supply the evidence: formal vocabulary cleanup was not the same as a shipped runtime registry shrinking; multiple extractors do not require multiple read models; source bindings feed generated documentation. +- Explain why adoption made the cost visible. Runtime code that could not import the package had to bind in tests, and source-file queries missed the actual units. +- Cover the comment-form return, optional plural bindings, `satisfies` versus `references`, and architectural `role`, `layer` and `context` where they are realized. +- Connect the ruling to Article 2: a projection cannot show a component, realization or design reference the graph does not hold. Anchors are identity and structure, never Spec prose, and that narrowness is what makes them safe to project. +- Include what did not return: sprawling tags, workflow statuses in code and hand-authored delivery facts. + +### Article 08: *Executable specifications without Cucumber glue* + +- Lead with regex rot and with the deeper problem of tests verifying values that no longer match the authored example. +- Explain carrier independence: Markdown and Gherkin can derive the same example space and contracts. +- Focus on the authored/generated direction, point-per-example, bound points and the outcome union. Point back to Part 3 for registrar mechanics. +- Explain the oracle's job and `unspecified`, including why [`spec:extraction.runnable-modules`](https://github.com/libar-dev/software-delivery-protocol/blob/b5e0a8bea1e3bedd48c616f96c9928e955792753/specs/extraction/runnable-modules.sdp.md) makes a bound example whose oracle selects `unspecified` fail rather than silently witness a gap. The engine at the source baseline has one exception: it still passes such an example when its Then step's own contract key is `unspecified`, a gap the whole-repository review in [PR #38](https://github.com/libar-dev/software-delivery-protocol/pull/38) records as L1-02. Re-check the rule at the article's own pin before stating it without that exception. +- Use the Protocol's own reader Spec in `.sdp.gherkin` as the self-hosting example. +- Keep the boundary clear: `has-verifier` records a binding, supplied run evidence records results, and `observed` remains a separate deferred fact. + +### Article 09: *Designing a large platform ahead of its code* + +- Use the first adopter's historical 243-Spec pin as the opening scale example, not as a current count. +- Show why design ahead of code needs entry addresses, pinned declarations and planned components rather than fabricated implementation bindings. +- Follow one design pass through a context bundle. Explain deterministic cuts, complete scopes and ordered parts. Do not repeat the retired claim that a bundle never returns a subset of members. +- Add the design slice as the whole-Pack reading and the design diff as the comparison between passes. +- Explain change proposals against a named base and corpus rules that remain separate from Protocol findings. +- Close with capability maps, design slices and the priority roadmap as current readings of planned work, with recipe 53 as the live source of next work. + +### Article 10: *Auditing your own corpus with cross-family AI reviewers* + +- Lead with the method, not the finding count: one lens checks Specs against code and tests, another checks cross-Spec consistency and language, and a different model family verifies each finding. +- Use the historical 258-Spec, 595-finding audit as the evidence at its own pin. Include the 59 refuted findings so review does not become a volume contest. +- Show what the method caught: code that broke promises, two homes of one law and stale authoring guidance. +- Explain why self-hosting makes drift testable and why the self-hosting oracle must be re-transcribed rather than inherited. +- Close with slimming candidates and curation advice as different recurring hygiene tasks. + +## 5. Editorial house rules + +1. Use ratified terminology from `CONTEXT.md`. Readiness is stated, never "claimed." The Protocol defines a contract and instances conform. Claim "deterministically validated," never "provably correct." +2. Label time and status. Current realization comes from the pinned graph and source. The seed design is pre-code evidence and direction. AI-assisted presentation is a future direction, not shipped functionality. +3. Do not present today's Studio layout as settled UX. It is a working projection used to learn which readings help. Conversely, do not treat an open visual question as permission to change graph law in prose. +4. Keep AI roles distinct: reading the graph, applying scoped intent and assisting presentation. If an AI-generated view varies, do not silently attribute the deterministic projection guarantee to it. +5. Derive every current count at the named commit. Cite historical adopter and audit figures to the pull request or review that measured them. +6. Run `npm run check:temporal`. Articles must not contain raw calendar dates, numbered plan paths or session/wave/fold tokens. Link the pull request that carried historical material instead of a dated lineage file. +7. Link sibling articles relatively. At publication, rewrite relative links to public URLs or commit-pinned repository URLs. Refresh sibling links after renumbering. +8. Use sentence-case headings and plain language. Cut repeated definitions; cross-link to the article that owns the detailed treatment. +9. Check code fences and links manually. The standard formatter ignores `docs/**`, and the green gate does not prove that article links resolve. +10. Regenerate article evidence at the publication pin and run its check mode. Screenshots are optional UX review material. Name the commit beside any screenshot or shared Pack export. + +## 6. Refresh playbook + +A refresh measures both numbers and qualitative claims. A design diff alone cannot tell you that the series now describes an old Studio phase or misses a new recipe parameter. + +### Step 1: Install and build + +Use Node `^20.19.0 || ^22.12.0 || >=24.0.0`, where the ranges declared by the installed test toolchain (Vitest, Vite and Rolldown) overlap. This source-development requirement is separate from the CLI's declared Node `>=20` runtime floor. + +```sh +npm ci +``` + +Use `npm run --silent sdp:q --` or `pnpm --silent sdp:q` in this checkout. Never invoke a bare `sdp`. + +### Step 2: Diff against the previous pin + +Generate the base graph in a temporary worktree outside the repository. The current previous pin is `b5e0a8bea1e3bedd48c616f96c9928e955792753`. + +```sh +set -euo pipefail +BASE_SHA="b5e0a8bea1e3bedd48c616f96c9928e955792753" +BASE_DIR="$(mktemp -d)/base" +trap 'git worktree remove --force "$BASE_DIR" 2>/dev/null || true' EXIT +git worktree add --detach "$BASE_DIR" "$BASE_SHA" +npm --prefix "$BASE_DIR" ci --ignore-scripts +npm --prefix "$BASE_DIR" run build +node "$BASE_DIR/dist/cli/sdp.js" build "$BASE_DIR" \ + --exclude explorations --exclude examples --exclude test/fixtures/import/parity +npm run --silent sdp:q -- "$(cat dist/recipes/35-the-design-diff.js)" \ + --params "base=@$BASE_DIR/generated/graph.json" --params '{"detail":true}' --json +``` + +Read the roadmap at the new head too: + +```sh +npm run --silent sdp:q -- "$(cat dist/recipes/53-the-roadmap.js)" \ + --params '{"pack":"pack:roadmap"}' --json +``` + +### Step 3: Re-derive corpus metrics + +```sh +set -euo pipefail +npm run --silent sdp:q -- 'const tally = (xs, f) => xs.reduce((a, x) => ((a[f(x)] = (a[f(x)] || 0) + 1), a), {}); return { + schemaVersion: graph.schemaVersion, + specs: g.specs().length, + packs: graph.nodes.filter((n) => n.nodeType === "Pack").map((p) => ({ id: p.id, members: p.members.length })), + nodes: graph.nodes.length, + nodeTypes: tally(graph.nodes, (n) => n.nodeType), + edges: graph.edges.length, + edgeClaims: tally(graph.edges, (e) => e.claim), + locations: graph.locations.length, + readiness: tally(g.specs(), (s) => s.statedReadiness), + kinds: tally(g.specs(), (s) => s.specKind), + deliveryFacts: { + implemented: g.specs().filter((s) => s.deliveryFacts.includes("implemented")).length, + hasVerifier: g.specs().filter((s) => s.deliveryFacts.includes("has-verifier")).length, + observed: g.specs().filter((s) => s.deliveryFacts.includes("observed")).length + } +}' --json + +npm run --silent sdp:q -- "$(cat dist/recipes/01-the-build-backlog.js)" --json +npm run --silent sdp:q -- "$(cat dist/recipes/02-the-drift-alarm.js)" --json +npm run --silent sdp:q -- "$(cat dist/recipes/20-open-question-register.js)" --json +npm run --silent sdp:q -- "$(cat dist/recipes/33-the-design-backlog.js)" --json +ls dist/recipes/*.js | wc -l +``` + + +Measured at commit `b5e0a8bea1e3bedd48c616f96c9928e955792753`: + +| Metric | Value | +|---|---| +| Schema | `0.12.0` | +| Specs and readiness | 456: 447 `ready`, 4 `scoped`, 5 `idea` | +| Packs | 9: `adopter-speed-v1`, `spec-studio-v1`, `corpus-checks-v1`, `design-iteration-v1`, `design-management-v1`, `obsidian-vault-v1`, `roadmap`, `roadmaps-and-slices-v1`, `self-hosting-v1` | +| Nodes, edges and locations | 1,072 nodes (456 Primitive, 9 Pack, 241 CodeNode, 366 Anchor); 2,978 edges (1,929 declared, 1,049 anchored); 3,120 locations | +| Delivery facts | 166 `implemented`, 397 `has-verifier`, 0 `observed` | +| Build backlog | 2, with 232 ready examples and 47 ready decisions audited and excluded | +| Drift alarm | 0 | +| Design backlog | 0 | +| Open questions | 16 across 12 Specs: 10 blocking (6 deferrals, 4 holds), 6 non-blocking | +| Validation | 0 errors and 3 intentional warnings (3 `honesty/gaps`) | +| Shipped recipes | 55 | +| Repository-owned skills | 5 | + + +### Step 4: Regenerate the article evidence + +Marked evidence blocks are generated, not maintained by hand. The sync reads this playbook's pin and fills a disposable worktree from that commit's raw Git blobs, with replacement objects disabled. It installs dependencies with `npm ci --ignore-scripts` and builds the CLI before collecting evidence. The supplied repository provides git objects only: its working files, sparse-checkout settings, attribute filters, installed dependencies and existing build are not used. It covers graph-derived tables, build and query output, carrier excerpts, source anchors, the oracle and Mermaid. Self-hosting validation accepts only `honesty/gaps` warnings; every other warning rule fails the sync. + +```sh +# SOURCE_DIR is a repository checkout containing the pinned commit. +node docs/articles/sync-evidence.mjs "$SOURCE_DIR" + +# Check mode leaves the manuscripts unchanged and fails when any block differs. +node docs/articles/sync-evidence.mjs --check "$SOURCE_DIR" +``` + +Both commands create a temporary worktree under the article checkout's `.scratch/`, install dependencies, build `dist/`, and regenerate `generated/`, test registrars and the example's generated outputs there. They temporarily register the worktree in `$SOURCE_DIR`'s Git metadata and remove it on success or failure; they do not overwrite outputs in `$SOURCE_DIR`. Installing dependencies may populate npm's cache and needs registry access or a populated cache. Sync mode also writes the marked manuscript blocks. Check mode writes no manuscript. + +Always pass the source repository explicitly. Add a marked block and its generator whenever a manuscript gains a reusable output excerpt. Missing, duplicate, unknown, nested or fenced markers fail the check. Keep elision explicit, as in the constant-anchor excerpt. Do not paste a modified version of source or command output and call it an example. + +Mermaid stays live source. Markdown tables come from graph queries. Screenshots remain optional supplements for discussing visual design, never the only record of behavior. + +### Step 5: Check version-sensitive prose + +- Recount recipes and skills in the [generated baseline metrics](#step-3-re-derive-corpus-metrics). +- Check `sdp q` bindings. The current sink injects `g`, `graph`, `report`, `params` and `recipes`. +- Re-read the Studio's carrying Specs and publish a current page set. Check lenses, roadmap, registers, queue, diff, runs, export and composing panel rather than saying "phase 0" from memory. +- Check context-bundle parameters and behavior. `part` now returns ordered parts; never restore the old blanket statement that partial member lists are refused. +- Re-run the worked example and refresh the [generated build output in Part 3](./03-walkthrough-intent-to-verified-code.md#the-worked-example). +- Keep run evidence, `has-verifier` and `observed` distinct. Supplied CTRF results do not derive runtime observation. +- Review command coverage. Any new reusable output belongs in a generated evidence block. +- Check every sibling link after renumbering, every commit-pinned path, every code fence and every image. +- Record which articles need conceptual changes, not only which tables need new counts. This is the feedback loop between the repository and the series. + +### Step 6: Verify + +Run `npm run check:articles -- "$SOURCE_DIR"` and `npm run check` after staging new article files so preflight can see them. CI runs the same article check with full git history, using its disposable build of the pinned source, so a manuscript branch behind the pin cannot merge stale evidence green. Then manually inspect the rendered Markdown, relative links and commit-pinned links. The main gate does not check those links or all Markdown fence structure. diff --git a/docs/articles/sync-evidence.d.mts b/docs/articles/sync-evidence.d.mts new file mode 100644 index 000000000..1dc6bfcd6 --- /dev/null +++ b/docs/articles/sync-evidence.d.mts @@ -0,0 +1,32 @@ +export function replaceEvidenceBlock( + original: string, + block: { readonly id: string; readonly file: string; readonly language: string | null }, + generated: string, +): string; + +export function summarizeValidation(output: string): string; + +export function evidenceMarkers( + original: string, + file: string, +): Map; +export function validateEvidenceMarkers( + documents: ReadonlyMap, + blocks: readonly { id: string; file: string }[], +): void; +export function parseArguments(args: string[]): { + check: boolean; + printPin: boolean; + source: string | undefined; +}; +export function withPinnedSource( + repository: string, + commit: string, + scratchRoot: string, + action: (source: string) => T, +): T; +export function measurementLabel(commit: string, body: string): string; +export function evidenceBlocks( + sourceRoot: string, + pinnedCommit: string, +): { id: string; file: string; language: string | null; generate: () => string }[]; diff --git a/docs/articles/sync-evidence.mjs b/docs/articles/sync-evidence.mjs new file mode 100644 index 000000000..b69a8674c --- /dev/null +++ b/docs/articles/sync-evidence.mjs @@ -0,0 +1,519 @@ +import { execFileSync, spawnSync } from "node:child_process"; +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + realpathSync, + rmSync, + symlinkSync, + writeFileSync, +} from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +// Markers are structural lines, never examples inside a Markdown fence. +export function evidenceMarkers(original, file) { + const blocks = new Map(); + let active; + let fence; + let offset = 0; + for (const line of original.split("\n")) { + const marker = /^$/.exec(line); + if (/\n${content}\n`; + return original.slice(0, range.start) + replacement + original.slice(range.end); +} + +export function parseArguments(args) { + let check = false; + let printPin = false; + let source; + let separator = false; + const usage = "Usage: sync-evidence.mjs [--check] [--] SOURCE_REPOSITORY | --print-pin"; + for (const argument of args) { + if (argument === "--" && !separator) separator = true; + else if (argument === "--check" && !separator && !check) check = true; + else if (argument === "--print-pin" && !separator && !printPin) printPin = true; + else if (argument.startsWith("-")) + throw new Error(`Unknown or repeated option: ${argument}\n${usage}`); + else if (source !== undefined) + throw new Error(`Only one source repository is allowed.\n${usage}`); + else source = argument; + } + if (printPin ? check || source !== undefined : !source) throw new Error(usage); + return { check, printPin, source }; +} + +// Read raw blobs rather than checking out files: sparse patterns, attributes and filter +// drivers belong to the supplied repository, not the pin. Preserve binary bytes and modes. +function materializeTree(git, commit, source) { + const entries = git(["ls-tree", "-r", "-z", commit]) + .toString("utf8") + .split("\0") + .filter(Boolean) + .map((entry) => { + const tab = entry.indexOf("\t"); + const [mode, type, oid] = entry.slice(0, tab).split(" "); + const file = entry.slice(tab + 1); + if (type !== "blob" || !["100644", "100755", "120000"].includes(mode)) + throw new Error(`Unsupported pinned tree entry: ${mode} ${file}`); + return { mode, oid, file }; + }); + const blobs = git(["cat-file", "--batch"], entries.map(({ oid }) => oid + "\n").join("")); + let offset = 0; + for (const { mode, oid, file } of entries) { + const newline = blobs.indexOf(10, offset); + const [actualOid, type, sizeText] = blobs.subarray(offset, newline).toString("utf8").split(" "); + const size = Number(sizeText); + const start = newline + 1; + const end = start + size; + if ( + newline < offset || + actualOid !== oid || + type !== "blob" || + !Number.isSafeInteger(size) || + size < 0 || + end >= blobs.length || + blobs[end] !== 10 + ) + throw new Error(`Could not read pinned blob: ${file}`); + const target = path.join(source, file); + mkdirSync(path.dirname(target), { recursive: true }); + const content = blobs.subarray(start, end); + if (mode === "120000") symlinkSync(content, target); + else writeFileSync(target, content, { mode: mode === "100755" ? 0o755 : 0o644 }); + offset = end + 1; + } +} + +// The supplied checkout provides objects only. Its files, checkout settings, ignored outputs +// and dependencies never participate. Worktree registration is temporary git metadata. +export function withPinnedSource(repository, commit, scratchRoot, action) { + const root = realpathSync(repository); + const env = { ...process.env, GIT_NO_REPLACE_OBJECTS: "1" }; + const git = (args, input) => + execFileSync("git", ["--no-replace-objects", "-C", root, ...args], { + env, + input, + stdio: "pipe", + maxBuffer: 128 * 1024 * 1024, + }); + const top = git(["rev-parse", "--show-toplevel"]).toString("utf8").trim(); + if (realpathSync(top) !== root) + throw new Error("Pass the source repository root, not a subdirectory."); + mkdirSync(scratchRoot, { recursive: true }); + const temporary = mkdtempSync(path.join(scratchRoot, "article-evidence-")); + const source = path.join(temporary, "source"); + let added = false; + try { + git(["worktree", "add", "--detach", "--no-checkout", source, commit]); + added = true; + materializeTree(git, commit, source); + execFileSync("npm", ["ci", "--ignore-scripts", "--no-audit", "--no-fund"], { + cwd: source, + env, + stdio: "pipe", + }); + execFileSync("npm", ["run", "build"], { cwd: source, env, stdio: "pipe" }); + return action(source); + } finally { + try { + if (added) git(["worktree", "remove", "--force", source]); + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + } +} + +export function measurementLabel(commit, body) { + return `Measured at commit \`${commit}\`:\n\n${body}`; +} + +export function summarizeValidation(output) { + const summary = output.match(/validate: (\d+) errors · (\d+) warnings/); + if (!summary) { + throw new Error("Could not read the self-hosting validation summary."); + } + const counts = new Map([["honesty/gaps", 0]]); + const warnings = [...output.matchAll(/\[warning\] (\S+) —/g)]; + for (const [, rule] of warnings) { + if (!counts.has(rule)) { + throw new Error(`Unexpected self-hosting warning rule: ${rule}`); + } + counts.set(rule, counts.get(rule) + 1); + } + if (warnings.length !== Number(summary[2])) { + throw new Error("Warning breakdown does not match the self-hosting validation summary."); + } + const breakdown = [...counts].map(([rule, count]) => `${count} \`${rule}\``).join(", "); + return `${summary[1]} errors and ${summary[2]} intentional warnings (${breakdown})`; +} + +export function evidenceBlocks(sourceRoot, pinnedCommit) { + const realSourceRoot = realpathSync(sourceRoot); + const exclusions = [ + "--exclude", + "explorations", + "--exclude", + "examples", + "--exclude", + "test/fixtures/import/parity", + ]; + + function runCli(args) { + const result = spawnSync( + process.execPath, + [path.join(sourceRoot, "dist/cli/sdp.js"), ...args], + { + cwd: sourceRoot, + encoding: "utf8", + maxBuffer: 20 * 1024 * 1024, + }, + ); + const output = `${result.stdout ?? ""}${result.stderr ?? ""}`; + if (result.status !== 0) { + throw new Error(`sdp ${args.join(" ")} failed:\n${output}`); + } + return output.replaceAll(realSourceRoot, "").replaceAll(sourceRoot, "").trim(); + } + + function query(body, root = sourceRoot, selfHosting = true) { + const args = ["q", body, "--root", root, "--json", ...(selfHosting ? exclusions : [])]; + return JSON.parse(runCli(args)); + } + + function recipe(number, parameters = {}, root = sourceRoot, selfHosting = true) { + return query( + `return await recipes(${number}, ${JSON.stringify(parameters)})`, + root, + selfHosting, + ); + } + + function sourceText(relativePath) { + return readFileSync(path.join(sourceRoot, relativePath), "utf8"); + } + + function selfHostingCensus() { + const specs = query(`return { + schemaVersion: graph.schemaVersion, + specs: g.specs().length, + packs: graph.nodes.filter((n) => n.nodeType === "Pack").map((p) => ({ id: p.id, members: p.members.length })), + nodes: graph.nodes.length, + nodeTypes: graph.nodes.reduce((a, n) => ((a[n.nodeType] = (a[n.nodeType] || 0) + 1), a), {}), + edges: graph.edges.length, + edgeClaims: graph.edges.reduce((a, e) => ((a[e.claim] = (a[e.claim] || 0) + 1), a), {}), + locations: graph.locations.length, + readiness: g.specs().reduce((a, s) => ((a[s.statedReadiness] = (a[s.statedReadiness] || 0) + 1), a), {}), + deliveryFacts: { + implemented: g.specs().filter((s) => s.deliveryFacts.includes("implemented")).length, + hasVerifier: g.specs().filter((s) => s.deliveryFacts.includes("has-verifier")).length, + observed: g.specs().filter((s) => s.deliveryFacts.includes("observed")).length + } + }`); + return { + ...specs, + backlog: recipe(1), + drift: recipe(2), + designBacklog: recipe(33), + questions: recipe(20).totals, + recipes: readdirSync(path.join(sourceRoot, "dist/recipes")).filter((file) => + file.endsWith(".js"), + ).length, + }; + } + + function articleCensus() { + const census = selfHostingCensus(); + return `| Measure | Value | +|---|---| +| Authored Specs | ${census.specs}: ${census.readiness.ready ?? 0} \`ready\`, ${census.readiness.scoped ?? 0} \`scoped\`, ${census.readiness.idea ?? 0} \`idea\` | +| Packs | ${census.packs.length} | +| Source and test bindings | ${census.nodeTypes.CodeNode ?? 0} CodeNodes and ${census.nodeTypes.Anchor ?? 0} Anchors | +| Graph | ${census.nodes.toLocaleString("en-US")} nodes, ${census.edges.toLocaleString("en-US")} edges, ${census.locations.toLocaleString("en-US")} locations | +| Delivery facts | ${census.deliveryFacts.implemented} \`implemented\`, ${census.deliveryFacts.hasVerifier} \`has-verifier\`, ${census.deliveryFacts.observed} \`observed\` | +| Operational build backlog | ${census.backlog.total} | +| Drift alarm | ${census.drift.total} | +| Runnable recipes | ${census.recipes} |`; + } + + function baselineMetrics() { + const census = selfHostingCensus(); + const validation = runCli(["validate", ".", ...exclusions]); + const validationSummary = summarizeValidation(validation); + const skills = readdirSync(path.join(sourceRoot, ".agents/skills"), { + withFileTypes: true, + }).filter( + (entry) => + entry.isDirectory() && + existsSync(path.join(sourceRoot, ".agents/skills", entry.name, "SKILL.md")), + ).length; + const packNames = census.packs.map((pack) => `\`${pack.id.replace(/^pack:/, "")}\``).join(", "); + return `| Metric | Value | +|---|---| +| Schema | \`${census.schemaVersion}\` | +| Specs and readiness | ${census.specs}: ${census.readiness.ready ?? 0} \`ready\`, ${census.readiness.scoped ?? 0} \`scoped\`, ${census.readiness.idea ?? 0} \`idea\` | +| Packs | ${census.packs.length}: ${packNames} | +| Nodes, edges and locations | ${census.nodes.toLocaleString("en-US")} nodes (${census.nodeTypes.Primitive} Primitive, ${census.nodeTypes.Pack} Pack, ${census.nodeTypes.CodeNode} CodeNode, ${census.nodeTypes.Anchor} Anchor); ${census.edges.toLocaleString("en-US")} edges (${census.edgeClaims.declared.toLocaleString("en-US")} declared, ${census.edgeClaims.anchored.toLocaleString("en-US")} anchored); ${census.locations.toLocaleString("en-US")} locations | +| Delivery facts | ${census.deliveryFacts.implemented} \`implemented\`, ${census.deliveryFacts.hasVerifier} \`has-verifier\`, ${census.deliveryFacts.observed} \`observed\` | +| Build backlog | ${census.backlog.total}, with ${census.backlog.excludedReadyExamples} ready examples and ${census.backlog.excludedReadyDecisions} ready decisions audited and excluded | +| Drift alarm | ${census.drift.total} | +| Design backlog | ${census.designBacklog.totals.specs} | +| Open questions | ${census.questions.questions} across ${census.questions.specs} Specs: ${census.questions.blocking} blocking (${census.questions.deferrals} deferrals, ${census.questions.holds} holds), ${census.questions.nonBlocking} non-blocking | +| Validation | ${validationSummary} | +| Shipped recipes | ${census.recipes} | +| Repository-owned skills | ${skills} |`; + } + + function exampleBuild() { + return runCli(["build", "examples/checkout-v1"]) + .split("\n") + .filter((line) => /^\d+ specs/.test(line) || line.startsWith("Wrote ")) + .join("\n"); + } + + function exampleDrift() { + const answer = recipe(2, {}, path.join(sourceRoot, "examples/checkout-v1"), false); + return JSON.stringify(answer, null, 2); + } + + function constantAnchor() { + const source = sourceText("examples/checkout-v1/src/orders/create-order.use-case.ts"); + const importLine = source.split("\n")[0]; + const opening = "export const createOrderUseCaseAnchor = codeAnchor({"; + const closing = "\n});"; + const start = source.indexOf(opening); + const end = start < 0 ? -1 : source.indexOf(closing, start + opening.length); + if (end < 0) throw new Error("Could not locate the create-order constant anchor."); + const anchor = source.slice(start, end + closing.length); + return `${importLine}\n\n// …\n\n${anchor}`; + } + + function commentAnchor() { + const source = sourceText("src/projections/census.ts"); + const anchorIndex = source.indexOf("@sdpAnchor impl:protocol.census-page"); + const start = source.lastIndexOf("/**", anchorIndex); + const end = source.indexOf("*/", anchorIndex); + if (anchorIndex < 0 || start < 0 || end < 0) + throw new Error("Could not locate the census comment anchor."); + return source.slice(start, end + 2); + } + + function oracle() { + const source = sourceText("examples/checkout-v1/test/orders/create-order.oracle.ts"); + const start = source.indexOf("export function expected("); + const end = source.indexOf("\n}", start); + if (start < 0 || end < 0) throw new Error("Could not locate the create-order oracle."); + return source.slice(start, end + 2); + } + + function byFile() { + return JSON.stringify(query(`return g.byFile("src/projections/census.ts")`), null, 2); + } + + function roadmapMermaid() { + runCli(["mermaid", "examples/checkout-v1"]); + const page = sourceText("examples/checkout-v1/generated/mermaid/pack/checkout-roadmap.md"); + const opening = "```mermaid\n"; + const start = page.indexOf(opening); + const end = start < 0 ? -1 : page.indexOf("\n```", start + opening.length); + if (end < 0) throw new Error("Could not read the generated checkout-roadmap Mermaid block."); + return page.slice(start + opening.length, end); + } + + const blocks = [ + { + id: "self-hosting-census", + file: "01-the-spec-is-the-prompt-is-the-design.md", + language: null, + generate: () => measurementLabel(pinnedCommit, articleCensus()), + }, + { + id: "roadmap-mermaid", + file: "02-one-graph-many-views.md", + language: "mermaid", + generate: roadmapMermaid, + }, + { + id: "example-build", + file: "03-walkthrough-intent-to-verified-code.md", + language: "text", + generate: exampleBuild, + }, + { + id: "create-order-carrier", + file: "03-walkthrough-intent-to-verified-code.md", + language: "markdown", + generate: () => sourceText("examples/checkout-v1/specs/orders/create-order.sdp.md"), + }, + { + id: "valid-cart-carrier", + file: "03-walkthrough-intent-to-verified-code.md", + language: "markdown", + generate: () => + sourceText("examples/checkout-v1/specs/orders/create-order-valid-cart.sdp.md"), + }, + { + id: "example-drift", + file: "03-walkthrough-intent-to-verified-code.md", + language: "json", + generate: exampleDrift, + }, + { + id: "constant-anchor", + file: "03-walkthrough-intent-to-verified-code.md", + language: "ts", + generate: constantAnchor, + }, + { + id: "comment-anchor", + file: "03-walkthrough-intent-to-verified-code.md", + language: "ts", + generate: commentAnchor, + }, + { + id: "oracle", + file: "03-walkthrough-intent-to-verified-code.md", + language: "ts", + generate: oracle, + }, + { + id: "by-file", + file: "04-stop-building-verb-walls-for-coding-agents.md", + language: "json", + generate: byFile, + }, + { + id: "baseline-metrics", + file: "README.md", + language: null, + generate: () => measurementLabel(pinnedCommit, baselineMetrics()), + }, + ]; + + return blocks; +} + +function main() { + const { check, printPin, source: sourceArgument } = parseArguments(process.argv.slice(2)); + const articlesDir = path.dirname(fileURLToPath(import.meta.url)); + const repositoryRoot = path.resolve(articlesDir, "../.."); + const playbook = readFileSync(path.join(articlesDir, "README.md"), "utf8"); + // Bound the link search on each candidate line; repeated incomplete labels must not + // cause rescans of the rest of the playbook. Accept the playbook's quoted bold label + // and the plain label used by small source fixtures. + const pinnedCommit = playbook + .split("\n") + .map((line) => + /^(?:> \*\*)?Pinned source baseline:[^\r\n]{0,200}tree\/([0-9a-f]{40})(?![0-9a-f])/.exec( + line, + ), + ) + .find((match) => match)?.[1]; + + if (!pinnedCommit) { + throw new Error("The publication plan does not name a full pinned source commit."); + } + + if (printPin) { + console.log(pinnedCommit); + return; + } + + const documents = new Map( + readdirSync(articlesDir) + .filter((name) => name.endsWith(".md")) + .map((name) => [name, readFileSync(path.join(articlesDir, name), "utf8")]), + ); + validateEvidenceMarkers(documents, evidenceBlocks(repositoryRoot, pinnedCommit)); + return withPinnedSource( + sourceArgument, + pinnedCommit, + path.join(repositoryRoot, ".scratch"), + (sourceRoot) => { + const blocks = evidenceBlocks(sourceRoot, pinnedCommit); + const drift = []; + for (const block of blocks) { + const filePath = path.join(articlesDir, block.file); + const original = readFileSync(filePath, "utf8"); + const updated = replaceEvidenceBlock(original, block, block.generate()); + if (updated === original) continue; + if (check) { + drift.push(`${block.file}: ${block.id}`); + } else { + writeFileSync(filePath, updated); + console.log(`Updated ${block.file}: ${block.id}`); + } + } + + if (drift.length > 0) { + console.error(`Article evidence is stale:\n${drift.join("\n")}`); + process.exitCode = 1; + } else { + console.log( + `${check ? "Checked" : "Synchronized"} ${blocks.length} evidence blocks at ${pinnedCommit}.`, + ); + } + }, + ); +} + +// Node resolves the main module through symlinks, so compare real paths: a symlinked checkout +// path must still run the check rather than exit 0 having checked nothing. +if ( + process.argv[1] && + realpathSync(path.resolve(process.argv[1])) === realpathSync(fileURLToPath(import.meta.url)) +) { + main(); +} diff --git a/docs/lineage/README.md b/docs/lineage/README.md index a97b972ab..cc9413b7c 100644 --- a/docs/lineage/README.md +++ b/docs/lineage/README.md @@ -1 +1 @@ -This folder holds the v0 design documents restored verbatim with lineage headers and the dated annotation scout, `annotation-scout-2026-10-04.md`, written as evidence for the owner's anchor decisions. Both are historical evidence, never intended truth. The v0 concept set was superseded 2026-06-06 by the reboot and restored from `ed5f6ad^` for the executable-verification design review, `reviews/14`. Intended truth lives in `specs/`, ratified decision Specs under `specs/decisions/`, with their names in `docs/concept/DECISIONS.md`, and current realization in `src/`; on any disagreement this folder is the stale side. Restored documents enter with a lineage header naming the superseding commit; the named scout is the authored-report exception already present here. The pre-ratification vocabulary files (`docs/concept/ubiquitous-language.md`, `docs/concept/GLOSSARY.md`) were deliberately not restored: the first was renamed to root `CONTEXT.md` (live), the second was a post-reboot intermediate it superseded. +This folder holds the v0 design documents restored verbatim with lineage headers and the annotation evidence written for the owner's anchor decisions. Start with [the short annotation context](./annotation-context.md). The [archived annotation scout](./archive-annotation-scout-2026-10-04.md) retains the detailed evidence temporarily and may be removed later. All of this is historical evidence, never intended truth. The v0 concept set was superseded 2026-06-06 by the reboot and restored from `ed5f6ad^` for the executable-verification design review, `reviews/14`. Intended truth lives in `specs/`, ratified decision Specs under `specs/decisions/`, with their names in `docs/concept/DECISIONS.md`, and current realization in `src/`; on any disagreement this folder is the stale side. Restored documents enter with a lineage header naming the superseding commit. The pre-ratification vocabulary files (`docs/concept/ubiquitous-language.md`, `docs/concept/GLOSSARY.md`) were deliberately not restored: the first was renamed to root `CONTEXT.md` (live), the second was a post-reboot intermediate it superseded. diff --git a/docs/lineage/annotation-context.md b/docs/lineage/annotation-context.md new file mode 100644 index 000000000..c72502661 --- /dev/null +++ b/docs/lineage/annotation-context.md @@ -0,0 +1,42 @@ +# Annotation context + +This is the short orientation to the annotation decisions. It is historical and design context, not current law. The carrying Specs under `specs/` and the vocabulary in `CONTEXT.md` remain authoritative. The detailed scout is retained temporarily as [archive material](./archive-annotation-scout-2026-10-04.md). + +## The lesson + +The first generation, `@libar-dev/architect`, proved that source annotations could join code, behavior, decisions and generated documentation in one graph. It also showed the cost of a broad tag registry: the formal vocabulary, parser, checks and guidance drifted apart. + +The pre-code seed design kept structured source markers in the model. The current Protocol's early design then overcorrected. It refused JSDoc identity tags and restricted source bindings too narrowly. Complete annotation exclusion was a mistake: without source bindings, projections cannot show which units realize a Spec, which tests verify it, how components relate or which design existing code follows. + +The correction was not a return to the old tag registry. It was a small binding model with two representations and one graph. + +## What an anchor contributes + +An anchor is a human-written pointer in source code or tests. It binds identity and optional structure into the graph. It does not carry behavior, rationale, readiness, workflow status or delivery facts. + +The graph needs those bindings for several kinds of projections: + +- `satisfies` identifies code that realizes a Spec and confers `implemented` when the edge resolves. +- `verifies`, on a test anchor, identifies a test that verifies a Spec. The edge is `anchored` and can confer `has-verifier`. +- `models` binds an expected-outcome oracle to an example space. +- `references` identifies code that answers to a design without claiming to realize it. It confers no delivery fact. +- `memberOf` and `uses` record curated architectural structure between code units. +- `role`, and `layer` and `context` on component anchors, describe the unit's architectural place. + +These bindings let a projection answer from repository evidence rather than reconstructing architecture from filenames, imports or prose. A binding still does not prove behavior. `implemented` is not liveness, and `has-verifier` is not a passing test. + +An example verifies differently. An example Spec states its own `verifies` relation in its carrier, with the `declared` claim, not through an anchor. That relation confers `has-verifier` on its target only when a resolving test anchor binds the example itself, which makes the example an enabled verifier. + +## The current representations + +The constant form uses a statically extracted top-level builder call. The comment form uses reserved `@sdp*` tags in a top-level block and needs no runtime import. Both feed the same closed anchor envelope and derive the same graph model. + +Bindings are optional and plural. A unit may name only structure, reference an unrealized design, or realize several Specs. Code that follows a decision uses `references`, not `satisfies`. + +Declared intent in Specs, anchored bindings in source and inferred mechanical structure keep their distinct claims. Multiple extractors and representations do not imply multiple read models. + +## What stayed excluded + +The old process metadata did not return. Source anchors do not carry ticket status, hand-authored completion or a second readiness model. Git records history. CI supplies run results separately. The graph derives delivery facts from resolving edges. + +The detailed evidence behind the correction lives in the archived scout and [PR #29](https://github.com/libar-dev/software-delivery-protocol/pull/29). Current law lives in `spec:model.anchors`, `spec:extraction.claim-taxonomy`, `spec:decisions.anchor-comment-form`, `spec:decisions.anchor-binding-grain` and `spec:decisions.architectural-annotation`. diff --git a/docs/lineage/annotation-scout-2026-10-04.md b/docs/lineage/archive-annotation-scout-2026-10-04.md similarity index 100% rename from docs/lineage/annotation-scout-2026-10-04.md rename to docs/lineage/archive-annotation-scout-2026-10-04.md diff --git a/package.json b/package.json index 6de740c47..47a1d89d3 100644 --- a/package.json +++ b/package.json @@ -71,6 +71,7 @@ "format": "prettier --write .", "format:check": "prettier --check .", "check:temporal": "node ./check-temporal.mjs", + "check:articles": "node docs/articles/sync-evidence.mjs --check", "generate:example": "node ./projection-suite.mjs examples/checkout-v1", "generate:self-hosting": "node ./projection-suite.mjs . --exclude explorations --exclude examples --exclude test/fixtures/import/parity", "check:self-hosting": "node ./projection-suite.mjs . --exclude explorations --exclude examples --exclude test/fixtures/import/parity --check-clean", diff --git a/test/article-evidence.test.ts b/test/article-evidence.test.ts new file mode 100644 index 000000000..f2a7086a3 --- /dev/null +++ b/test/article-evidence.test.ts @@ -0,0 +1,650 @@ +import { execFileSync, spawnSync } from "node:child_process"; +import { + chmodSync, + copyFileSync, + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + readlinkSync, + rmSync, + statSync, + symlinkSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +import { describe, expect, it } from "vitest"; + +import { + evidenceBlocks, + evidenceMarkers, + measurementLabel, + parseArguments, + replaceEvidenceBlock, + summarizeValidation, + validateEvidenceMarkers, + withPinnedSource, +} from "../docs/articles/sync-evidence.mjs"; + +const repositoryRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); + +describe("article evidence delimiter scans", () => { + it.each([ + ["backtick", "`", "carriage return", "\r"], + ["backtick", "`", "line separator", "\u2028"], + ["backtick", "`", "paragraph separator", "\u2029"], + ["tilde", "~", "carriage return", "\r"], + ["tilde", "~", "line separator", "\u2028"], + ["tilde", "~", "paragraph separator", "\u2029"], + ])( + "%s fence (%s) followed by %s completes within the subprocess deadline", + (_name, delimiter, _tailName, tail) => { + const script = path.join(repositoryRoot, "docs/articles/sync-evidence.mjs"); + const result = spawnSync( + process.execPath, + [ + "--input-type=module", + "--eval", + `import { evidenceMarkers } from ${JSON.stringify(script)}; + const article = "# Article\\n\\n" + ${JSON.stringify(delimiter)}.repeat(100_000) + + ${JSON.stringify(tail)} + "\\n\\nprose\\n"; + console.log(JSON.stringify([...evidenceMarkers(article, "article.md")]));`, + ], + { encoding: "utf8", timeout: 5_000 }, + ); + expect(result.error).toBeUndefined(); + expect(result.status).toBe(0); + expect(result.stdout.trim()).toBe("[]"); + }, + 15_000, + ); + + const pin = "a".repeat(40); + const cases = [ + { + id: "pin", + file: "README.md", + prefix: "Pinned source baseline:x", + complete: `> **Pinned source baseline:** commit [pin](https://example.test/tree/${pin}).\n`, + expected: pin, + error: "The publication plan does not name a full pinned source commit.", + }, + { + id: "constant-anchor", + file: "examples/checkout-v1/src/orders/create-order.use-case.ts", + prefix: "export const createOrderUseCaseAnchor = codeAnchor({", + complete: + 'import { codeAnchor } from "./anchors";\n// earlier delimiter\n});\n' + + 'export const createOrderUseCaseAnchor = codeAnchor({\n id: "impl:probe",\n});\ntrailing text', + expected: + 'import { codeAnchor } from "./anchors";\n\n// …\n\n' + + 'export const createOrderUseCaseAnchor = codeAnchor({\n id: "impl:probe",\n});', + error: "Could not locate the create-order constant anchor.", + }, + { + id: "roadmap-mermaid", + file: "examples/checkout-v1/generated/mermaid/pack/checkout-roadmap.md", + prefix: "```mermaid\nx", + complete: "Earlier fence\n```\n\n```mermaid\ngraph TD\n A --> B\n```\ntrailing text", + expected: "graph TD\n A --> B", + error: "Could not read the generated checkout-roadmap Mermaid block.", + }, + ]; + + for (const probe of cases) { + it.each(["complete", "missing opening", "missing closing", "adversarial"])( + `${probe.id}: handles %s input within the subprocess deadline`, + (scenario) => { + const temporary = mkdtempSync(path.join(tmpdir(), "sdp-article-scan-")); + try { + const input = + scenario === "complete" + ? probe.complete + : scenario === "missing opening" + ? "unrelated text\n});\n```\n" + : scenario === "missing closing" + ? probe.prefix + : probe.prefix.repeat(100_000) + + (probe.id === "pin" ? `\n${probe.complete}` : ""); + const file = path.join(temporary, probe.file); + mkdirSync(path.dirname(file), { recursive: true }); + writeFileSync(file, input); + // The Mermaid generator runs the CLI first. Keep this probe about parsing its + // generated page, without invoking a full graph build on adversarial text. + mkdirSync(path.join(temporary, "dist/cli"), { recursive: true }); + writeFileSync(path.join(temporary, "dist/cli/sdp.js"), ""); + const script = path.join(repositoryRoot, "docs/articles/sync-evidence.mjs"); + const copiedScript = path.join(temporary, "sync-evidence.mjs"); + copyFileSync(script, copiedScript); + const args = + probe.id === "pin" + ? [copiedScript, "--print-pin"] + : [ + "--input-type=module", + "--eval", + `import { evidenceBlocks } from ${JSON.stringify(copiedScript)}; + console.log(evidenceBlocks(${JSON.stringify(temporary)}, "pin") + .find((block) => block.id === ${JSON.stringify(probe.id)}).generate());`, + ]; + // Synchronous regex work cannot be interrupted by Vitest's own timeout. A + // generous child deadline catches polynomial scans without a tight benchmark. + const result = spawnSync(process.execPath, args, { + encoding: "utf8", + timeout: 5_000, + }); + expect(result.error, `${probe.id}: ${scenario}`).toBeUndefined(); + const succeeds = + scenario === "complete" || (scenario === "adversarial" && probe.id === "pin"); + expect(result.status).toBe(succeeds ? 0 : 1); + if (succeeds) expect(result.stdout.trim()).toBe(probe.expected); + else expect(result.stderr).toContain(probe.error); + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + }, + 15_000, + ); + } +}); + +describe("article evidence CLI entry", () => { + it("runs when invoked through a symlinked checkout path", () => { + const linkParent = mkdtempSync(path.join(tmpdir(), "sdp-article-link-")); + try { + const linkedRoot = path.join(linkParent, "checkout"); + symlinkSync(repositoryRoot, linkedRoot, "dir"); + const result = spawnSync( + process.execPath, + [path.join(linkedRoot, "docs/articles/sync-evidence.mjs"), "--print-pin"], + { encoding: "utf8" }, + ); + expect(result.status).toBe(0); + expect(result.stdout.trim()).toMatch(/^[0-9a-f]{40}$/u); + } finally { + rmSync(linkParent, { recursive: true, force: true }); + } + }); + + it("refuses a repository without the pin before executing its CLI", () => { + const temporary = mkdtempSync(path.join(tmpdir(), "sdp-article-unpinned-")); + const sentinel = path.join(temporary, "executed"); + try { + execFileSync("git", ["init", temporary], { stdio: "pipe" }); + mkdirSync(path.join(temporary, "dist/cli"), { recursive: true }); + writeFileSync( + path.join(temporary, "dist/cli/sdp.js"), + `require("node:fs").writeFileSync(${JSON.stringify(sentinel)}, "executed");`, + ); + const result = spawnSync( + process.execPath, + [path.join(repositoryRoot, "docs/articles/sync-evidence.mjs"), "--check", temporary], + { encoding: "utf8" }, + ); + expect(existsSync(sentinel)).toBe(false); + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("invalid reference"); + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + }); + + it("generates evidence from the materialized pin, never the supplied checkout", () => { + const temporary = mkdtempSync(path.join(tmpdir(), "sdp-article-main-")); + const articles = path.join(temporary, "articles/docs/articles"); + const repo = path.join(temporary, "repository"); + const sentinel = path.join(temporary, "executed"); + const git = (...args: string[]) => + execFileSync("git", ["-C", repo, ...args], { encoding: "utf8" }).trim(); + // Stop at the first generator's CLI call: its origin is the observation under test. + const cli = (origin: string) => + `require("node:fs").appendFileSync(${JSON.stringify(sentinel)}, ${JSON.stringify(origin + "\n")}); process.exit(1);`; + try { + mkdirSync(articles, { recursive: true }); + mkdirSync(repo); + git("init", "-q"); + writeFileSync( + path.join(repo, "package.json"), + JSON.stringify({ + name: "pin-probe", + version: "1.0.0", + scripts: { build: "node build.cjs" }, + }), + ); + writeFileSync( + path.join(repo, "package-lock.json"), + JSON.stringify({ + name: "pin-probe", + version: "1.0.0", + lockfileVersion: 3, + packages: { "": { name: "pin-probe", version: "1.0.0" } }, + }), + ); + writeFileSync( + path.join(repo, "build.cjs"), + `const fs = require("node:fs"); fs.mkdirSync("dist/cli", { recursive: true }); fs.writeFileSync("dist/cli/sdp.js", ${JSON.stringify(cli("materialized"))});`, + ); + writeFileSync(path.join(repo, ".gitignore"), "dist/\n"); + git("add", "package.json", "package-lock.json", "build.cjs", ".gitignore"); + git( + "-c", + "user.name=Test", + "-c", + "user.email=test@example.invalid", + "-c", + "commit.gpgsign=false", + "commit", + "-qm", + "pin", + ); + const pin = git("rev-parse", "HEAD"); + mkdirSync(path.join(repo, "dist/cli"), { recursive: true }); + writeFileSync(path.join(repo, "dist/cli/sdp.js"), cli("supplied")); + + // Run the real entry point with a toy pin, independent of HEAD's engine and corpus. + copyFileSync( + path.join(repositoryRoot, "docs/articles/sync-evidence.mjs"), + path.join(articles, "sync-evidence.mjs"), + ); + const documents = new Map([["README.md", `Pinned source baseline: x/tree/${pin}\n`]]); + for (const block of evidenceBlocks(repo, pin)) { + documents.set( + block.file, + (documents.get(block.file) ?? "") + + `\nold\n\n`, + ); + } + for (const [file, text] of documents) writeFileSync(path.join(articles, file), text); + const result = spawnSync( + process.execPath, + [path.join(articles, "sync-evidence.mjs"), "--check", repo], + { encoding: "utf8" }, + ); + expect(result.error).toBeUndefined(); + expect(result.status).toBe(1); + expect(readFileSync(sentinel, "utf8")).toBe("materialized\n"); + expect(git("worktree", "list", "--porcelain").match(/^worktree /gm)).toHaveLength(1); + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + }); +}); + +// Import the CLI's helpers directly: an import must not validate a checkout or regenerate files. +describe("article evidence replacement", () => { + it.each([null, "text"])("preserves replacement tokens literally with language %s", (language) => { + const opening = ""; + const closing = ""; + const original = `Before\n${opening}\nold evidence\n${closing}\nAfter\n`; + const body = "Literal $$, $&, $`, and $' must survive."; + const block = { id: "probe", file: "article.md", language }; + const expectedBody = language === null ? body : `\`\`\`text\n${body}\n\`\`\``; + const expected = `Before\n${opening}\n${expectedBody}\n${closing}\nAfter\n`; + + const updated = replaceEvidenceBlock(original, block, body); + expect(updated).toBe(expected); + expect(replaceEvidenceBlock(updated, block, body)).toBe(expected); + }); + + it("refuses a missing evidence block", () => { + expect(() => + replaceEvidenceBlock( + "No markers", + { id: "probe", file: "article.md", language: null }, + "body", + ), + ).toThrow("article.md is missing the probe evidence markers"); + }); +}); + +describe("article validation evidence", () => { + it("counts only honesty gaps, including located diagnostics", () => { + expect( + summarizeValidation( + "validate: 0 errors · 3 warnings\n" + + "[warning] honesty/gaps — missing verifier\n" + + "[warning] honesty/gaps — another missing verifier\n" + + "specs/probe.sdp.md:9 — [warning] honesty/gaps — missing verifier\n", + ), + ).toBe("0 errors and 3 intentional warnings (3 `honesty/gaps`)"); + }); + + it("reports zero counts for documented rules without warnings", () => { + expect(summarizeValidation("validate: 0 errors · 0 warnings")).toBe( + "0 errors and 0 intentional warnings (0 `honesty/gaps`)", + ); + }); + + it.each(["honesty/orphans", "conformance/prose-mentions"])( + "refuses to label %s intentional", + (rule) => { + expect(() => + summarizeValidation(`validate: 0 errors · 1 warnings\n[warning] ${rule} — unlinked\n`), + ).toThrow(`Unexpected self-hosting warning rule: ${rule}`); + }, + ); + + it("refuses unparsed warnings and missing summaries", () => { + expect(() => summarizeValidation("validate: 0 errors · 1 warnings\nunknown format")).toThrow( + "Warning breakdown does not match", + ); + expect(() => summarizeValidation("No summary")).toThrow( + "Could not read the self-hosting validation summary", + ); + }); +}); + +const probeBlock = { id: "probe", file: "article.md", language: null }; +const probeText = "\nbody\n"; + +describe("article evidence structure", () => { + it.each([ + ["duplicate", `${probeText}\n${probeText}`], + ["fenced", `\`\`\`\`markdown\n${probeText}\n\`\`\`\``], + ["tilde fenced", `~~~\n${probeText}\n~~~`], + ["shadowed", `\`\`\`\`\n${probeText}\n\`\`\`\`\n${probeText}`], + ["nested", "\n" + probeText + "\n"], + ["unclosed", "\nbody"], + ["unmatched", ""], + ["inline", `prefix ${probeText}`], + ])("refuses %s markers", (_name, text) => { + expect(() => replaceEvidenceBlock(text, probeBlock, "new body")).toThrow(); + }); + + it("refuses unknown, misplaced and missing blocks across every article", () => { + expect(() => { + validateEvidenceMarkers( + new Map([ + ["article.md", probeText], + ["extra.md", probeText.replaceAll("probe", "unknown")], + ]), + [probeBlock], + ); + }).toThrow("unknown evidence block"); + expect(() => { + validateEvidenceMarkers( + new Map([ + ["article.md", probeText], + ["extra.md", probeText], + ]), + [probeBlock], + ); + }).toThrow("duplicate evidence block"); + expect(() => { + validateEvidenceMarkers(new Map([["extra.md", probeText]]), [probeBlock]); + }).toThrow("misplaced evidence block"); + expect(() => { + validateEvidenceMarkers(new Map(), [probeBlock]); + }).toThrow("missing"); + }); + + it.each(["```", "~~~"])("recognizes markers after a CRLF %s closing fence", (fence) => { + const original = `${fence}text\nexample\n${fence}\r\n${probeText}`; + const range = evidenceMarkers(original, "article.md").get("probe"); + expect(range).toBeDefined(); + expect(original.slice(range?.start, range?.end)).toBe(probeText); + }); + + it("preserves nested carrier fences and detects edits independently of HEAD", () => { + const temporary = mkdtempSync(path.join(tmpdir(), "sdp-article-carriers-")); + try { + const orders = path.join(temporary, "examples/checkout-v1/specs/orders"); + mkdirSync(orders, { recursive: true }); + const blocks = evidenceBlocks(temporary, "pin"); + for (const [id, file] of [ + ["create-order-carrier", "create-order.sdp.md"], + ["valid-cart-carrier", "create-order-valid-cart.sdp.md"], + ] as const) { + // Deliberately differs from the published pin: future carrier edits must not + // require an article refresh. check:articles owns equality at the pin. + const source = + `---\nreadiness: ready\n---\nA later carrier edit: ${id}\n\n` + + "```gwt\nGiven a changed example\n```\n"; + writeFileSync(path.join(orders, file), source); + const block = blocks.find((entry) => entry.id === id); + expect(block, id).toBeDefined(); + if (!block) throw new Error(id); + expect(block.generate()).toBe(source); + const original = readFileSync( + path.join(repositoryRoot, "docs/articles", block.file), + "utf8", + ); + const rendered = replaceEvidenceBlock(original, block, source); + expect(replaceEvidenceBlock(rendered, block, source)).toBe(rendered); + const range = evidenceMarkers(rendered, block.file).get(id); + if (!range) throw new Error(id); + const changed = + rendered.slice(0, range.start) + + rendered.slice(range.start, range.end).replace("readiness:", "altered:") + + rendered.slice(range.end); + expect(replaceEvidenceBlock(changed, block, block.generate())).toBe(rendered); + expect(rendered.slice(range.start, range.end)).toContain("````markdown"); + expect(rendered.slice(range.start, range.end)).toContain("```gwt"); + } + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + }); + + it("renders the entire drift recipe answer including firstUnmetClause", () => { + const expected = execFileSync( + process.execPath, + [ + path.join(repositoryRoot, "dist/cli/sdp.js"), + "q", + "return recipes(2)", + "--root", + "examples/checkout-v1", + "--json", + ], + { cwd: repositoryRoot, encoding: "utf8" }, + ); + const block = evidenceBlocks(repositoryRoot, "pin").find( + (entry) => entry.id === "example-drift", + ); + if (!block) throw new Error("Missing drift generator"); + expect(JSON.parse(block.generate())).toEqual(JSON.parse(expected)); + expect(block.generate()).toContain('"firstUnmetClause"'); + }); + + it("moves measurement labels with the pin inside both generated blocks", () => { + expect(measurementLabel("new-pin", "table")).toBe("Measured at commit `new-pin`:\n\ntable"); + for (const id of ["self-hosting-census", "baseline-metrics"]) { + const block = evidenceBlocks(repositoryRoot, "new-pin").find((entry) => entry.id === id); + if (!block) throw new Error(id); + if (id === "self-hosting-census") { + expect(block.generate()).toMatch(/^Measured at commit `new-pin`:/); + } + const original = readFileSync(path.join(repositoryRoot, "docs/articles", block.file), "utf8"); + const range = evidenceMarkers(original, block.file).get(id); + if (!range) throw new Error(id); + expect(original.slice(range.start, range.end)).toMatch(/Measured at commit `[0-9a-f]{40}`/); + const updated = replaceEvidenceBlock(original, block, measurementLabel("new-pin", "table")); + expect(updated.slice(range.start)).toContain("Measured at commit `new-pin`"); + } + }, 60_000); + + it("describes build diagnostics without calling them validation totals", () => { + const article = readFileSync( + path.join(repositoryRoot, "docs/articles/03-walkthrough-intent-to-verified-code.md"), + "utf8", + ); + expect(article).not.toContain("build output above carries the validation totals"); + }); +}); + +describe("article evidence arguments", () => { + it("accepts the forwarded pnpm separator", () => { + expect(parseArguments(["--check", "--", "/source"])).toEqual({ + check: true, + printPin: false, + source: "/source", + }); + }); + + it.each( + [ + ["/source", "--chek"], + ["--verbose", "/source"], + ["one", "two"], + ["--print-pin", "--chek"], + [], + ["--check"], + ["--print-pin", "/source"], + ].map((args) => [args]), + )("refuses invalid arguments %j before touching manuscripts", (args) => { + const article = path.join(repositoryRoot, "docs/articles/README.md"); + const before = readFileSync(article, "utf8"); + const result = spawnSync( + process.execPath, + [path.join(repositoryRoot, "docs/articles/sync-evidence.mjs"), ...args], + { encoding: "utf8" }, + ); + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("Usage:"); + expect(readFileSync(article, "utf8")).toBe(before); + }); + + it("does not offer an implicit source repository", () => { + const playbook = readFileSync(path.join(repositoryRoot, "docs/articles/README.md"), "utf8"); + expect(playbook).not.toContain("omit `$SOURCE_DIR`"); + }); +}); + +// Small real git repositories exercise materialization, npm's lockfile install and the build +// without reinstalling the Protocol's full dependency tree for every regression probe. +describe("pinned evidence source", () => { + it.each(["source", "build", "outputs", "sparse checkout", "replace ref", "smudge filter"])( + "uses only the pin and isolates %s", + (focus) => { + const temporary = mkdtempSync(path.join(tmpdir(), "sdp-article-pin-")); + const repo = path.join(temporary, "repository"); + const scratch = path.join(temporary, "scratch"); + mkdirSync(repo); + const git = (...args: string[]) => + execFileSync("git", ["-C", repo, ...args], { encoding: "utf8" }).trim(); + const read = (root: string, file: string) => readFileSync(path.join(root, file), "utf8"); + try { + git("init"); + writeFileSync( + path.join(repo, "package.json"), + JSON.stringify({ + name: "pin-probe", + version: "1.0.0", + scripts: { build: "node build.cjs" }, + }), + ); + writeFileSync( + path.join(repo, "package-lock.json"), + JSON.stringify({ + name: "pin-probe", + version: "1.0.0", + lockfileVersion: 3, + packages: { "": { name: "pin-probe", version: "1.0.0" } }, + }), + ); + writeFileSync(path.join(repo, "source.txt"), "pinned source"); + writeFileSync(path.join(repo, "omitted.txt"), "always present"); + writeFileSync(path.join(repo, "binary"), Buffer.from([0, 255, 10, 13])); + chmodSync(path.join(repo, "binary"), 0o755); + symlinkSync("source.txt", path.join(repo, "source-link")); + writeFileSync( + path.join(repo, ".gitignore"), + "*.test.generated.ts\ndist/\nnode_modules/\ngenerated/\n", + ); + writeFileSync( + path.join(repo, "build.cjs"), + "const fs = require('node:fs'); fs.mkdirSync('dist'); fs.writeFileSync('dist/result', fs.readFileSync('source.txt'));\n", + ); + git( + "add", + "package.json", + "package-lock.json", + "source.txt", + "omitted.txt", + "binary", + "source-link", + ".gitignore", + "build.cjs", + ); + git( + "-c", + "user.name=Test", + "-c", + "user.email=test@example.invalid", + "-c", + "commit.gpgsign=false", + "commit", + "-m", + "pin", + ); + const commit = git("rev-parse", "HEAD"); + git("update-index", "--skip-worktree", "source.txt"); + writeFileSync(path.join(repo, "source.txt"), "hidden edit"); + writeFileSync( + path.join(repo, "phantom.test.generated.ts"), + "/** @sdpAnchor impl:phantom */", + ); + writeFileSync(path.join(repo, "untracked.txt"), "hidden untracked"); + git("config", "status.showUntrackedFiles", "no"); + for (const dir of ["dist", "node_modules", "generated"]) mkdirSync(path.join(repo, dir)); + writeFileSync(path.join(repo, "dist/result"), "stale build"); + writeFileSync(path.join(repo, "node_modules/foreign.txt"), "foreign dependency"); + writeFileSync(path.join(repo, "generated/sentinel"), "keep output"); + expect(git("status", "--porcelain")).toBe(""); + if (focus === "sparse checkout") { + git("sparse-checkout", "set", "--no-cone", "/*", "!/omitted.txt"); + expect(existsSync(path.join(repo, "omitted.txt"))).toBe(false); + } + if (focus === "replace ref") { + const original = git("rev-parse", `${commit}:source.txt`); + const replacement = execFileSync("git", ["-C", repo, "hash-object", "-w", "--stdin"], { + input: "replaced source", + encoding: "utf8", + }).trim(); + git("replace", original, replacement); + expect(git("show", `${commit}:source.txt`)).toBe("replaced source"); + } + if (focus === "smudge filter") { + writeFileSync(path.join(repo, ".git/info/attributes"), "source.txt filter=probe\n"); + git("config", "filter.probe.smudge", "sed s/pinned/filtered/"); + git("config", "filter.probe.required", "true"); + } + let materialized = ""; + withPinnedSource(repo, commit, scratch, (source) => { + materialized = source; + if (focus !== "build" && focus !== "outputs") + expect(read(source, "source.txt")).toBe("pinned source"); + expect(read(source, "omitted.txt")).toBe("always present"); + expect(readFileSync(path.join(source, "binary"))).toEqual(Buffer.from([0, 255, 10, 13])); + expect(statSync(path.join(source, "binary")).mode & 0o100).toBe(0o100); + expect(readlinkSync(path.join(source, "source-link"))).toBe("source.txt"); + if (focus === "source") { + expect(existsSync(path.join(source, "phantom.test.generated.ts"))).toBe(false); + expect(existsSync(path.join(source, "untracked.txt"))).toBe(false); + expect(existsSync(path.join(source, "node_modules/foreign.txt"))).toBe(false); + } + if (focus === "build") expect(read(source, "dist/result")).toBe("pinned source"); + mkdirSync(path.join(source, "generated"), { recursive: true }); + writeFileSync(path.join(source, "generated/sentinel"), "new output"); + }); + expect(read(repo, "generated/sentinel")).toBe("keep output"); + expect(existsSync(materialized)).toBe(false); + expect(read(repo, "dist/result")).toBe("stale build"); + expect(read(repo, "source.txt")).toBe("hidden edit"); + expect(() => + withPinnedSource(repo, commit, scratch, () => { + throw new Error("probe failure"); + }), + ).toThrow("probe failure"); + expect(readdirSync(scratch)).toEqual([]); + expect(git("worktree", "list", "--porcelain").match(/^worktree /gm)).toHaveLength(1); + } finally { + rmSync(temporary, { recursive: true, force: true }); + } + }, + ); +});