Skip to content

The remaining 167, and the conditions under which the stack lands #216

Closed as not planned
Closed as not planned
The remaining 167, and the conditions under which the stack lands#216

Description

@abernier
abernier
opened on Aug 14, 2026

Part of the distilled-docs spec (#192). Downstream of #214, which settled how an
explainer is written; this one carries the remaining 167 and the conditions under
which the whole thing lands.

Everything below was decided, with reasons, on 2026-08-14. It is written here rather
than left in a session because a cleared session re-derives it wrong — that has already
happened once, and the wrong answer it produced was "merge the stack".

Where it stands

remaining
description empty 33
no apis field 159
no explainer 167

Done, per #192, means all 170 carry a description, an apis list and a README.md
explainer. Three examples are finished: aquarium (#211), inverted-stencil-buffer and
gpgpu-curl-noise-dof (#215).

Decision 1 — nothing merges out of stack #205 until this is done

Not for the append-only reason recorded on #192 earlier. For a different one: the stack
carries lab tooling that nobody wants permanently in main — the describe-example
skill, bin/eval.mjs, the measurement records, and an exception list whose whole subject
is work that has not been done yet.

That tooling is not fat while 167 explainers remain to be written. It is the scaffolding
of the building site: the skill is what produces the prose, the three lints are what keep
it honest, and the ratchet is what guarantees the 33 empties actually get written. "The
smallest possible set of tools" describes the final state, not what should land now.

So the stack stays open, this work stacks on top of it, and the whole thing merges at
once at the end.

Decision 2 — the trim is the last PR

Its condition is written, and it is the same one that closes #192: all 170 carry the
three artifacts. Then, in one PR:

thrown kept
the describe-example skill apis ⊆ imports
bin/eval.mjs the description bound and non-emptiness
bin/eval/baseline.md, bin/eval/pilot-eight.md backticked identifiers ⊆ src/
bin/description-exceptions.mjs, once both lists are empty bin/eval/questions.json

The criterion is the one #198 settled and the grilling confirmed: keep what runs on its
own and fails loudly, drop what only a human can be relied on to run.
questions.json is the named exception — twenty hand-written answers are the expensive
half of an eval and the half a future run could not reconstruct.

Because that criterion condemns the skill, the description and apis contracts were
moved onto their two fields in schemas/pmndrs.schema.json first (commit c8a2396b).
That is where the author of a new example meets them, in an editor tooltip, and it
outlives every tool in the left column. Each file listed there repeats its own expiry in
its own header, so reaching the finish line is not mistaken for a reason to keep them.

Decision 3 — one example per pass, all three fields together

description, apis and the explainer come from one reading of the example, through
the teach orchestration #214 validated. Never split: deriving the one-liner separately
from the explainer is how they end up describing two different demos.

This is why there is no cheap path for the 159 missing apis. Writing them in a bare
pass now would mean reading every example twice, and the second reading would be the one
that counts.

What #214 established about running it, so it is not rediscovered per example:

  • one pass is the normal number. The cap of three is for the demo that genuinely
    resists, not a target. Both pilots finished in one.
  • the orchestrator does not read the example's source before the passes. Read it and
    the distillation arrives as confirmation of something already believed, and the only
    check on the prose is gone.
  • the quiz is the mechanism — the pass predicts, then checks against installed source.
    Both pilots were wrong on most of their predictions, nine of twelve in one case. That
    falsification step is what direct reading does not have.

Decision 4 — the 89 good lines are not rewritten

The ~81 scope from #198 holds, on the signature argument: a line written by the demo's
author that passes every stated test is not replaced by a model's. Evidence on that issue:
transparent-aesop-bottles was rewritten, moved no eval question, lost thickness: 500,
and was reverted.

Note the interaction, which #198 could not have seen: since every example is opened anyway
for its explainer, the cost argument for ~81 has disappeared. Only the signature argument
survives — and it is enough.

What this costs

At the pilot's measured rate — one pass, ~125k tokens, ~13 minutes:

Decision 5 — review the corrections, not the prose

167 maintainer approvals is the binding constraint, not tokens or wall-clock. Settled:
the maintainer approves on the pass's "non-obvious findings" — the list of claims
that could be false, a dozen lines rather than four hundred words — and spot-checks the
prose.

The reasoning is that this is not a weakening. The three lints already carry everything
mechanical. What a human adds is is this true, and the findings section is precisely
where a pass's falsifiable claims are concentrated: it is what the pass got wrong when it
sat its own quiz, stated plainly. Reading four hundred words of confident prose is a worse
test of truth than reading the ten claims the prose rests on.

Rejected: batching by technique family. Faster, and exactly the drift #198 and #214 both
refused — one loaded context approving six examples is the rubber stamp under another name.

Done when

All 170 examples carry a description, an apis list and a README.md explainer;
bin/description-exceptions.mjs has two empty lists; the trim PR of Decision 2 has landed
on top of the stack; #192 closes.

Amendment — what a new example meets after the trim

Asked while writing this: a new example lands with an empty README.md and a fresh
pmndrs.json. Has enough been kept to make it right? Checked against the code rather than
assumed. One of three holds; two do not; and there is a fourth nobody had named.

Holds — an empty description. bin/validate-pmndrs-metadata.mjs already errors on
it, and when bin/description-exceptions.mjs is deleted the UNDESCRIBED.includes(...)
escape goes with it, so the rule becomes unconditional at exactly the right moment. The
contract is in the field's schemas/pmndrs.schema.json tooltip, which is where the author
meets it.

Does not hold — apis. The lint is guarded by if (metadata.apis !== undefined) and
the field is absent from the schema's required. A new example with no apis passes
silently. §PR2 of #192 always intended this: "optional for now — it becomes required only
once every example carries one". Nobody was ever given that step. It belongs in the trim
PR: add apis to required, drop the !== undefined guard.

Does not hold — the explainer's existence. explainerOf() returns nothing for a README
that carries only badges, and the file is then skipped: both prose documents are optional
by construction. The backtick rule verifies the identifiers of prose that exists; it never
reports prose that is missing. A new example with an empty README trips nothing.
Also the trim PR — and it lands unconditional, with no exception list, because by then
all 170 have one. (CONTEXT.md stays optional: #192 §10 is explicit that a glossary which
would only paraphrase the README should not exist.)

Does not hold, and worst — the method. The trim deletes the describe-example skill,
and explain.md with it. That file is the only place where how an explainer is produced
exists: the ephemeral workspace outside the repo, the mission written on the example's
behalf, the teach passes, the quiz that forces falsification, and the rule that the
orchestrator must not read the source first. What was moved into the schema is the
contract of two pmndrs.json fields — not the procedure for the third artifact.
After the trim, a contributor has a lint saying "your backticked identifiers must appear in
src/" and nothing saying what a good explainer is or how to arrive at one.

The writing contract at least — fixed structure (the problem, the technique, the pitfalls),
never walk the code, backticks ⊆ src/ — has to survive somewhere a new example's author
actually looks. docs/agents/domain.md is the candidate: it already carries the two
CONTEXT.md rules from #199, and it is not scaffolding.

Settled: the capability survives the trim. The gallery accepts new examples, so the
means to document one is not scaffolding — it is part of the repo. What gets deleted is
what only ever existed for this campaign: the eval script, the measurement records, the
exception list, and the framing that counts how many of the 170 are still empty. What
survives is what a contributor arriving with example 171 needs:

  • the writing contract — fixed structure (the problem, the technique, the pitfalls),
    never walk the code, every backticked identifier present in that example's src/
  • the procedure — the ephemeral workspace outside the repo, the mission written on the
    example's behalf, the teach passes, the quiz, and the rule that whoever orchestrates
    does not read the source first
  • the two field contracts, already on schemas/pmndrs.schema.json (commit c8a2396b)

The vessel is the trim PR's call — docs/agents/domain.md, which already carries #199's
two CONTEXT.md rules and is not scaffolding, or a describe-example shorn of its
campaign framing. What is not open is whether the capability is kept: it is. A repo that
can lint an explainer's identifiers but can no longer say what a good explainer is has kept
the alarm and thrown the instructions.

What the trim PR actually contains

Traced through the code rather than recalled, so the last PR of the stack has a brief and
not a slogan.

Deleted
.claude/skills/describe-example/SKILL.md the campaign machine
bin/eval.mjs 358 lines, no npm script, run by hand
bin/eval/baseline.md, bin/eval/pilot-eight.md measurement records
bin/description-exceptions.mjs once both lists are empty
test/description-exceptions.test.ts its only subject is those lists

Rewired — the deletions have three live wires, and this is the part that would
otherwise be discovered by breaking the lint:

  • DESCRIPTION_MAX_LENGTH = 120 lives in bin/description-exceptions.mjs and is
    consumed by bin/validate-pmndrs-metadata.mjs. The constant moves into the validator.
  • the two escapes if (!UNDESCRIBED.includes(name)) and if (!(name in OVERLONG)) go, and
    both description rules become unconditional.
  • turbo.json lists bin/description-exceptions.mjs as a lint:metadata input; the entry
    goes with the file.

Added — the two holes a new example falls through today:

  • apis joins the schema's required, and the if (metadata.apis !== undefined) guard in
    the lint is dropped.
  • an explainer presence rule: explainerOf() must return something for every example.
    Unconditional and with no exception list, because by then all 170 carry one.
    bin/lib/explainer.mjs already supplies what it needs and stays — bin/build-llms.mjs
    uses it too. CONTEXT.md stays optional, per Distilled docs: one line to choose, an explainer once chosen #192 §10.
  • tests for both.

Moved — the capability of the amendment above. The writing contract and the
orchestration procedure leave explain.md for docs/agents/domain.md, which already
carries #199's two CONTEXT.md rules and is not scaffolding.

Kept untouched: bin/eval/questions.json, the three bin/lib/*.mjs and their tests,
the lints in bin/validate-pmndrs-metadata.mjs, and the field contracts on
schemas/pmndrs.schema.json.

One deliberate oddity, recorded so it does not read as an oversight: the questions are kept
and the runner is thrown. Rewriting a script that puts twenty questions to a model costs an
hour; rewriting twenty correct answers by hand costs more, and nobody would do it again.

Net effect: lint:metadata comes out stricter, not looser — five unconditional rules
where there are three plus two exception lists today.

Progress

Kept here rather than in a session, for the same reason as everything else on this issue.

The mechanism — done, in the stack, nothing merged

The plan — this issue

  • Decision 1 — nothing merges out of the stack until the end
  • Decision 2 — the trim is the last PR
  • Decision 3 — one pass per example, all three fields together
  • Decision 4 — the 89 good description lines are not rewritten
  • Decision 5 — review the corrections, not the prose
  • Amendment — what a new example meets after the trim
  • The trim PR's exact contents
  • Point 1 — one PR for the whole rollout, one commit per batch. Proposed as ten-per-PR and reversed on the question "why not all of them in one". Decision 5 moved review off the diff — approval happens on the pass's findings, in session, before anything is written — so splitting a document nobody reads as a diff buys nothing; and Decision 1 merges the stack as a block, so it buys nothing at merge time either. What it costs is real: seventeen gh stack adds and seventeen branches to rebase every time anything below them moves, on a stack already eight deep and frozen for weeks. The reviewable unit is the commit, whose message carries the corrections that example returned; git log on the branch is the campaign's journal.
  • Point 2 — order: the empty descriptions first, alphabetical within them. The only list under test, the one that closes Distilled docs: one line to choose, an explainer once chosen #192, and every departure from it is measurable. The other 134 already carry a line that holds.
  • Point 3 — the final eval runs before the trim, which deletes bin/eval.mjs. Baseline 15/20, 19/20 after the Rewrite eight descriptions chosen to break the contract #198 pilot; the last run measures what the whole campaign produced, and after the trim nothing can. ~15-20 min, once, and the number lands here.
  • Point 4 — the surviving method lives in a describe-example shorn of its campaign framing. docs/agents/domain.md is read by an agent exploring a repo, not by someone sitting down to write an explainer; a procedure that can be invoked is found more easily than one that must be looked for. What is thrown is the campaign inside it: the counts of the 170, the expiry section, and the bare mode with nothing left to repair.
  • Point 5 — explain.md's "absence is the normal state" is corrected, in the rollout's second batch. It is the file re-read at every example; leaving it wrong 165 more times is how it would end up being right. Distilled docs: one line to choose, an explainer once chosen #192 §10 keeps the same sentence deliberately — a spec is evidence of what was decided when, and its status comment already marks the passage superseded.
  • Point 6 — the tags work of Distilled docs: one line to choose, an explainer once chosen #192 §4 stays outside this, and is now Normalise the tags, then close a core #217, opened rather than left as "whenever". Standalone, not a sub-issue of Distilled docs: one line to choose, an explainer once chosen #192: that spec's done-when covers description, apis and explainers only, so gating it on tags would contradict a written definition. Its PR branches from main, not from the frozen stack.

The work — 167 remaining

Delivered on #218 unless noted. 7 of 170.

The trim's acceptance test

"We kept enough" is an opinion until something proves it. After the trim lands, and before
this issue closes: add an incomplete example and check the chain still handles it.

A new directory with a pmndrs.json whose description is empty, no apis, and a
README.md carrying only the badges. Then:

  • does lint:metadata demand the description, the apis list and the explainer — all
    three, unconditionally, with no exception list left to hide in?
  • is the writing contract still findable by someone who has not read this issue?
  • can a fresh agent, given only the repo, produce an explainer that passes the lints?

Whatever fails is what the trim took that it should not have. This is the only check that
distinguishes a repo which kept the alarm from one which kept the alarm and the
instructions.

Decision 5, as amended at volume

Settled per Decision 5 that the maintainer approves on the findings before anything is
written. At the point where the passes ran twenty at a time, that became the thing holding
the pipeline, and it was amended: each example is applied and committed as its proposal
lands, and the findings are surfaced afterwards — in the orchestrator's message and in the
commit message, which is where they are durable.

What that preserves and what it gives up, stated plainly. It preserves the substance of
Decision 5: review is still on the falsifiable claims rather than on four hundred words of
prose, and the three lints still run before every commit. It gives up the before. The
safety net is that nothing merges until the whole stack does, so a line the maintainer
dislikes is corrected by another commit rather than by a revert — the same net that makes
the frozen stack tolerable in the first place.

This is recorded rather than done quietly, because a plan that describes a gate nobody
operates is worse than one that admits where the gate moved.

Decision 6 — the lesson is the explainer, transcribed, code and all

Reversing explain.md's original rule, on evidence rather than in the abstract.

The rule was: the reference documents leave the teach workspace and become the explainer, while the lesson, its quiz and the learning records die with the temp directory. Thirty-four explainers were written that way. Then teach-backdrop-and-cables/lessons/0001-*.html was read next to the explainer written from it, and the lesson was plainly the better document. Four things made it better, and the rule had discarded all four:

  • a worked code sample — the twelve lines that actually run the technique, in the shape a reader would write them in their own scene
  • a table — piece, and why it is there, for the cases where three things are each individually load-bearing and prose is bad at saying so
  • pitfalls carrying what settled them — the measurement, the version, the line of installed source
  • a check-yourself — three questions on what the pass itself got wrong, which is the cheapest surviving trace of the falsification the whole orchestration exists to buy

So the lesson is transcribed into README.md as the explainer. Transcribed, not paraphrased: keep everything by default, and drop only what cannot survive the move — the workspace-relative links (the claim is restated inline, since the workspace is not going anywhere public), the HTML scaffolding and its stylesheet, and the sign-off addressed to a learner who has a teaching agent. A gallery reader does not.

No lint change is needed, and that is not luck. bin/lib/explainer.mjs already exempts fenced blocks from the identifier check, with the reason written down at the time: quoted code "may show a reader how to carry the technique into their own scene, where the names are theirs and not ours". Inline spans are still checked, so a claim about this demo still has to name something in its src/.

Three alternatives were considered and dropped. A separate docs/teach/<name>.md, and a per-example examples/<name>/docs/teach.md: both split the explanation away from README.md, which is where bin/build-llms.mjs lifts it from into llms.txt and /examples/<name>.md — the exact thing #192 exists to fix. Keeping it in the README costs nothing and keeps that pipeline whole. Shipping the lesson as HTML: it would need a stylesheet, a serving story that does not exist, and it would leave prettier, the identifier lint and build-llms all behind, for the sole gain of the CSS.

What this costs

The thirty-four explainers already committed are rewritten from their workspaces — richer, not different: the technique they name was established by the same passes. Nothing is thrown away.

The fourteen wave-2 passes that returned without a lessons/*.html do not need re-running. That gap is this issue's own fault, not the passes': the wave-2 MISSION.md said "the end state is a distillation another writer can turn into a 3-section explainer", and one pass quoted it back while explaining that it had therefore skipped the lesson. The mission template has been corrected to name the lesson as the deliverable, with its required shape. For the fourteen already run, the transcriber builds the page from what the workspace does hold — learning records, glossary, resources — plus the example's own src/ for the code sample.

explain.md § "4. The lesson comes back, transcribed" and its writing contract now carry all of this.

Decision 7 — the workspace is archived under the example

The explainer is a transcription: it keeps the lesson's conclusions and drops the apparatus
that established them. That is right for the README and wrong for the record. A pitfall
reading "measured 6.6 ms per bake" arrives with nothing behind it the day somebody disputes
it, and explain.md was going to let the evidence die with the temp directory.

So bin/archive-teach.mjs copies the durable half of each teach workspace to
examples/<name>/docs/teach/ — lessons, reference sheets, learning records, the stylesheets
those pages use, and the four workspace markdown files — and the explainer ends with a line
pointing at it. It travels with degit like everything else in the directory.

The tree shape is preserved rather than inlined into a single page. The lessons link to
../GLOSSARY.md and ../learning-records/… by relative path, 235 times across the set, and
those links keep working for free if the shape survives the move. Inlining was written first
and thrown away: it has to break every one of those links to avoid shipping a page of 404s,
and it needs a markdown renderer in bin/, on the critical path bin/lib/explainer.mjs
argues against having a parser on.

What is copied is an allowlist, not the directory. A pass leaves demo-run/, probe/,
preview/, vite caches and symlinks back into node_modules beside its findings; the scratch
root runs to 100 MB against roughly 120 KB of durable material per example. The allowlist also
fails safe — a pass that invents a new scratch directory is skipped by default rather than
committed by default. Measured cost for the 53 archived so far: 6.1 MB, against an examples/
tree that is already 1.4 GB.

The archiver refuses a workspace where no pass has run: MISSION.md is seeded before the
pass, so its presence proves nothing, and a directory that looks like a record while holding
only a brief is worse than no directory. Fifty-three examples are archived; the other
hundred-and-seventeen are archived as their passes land.

Live progress

Rewritten by the rollout itself at each milestone, so the state survives a lost session.
Everything below is measured by node bin/explainer-report.mjs, not asserted.
Examples carrying an explainer 170 / 170
…with a worked code sample 170
…with a check-yourself 170
Examples carrying a CONTEXT.md 170
UNDESCRIBED 0
OVERLONG 0

teach workspaces exist for all 170, and 170 of them hold a lesson. That is the
measurable number, and it is a floor on passes rather than a count of them: a workspace read
twice keeps one lesson, so the reruns -- the deliberate ones and the accidental collisions --
do not show up here. The branch is explainer-rollout
(PR #218), and the two lists in bin/description-exceptions.mjs reaching zero is what lets the
file go in the trim.

What is left, in order. The remaining teach passes and their transcriptions; the
OVERLONG rewrites; the final eval run; the trim PR of Decision 2 and its acceptance test;
then the stack merges and #192 closes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions