Skip to content

chore: Split docs/knowledge/ into a harvest-only tree and a notes tree, and add audit-scale group queries - #61

Merged
Wahbeh-Mohammad merged 1 commit into
mvpfrom
56-split-docsknowledge-into-a-harvest-only-tree-and-a-notes-tree-and-add-audit-scale-group-queries
Aug 31, 2026
Merged

chore: Split docs/knowledge/ into a harvest-only tree and a notes tree, and add audit-scale group queries#61
Wahbeh-Mohammad merged 1 commit into
mvpfrom
56-split-docsknowledge-into-a-harvest-only-tree-and-a-notes-tree-and-add-audit-scale-group-queries

Conversation

@Wahbeh-Mohammad

Copy link
Copy Markdown
Contributor

Closes #56.

What changed

docs/knowledge/ is now two trees.

  • harvested/ holds what the source documents say. The roles are spec,
    design and styleguide. The knowledge-harvest skill generates this tree.
    Nobody edits it by hand. It holds 38 topic files, 1457 entries, INDEX.md
    and SOURCES.md.
  • notes/ holds what the implementation found. The role is review, with a
    manual sha: marker. A person writes each note by hand. The tree holds 3
    files and 5 entries. A note overrides the harvested entry that it names.

A <sub> line records the sha256 of the whole source file. It does not record
the sha of the entry. Every entry from one document therefore carries the same
value, and an edit to an entry text changes no sha. The next harvest cannot see
the edit. The harvest then writes the original text again, or it writes a
duplicate. A correction inside harvested/ was therefore scheduled for silent
deletion. This split removes that failure.

docs/knowledge/README.md states the contract one time. INDEX.md and
SOURCES.md point to it, because a harvest overwrites both files.

Query surface (scripts/knowledge.mjs)

  • The CLI reads both trees. Each entry carries an origin. Use --origin to
    select one tree.
  • --prefix HTTP selects a whole requirement family. This is the audit-scale
    filter. --req cannot express it. An unknown prefix exits 2. The CLI
    validates the prefix against appendix C.
  • Each entry carries a stable key, <topic>/<8 hex>, made from the entry text.
    A note cites that key. The CLI resolves the citation. It prints
    [overridden by notes/...] on the harvested entry. Use --key to resolve a
    key on demand. The key changes when the rule text changes. That is the moment
    when the note needs a new check.
  • The CLI refuses an empty filter value. --topic '', and the one-character
    error --topic 'a,b,', matched every entry and printed the whole corpus.
  • A query with no result tests each filter alone before it blames one filter.
    For --prefix HTTP --req PAGE-11 the CLI reported "PAGE-11 is canonical but
    no entry cites it yet". That statement is false. Three entries cite PAGE-11.
    None of them cites an HTTP ID.
  • The parser removes a BOM and accepts CRLF. A CRLF topic file parsed to zero
    entries, and the CLI gave no error.

Two checks

  • bun run verify:knowledge-structure is blocking. It is the first step in
    ci.yml, because it runs plain Node over Markdown and needs no build. It
    rejects a review role or an invented role under harvested/. It rejects a
    ci.yml, because it runs plain Node over Markdown and needs no build. It
    rejects a review role or an invented role under harvested/. It rejects a
    Superseded entry there. It rejects an entry with no provenance line. It
    rejects a <sub> that cites a path outside the three source roots that
    SOURCES.md names. It rejects a note that does not carry the review role.
    It rejects a .md file at the root of docs/knowledge/, because a harvest
    without --corpus docs/knowledge/harvested writes there, and no query reads
    that location. The check refuses to report OK below 1000 harvested entries,
    so a failed parse cannot pass over an empty tree.
  • bun run knowledge:drift is a report. It stays out of CI. It compares each
    SOURCES.md sha against the file on disk. It also reports each note citation
    that no entry carries. Three sources drift today. All three are in
    docs/sdk-design-nodejs/. The 16 styleguide sources are NOT VERIFIABLE
    away from the harvest machine. That state is expected. It is never a failure.

Migration

  • Four resolved Conflicts entries are split. Each statement stays in
    harvested/. Each resolution becomes a note. Each note names its statement
    by key.
  • The PAGE-11 erratum moves to notes/pagination.md, under Superseded. The
    inline erratum marker is removed from the harvested Reference entry. That
    entry now prints the override tag instead.
  • deliberate-deviations.md leaves the corpus, together with its SOURCES.md
    row. A register collects rows over time, so a harvest of a register is a
    stale part of it. The copy held 13 entries against a 17-item register, at a
    sha three revisions old, and two entries were false.
    notes/deliberate-deviations.md is the pointer. docs/open-items.md is the
    second register under the same rule.

Documentation

  • .claude/skills/knowledge-lookup/SKILL.md is rewritten. Phase start is two
    queries: --origin note and --section conflicts. The two sets are
    different. A conflict without an override tag is still open. The skill adds
    the audit query as a stated exception to the rule against a broad topic
    query. It records the API-surface audit group and the audit loop. It adds the
    prefix-to-chapter table that completes the roll-up path. Frozen counts are
    removed.
  • CLAUDE.md records the two trees, both checks, and the citation convention.
  • 107 corpus citations in packages/, test/, tests/ and .changeset/ now
    point at harvested/. Two citations named the deleted file. They now name
    the ledger. docs/superpowers/ keeps the paths from before the split. Those
    files are dated records of what a phase planned. Nobody edits them later.

Open items

docs/open-items.md section O records three gaps. O1: the global
knowledge-harvest skill still uses docs/knowledge/ as its default
--corpus, and no query reads that location. O2: a report checks a note key
citation, and no gate checks it. O3: the docs/superpowers/ paths stay as they
are, on purpose. H13 rises in priority, because a blocking gate now depends on
a parser that no CI job tests.

No changeset. No published package surface changes.

Reviews done

Three Opus reviewers ran in parallel against the frozen staged snapshot. Each
reviewer received the issue, the snapshot and one lens. No reviewer saw another
reviewer's report. No reviewer could edit, stage or run a mutating command.

  • Pass 1, shape. The change against the requirement.
  • Pass 2, adversarial. The artifact in use. The reviewer enumerated
    boundaries, failure paths and the state after each failure.
  • Pass 3, reader. The whole artifact, read cold.

Pass 1 accepted the shape. It reported no redesign. Passes 2 and 3 therefore
stand. The three reports gave 25 distinct findings after removal of duplicates.
The session ruled on each finding and applied the fixes. The reviewers applied
nothing.

Findings applied

  • Pass 2 found three blocking defects. A CRLF or BOM topic file parsed to zero
    entries, and the gate reported OK over the hole. A hand-written entry with no
    provenance line, or with an invented role, passed every rule. The key
    mechanism had no way to resolve a key, so the override existed only as prose
    inside one note. All three are fixed.
  • Passes 1 and 2 found that the new phase-start query dropped the two
    unresolved conflicts from view. Phase start is two queries again. The
    override tag separates a resolved conflict from an open one, and the tag
    survives --brief.
  • All three passes found stale corpus citations after the move. The 107 live
    citations are repointed. The paths under docs/superpowers/ stay, and
    CLAUDE.md states why.
  • Passes 1 and 3 found no record for the parts that this change does not fix.
    docs/open-items.md section O is that record.
  • Pass 3 found several counts and claims in prose that were already wrong. Each
    one is corrected or removed.
  • Pass 3 found that the drift script used a verb that no other script in
    scripts/ uses. The file is now knowledge-drift.mjs, which follows
    knowledge.mjs and changeset.mjs.

Findings rejected

  • Pass 1 asked to delete the pagination Conflicts statement, because the
    source document now carries an erratum. The snippet still shows the ordering
    that the statement describes, and the deletion breaks the entry count that
    the issue states. The statement stays. The override tag now shows the
    resolution from the statement.
  • Pass 2 reported no violation of the 70-line function limit, because the lint
    configuration does not apply the rule to .mjs files. Pass 3 reported the
    violation against the styleguide rule itself. Pass 3 is correct. The function
    is split.

Verification after the fixes

Command Result
bun run test:scripts 85 pass, 0 fail
bun run lint exit 0
bun run build exit 0
bun run test 2175 pass, 0 fail
bun run verify:knowledge-structure OK, 1457 harvested entries, 5 notes
bun run knowledge:drift 44 OK, 3 DRIFT, 5 note citations resolve, 0 stale

#56)

A `<sub>` sha digests the whole source file, not the entry. Every entry
harvested from one document therefore carries the same value, and an edit to
an entry's text changes no sha. The next harvest cannot see the edit: it
regenerates the original text, or it writes a duplicate. Hand-written
knowledge under the harvested tree was scheduled for silent deletion.

The corpus is now two trees under docs/knowledge/:

- harvested/ — what the documents say. Roles spec, design, styleguide.
  Generated by knowledge-harvest. Never hand-edited. 38 topic files, 1457
  entries, INDEX.md, SOURCES.md.
- notes/ — what the implementation found. Role review, a manual sha: marker.
  Hand-written. 3 files, 5 entries. A note overrides the harvested entry it
  names.

docs/knowledge/README.md states that contract once. INDEX.md and SOURCES.md
point at it rather than restating it, because a harvest overwrites both.

Query surface (scripts/knowledge.mjs):

- Reads both trees. Every entry carries an `origin`, and `--origin` selects
  one.
- `--prefix HTTP` selects a whole requirement family. It is the audit-scale
  filter that `--req` cannot express. An unknown prefix exits 2, validated
  against appendix C.
- Every entry carries a stable key, <topic>/<8 hex>, digested from the entry
  text. A note cites that key. The CLI resolves the citation, prints
  `[overridden by notes/...]` on the harvested entry, and resolves one on
  demand with `--key`. The key changes when the rule's text changes, which is
  when the note needs a fresh check.
- An empty filter value is refused. `--topic ''` and the one-character typo
  `--topic 'a,b,'` matched every entry and printed the whole corpus.
- A zero-result query tests each filter alone before it blames one. It used
  to report "PAGE-11 is canonical but no entry cites it yet" for
  `--prefix HTTP --req PAGE-11`, which is false: three entries cite PAGE-11,
  and none of them cites an HTTP ID.
- The parser strips a BOM and accepts CRLF. A CRLF topic file parsed to zero
  entries, with no error.

Two checks:

- `bun run verify:knowledge-structure` — blocking, and the first step in
  ci.yml because it is pure Node over Markdown and needs no build. It rejects
  a review or invented role under harvested/, a Superseded entry there, an
  entry with no provenance line, a `<sub>` that cites outside the three
  source roots SOURCES.md names, a note that is not review-role, and a .md
  stranded at the root of docs/knowledge/ — which is where a harvest without
  `--corpus docs/knowledge/harvested` writes. It refuses to report OK below
  1000 harvested entries, so a failing parse cannot pass vacuously.
- `bun run knowledge:drift` — a hand-run report, deliberately not in CI. It
  compares each SOURCES.md sha against the file on disk, and it reports every
  note citation that no entry carries any more. Three sources drift today,
  all in docs/sdk-design-nodejs/. The 16 styleguide sources are NOT
  VERIFIABLE off the harvest machine, which is expected and never a failure.

Migration:

- Four resolved Conflicts entries are split. Each statement stays in
  harvested/. Each resolution is a note that names the statement by key.
- The PAGE-11 erratum moves to notes/pagination.md, under Superseded. The
  inline erratum marker it duplicated is removed from the harvested Reference
  entry, which now prints the override tag instead.
- deliberate-deviations.md leaves the corpus, with its SOURCES.md row. A
  register accumulates rows, so any harvest of one is a stale fraction of it.
  The copy held 13 entries against a 17-item register, at a three-revision-old
  sha, with two entries substantively false. notes/deliberate-deviations.md
  is the pointer. docs/open-items.md is the second register under the same
  rule.

Documentation:

- .claude/skills/knowledge-lookup/SKILL.md rewritten. Phase start is two
  queries, `--origin note` and `--section conflicts`; they are different sets,
  and a conflict with no override tag is still open. Adds the audit query as
  a stated exception to the rule against broad topic queries, the API-surface
  audit group, the audit loop, and the prefix-to-chapter table that completes
  the roll-up path. Frozen counts removed.
- CLAUDE.md records the two trees, both checks, and the citation convention.
- 107 corpus citations in packages/, test/, tests/ and .changeset/ are
  repointed to harvested/. Two cited the deleted file and now cite the ledger.
  docs/superpowers/ keeps its pre-split paths: those files are dated records
  of what a phase planned, and they are not retro-edited.

docs/open-items.md section O records three gaps: the global knowledge-harvest
skill still defaults --corpus to the tree no query reads (O1), a note's key
citation is checked by a report and not by a gate (O2), and the
docs/superpowers/ paths are deliberate (O3). H13 is raised in priority: a
blocking gate now depends on a parser that no CI job tests.

No changeset. No published package surface changes.

Verified: test:scripts 85 pass, lint 0, build 0, test 2175 pass / 0 fail,
structure gate OK, drift 44 OK / 3 DRIFT / 5 note citations resolve.
@Wahbeh-Mohammad
Wahbeh-Mohammad force-pushed the 56-split-docsknowledge-into-a-harvest-only-tree-and-a-notes-tree-and-add-audit-scale-group-queries branch from f920f86 to 2b3a197 Compare August 31, 2026 19:41
@Wahbeh-Mohammad

Copy link
Copy Markdown
Contributor Author

Must an unresolvable note key fail the gate? Today a report finds it. A gate would make a re-harvest that orphans a note a red build. The cost is that the harvest cannot land until the same commit updates the notes. Recorded as O2.

@Wahbeh-Mohammad
Wahbeh-Mohammad merged commit f93ccd9 into mvp Aug 31, 2026
3 checks passed
@Wahbeh-Mohammad
Wahbeh-Mohammad deleted the 56-split-docsknowledge-into-a-harvest-only-tree-and-a-notes-tree-and-add-audit-scale-group-queries branch August 31, 2026 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant