New program: doc-drift (documentation drift detection + doc updates) #8

Why

RepoMan has programs for links, Dependabot bumps, PR review, clone sync, a dashboard and
weekly reports. Nothing checks whether a repo's docs still match its code. Today that
drift is only found by hand. RepoMan's own repo already has several cases (as of b3f128c):

• The entrypoint SKILL.md lists 5 canonical programs, but KNOWN_PROGRAMS in
  repoman-setup.sh has 6 (it adds weekly-report).
• The entrypoint SKILL.md says the run path is "not implemented in this skill", but the
  bundle ships repoman-run.sh, which implements it.
• The repo-sync SKILL.md says the script lives in the automation repo. It is bundled at
  skills/repoman/scripts/repo-sync.sh.
• The repoman-setup.sh header lists 4 subcommands, but the script also implements
  set-requirements.
• The dep-bump fixer's comment signature still says "Kagenti Dep Bump Fixer" (branding from
  before the rossoctl rename).

None of these are broken links, so link-health can't catch them. The PR reviewer only
sees the diff, so it misses docs that a code change made stale somewhere else.

Proposal

Add a doc-drift program (friendly name Doc-Drift-Manager) that follows the existing
scanner/fixer pattern.

Scanner: doc-drift-scanner

Two layers, deterministic first:

1. Deterministic checks (engine script, no model). These are cheap, need no tokens
   and are unit-testable.
   • Referenced paths exist: file and dir paths in backticks or links inside md/rst
     (scripts/foo.sh, skills/x/SKILL.md) are checked against the tree.
   • CLI/flag parity: flags and subcommands named in docs (--dry-run,
     set-requirements) are compared with what the script actually parses (case
     arms, argparse, usage() text, --help output where safe to run).
   • Enumerations: lists the docs repeat (program allowlists, env vars, config keys)
     are compared with their source of truth. Repos opt in with a small rule file
     (below).
   • Stale names: a per-repo denylist of retired names and orgs (e.g. kagenti,
     old org slugs), with an allowlist for CHANGELOG/history.
   • Version strings: versions in README/docs vs marketplace.json, plugin.json,
     package.json, pyproject.toml, tags.
2. Semantic pass (model, optional, capped). For the doc sections that sit next to
   files changed since the last scan (git log since the stored SHA), ask the model one
   question: "Does this section still describe this code?" It must return structured
   JSON with evidence (doc line and code line). No evidence, no finding. The
   per-run cap is set with --max-sections.

Output is like the other scanners: latest.json/history.json diffed against the last
run (new / fixed / recurring), one [doc-drift] issue per finding (or one rollup per
repo, see open questions), resolved issues auto-closed, and an _index.json entry.

Fixer: doc-drift-fixer

• Re-verifies each scanner issue against the current default branch.
• Mechanical fixes (wrong path, stale name, version string, missing enum entry where
  the source of truth is clear) are made in a fork and opened as a PR, as link-health
  does.
• Judgement fixes (prose that no longer describes behaviour) get an issue comment
  with a suggested patch. No PR is opened, and the maintainer decides.
• It never edits code to match the docs. Code is always the source of truth.

Doc updates on code PRs (pr-review hook)

"Docs not updated" should be caught before merge, not only after. A small check in
github-pr-review: if a PR changes CLI flags, config keys, env vars or public script
names and touches no docs that mention them, the review adds a suggestion-severity
comment listing the doc files to update. It reuses the deterministic matchers from the
scanner.

Per-repo config (opt-in)

A .repoman/doc-drift.yaml (optional). Without it the scanner runs only the generic checks
(paths, flags, versions):

sources_of_truth:
  - name: programs
    code: { file: skills/repoman/scripts/repoman-setup.sh, pattern: 'KNOWN_PROGRAMS="([^"]+)"' }
    docs: [skills/repoman/SKILL.md, README.md]
retired_names: [kagenti]
ignore_paths: [CHANGELOG.md, docs/history/**]
semantic: { enabled: false, max_sections: 20 }

Requirements block (draft)

programs: [scanner]        # doc-drift-scanner
scopes: [repo]             # issues; fixer adds fork + PR like link-health
labels: [doc-drift]

Wire it into KNOWN_PROGRAMS, the friendly-name table, the repoman-run.sh profile table
(scanner profile, clone refresh yes, writes index yes, --dry-run yes), and the
automation-health dashboard.

Acceptance criteria

• doc-drift-scanner finds all five b3f128c inconsistencies listed above in
  rossoctl/repoman (fixture or pinned SHA in tests).
• The deterministic layer runs with no model and no network beyond gh.
• Re-running on an unchanged tree produces zero new issues (idempotent).
• Each finding has a file:line on both the doc side and the code side.
• The fixer opens a PR for a mechanical finding and only comments for a semantic one.
• --dry-run and --issue-limit behave as in the other scanners.
• The pr-review "docs not updated" suggestion fires on a PR that adds a flag and
  leaves docs alone, and stays silent when the docs are updated in the same PR.
• The dashboard shows the doc-drift open/fixed counts.
• The engine lands in rossoctl/automation and is vendored with vendor-engine.sh.

Open questions

1. One issue per finding, or one rollup issue per repo with a checklist? Decided: rollup. One rolling update PR + rollup issue per repo; verified corrections get their own small PR (see the prior-art comment). (Drift findings
   are small and numerous, so a rollup may be less noisy.)
2. Should the semantic pass be on by default for enrolled repos, or opt-in only (token
   cost)?
3. Ship the pr-review hook in v1 or as a follow-up?
4. Should the scanner also check SKILL.md Requirements blocks against what each script
   actually calls (gh scopes, labels)? That overlaps with repoman-check.sh.

Related

• Prior art: https://github.com/dam-agents/docs-maintainer (boundary, correction/addition, queue + triage ledger, verification rules; see comment)
• Package RepoMan as a DAM starter kit (curated/repoman) #2 (DAM starter kit: add a schedule entry once this program exists)
• Resync standalone scanner/fixer skills onto the current modular engine lib #5 (standalone skill copies vs the modular engine are drift of the same kind, in code)
• Upstream: rossoctl/automation epic #72