Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,10 @@ jobs:
# tests/test-repoman-setup.sh is deliberately NOT listed here because it
# carries intentional test idioms (SC2088 literal-tilde, SC2034 var read
# by a sourced function) that are load-bearing for what it asserts.
run: shellcheck --severity=warning tests/test-repoman-requirements.sh tests/test-github-api.sh tests/test-repoman-check.sh tests/test-repoman-run.sh tests/test-repo-sync.sh tests/test-repoman-index.sh tests/test-repoman-match-name.sh tests/test-automation-health-dashboard.sh tests/test-link-health-index-wiring.sh tests/test-dep-bump-index-wiring.sh tests/test-skill-meta.sh tests/test-attribution-consumers.sh tests/test-pr-review-integration.sh tests/test-pr-review-impact.sh tests/test-pr-review-state.sh tests/test-weekly-report.sh tests/test-link-health-subset-carryforward.sh tests/test-dep-bump-subset-carryforward.sh tests/test-pr-review-subset-carryforward.sh tests/test-link-health-dryrun-baseline.sh
run: shellcheck --severity=warning tests/test-repoman-requirements.sh tests/test-github-api.sh tests/test-repoman-check.sh tests/test-repoman-run.sh tests/test-repo-sync.sh tests/test-repoman-index.sh tests/test-repoman-match-name.sh tests/test-automation-health-dashboard.sh tests/test-link-health-index-wiring.sh tests/test-dep-bump-index-wiring.sh tests/test-skill-meta.sh tests/test-attribution-consumers.sh tests/test-pr-review-integration.sh tests/test-pr-review-impact.sh tests/test-pr-review-state.sh tests/test-weekly-report.sh tests/test-link-health-subset-carryforward.sh tests/test-dep-bump-subset-carryforward.sh tests/test-pr-review-subset-carryforward.sh tests/test-link-health-dryrun-baseline.sh tests/test-doc-drift-check.sh tests/test-doc-drift-precheck.sh
- name: shellcheck (doc-drift)
# New consumer scripts stay warning-clean (no SC2034 arg-parsing idiom).
run: shellcheck --severity=warning scripts/doc-drift-scanner.sh scripts/doc-drift-fixer.sh scripts/doc-drift-precheck.sh
- name: Run test suite
# NOTE: this list and the shellcheck lists above are EXPLICIT, not globs.
# A new test file must be added here (and to the relevant shellcheck
Expand Down Expand Up @@ -70,7 +73,9 @@ jobs:
tests/test-link-health-subset-carryforward.sh \
tests/test-dep-bump-subset-carryforward.sh \
tests/test-pr-review-subset-carryforward.sh \
tests/test-link-health-dryrun-baseline.sh; do
tests/test-link-health-dryrun-baseline.sh \
tests/test-doc-drift-check.sh \
tests/test-doc-drift-precheck.sh; do
echo "== $t =="
bash "$t"
done
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,6 @@ reports/
# Per-skill provenance written at runtime by resolve_skill_meta (github-api.sh);
# pinned to each checkout, never source-of-truth.
_meta.*.json

__pycache__/
*.pyc
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ reports/ (gitignored) Runtime data stays on remote host only
| PR Review | `scripts/pr-review-scanner.sh` | `scripts/pr-review-fixer.sh` | Scanner finds PRs labeled for review; fixer performs the AI review (posts review + inline comments, advances the review-state label). `scripts/pr-review-impact.sh` measures time-to-merge impact |
| Dependency Bump | `scripts/dep-bump-scanner.sh` | `scripts/dep-bump-fixer.sh` | Tracks and triages Dependabot PRs across repos (comments, closes stale, audits coverage) |
| Health Dashboard | `scripts/automation-health-dashboard.sh` | — | Aggregates program run health into a dashboard |
| Doc Drift | `scripts/doc-drift-scanner.sh` | `scripts/doc-drift-fixer.sh` | Checks a pinned ref for doc drift (paths, flags, enumerations, versions, changelog). See `docs/specs/2026-11-02-doc-drift-design.md` |

Shared helpers live in `scripts/program-lib.sh`, a thin aggregator that sources four focused library modules (see [Library structure & portability](#library-structure--portability)). Repo coverage comes from the `config/core-repos.txt` allowlist via `get_core_repos()` (one **bare repo name** per line -- the owner is prepended from the active org; `#` comments and blank lines allowed) -- edit that file to change which repos are scanned.

Expand All @@ -50,7 +51,7 @@ bash scripts/repoman-run.sh <program> owner/name [owner/name ...]
bash scripts/repoman-run.sh <program> --dry-run
```

`<program>` is one of `link-health`, `dep-bump`, `pr-review`, `weekly-report`, `automation-health`, `repo-sync` (the full set is whatever `repoman-run.sh --help` lists). The orchestrator resolves the repo set, runs the capability check and clone refresh the program needs, execs it, and writes its `_index.json` entry — it is the **sole writer of `_index.json`**, so a program run directly (the commands in the table above) no longer updates the registry the dashboard reads. `repoman run <program>` is the blessed path for an indexed run.
`<program>` is one of `link-health`, `dep-bump`, `pr-review`, `weekly-report`, `automation-health`, `repo-sync`, `doc-drift` (the full set is whatever `repoman-run.sh --help` lists). The orchestrator resolves the repo set, runs the capability check and clone refresh the program needs, execs it, and writes its `_index.json` entry — it is the **sole writer of `_index.json`**, so a program run directly (the commands in the table above) no longer updates the registry the dashboard reads. `repoman run <program>` is the blessed path for an indexed run.

There is no `repoman` executable in this repo yet; `repoman run <program>` maps to `bash scripts/repoman-run.sh <program>`. In a RepoMan deployment the `/repoman` skill (in `rossoctl/agent-skills`) performs that mapping from a Claude Code / OpenClaw agent; scheduled DAM/OpenClaw jobs invoke the same command string. As of v0.1.0 that skill ships as a **self-contained plugin** that bundles this engine (the `scripts/` here are vendored into it), so the skill no longer assumes a checkout of this repo is present. Standing-orders that still call scanners directly are tracked for migration in [#118](https://github.com/rossoctl/automation/issues/118). See the run-path design in `docs/specs/` and the epic [#72](https://github.com/rossoctl/automation/issues/72).

Expand Down
133 changes: 133 additions & 0 deletions docs/specs/2026-11-02-doc-drift-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Doc drift — design

**Status:** implemented (2026-11-02)
**Issue:** [rossoctl/repoman#8](https://github.com/rossoctl/repoman/issues/8)

## Summary

`doc-drift` checks a repo at a pinned git ref and reports places where the
docs disagree with the tree. The checker is offline: `scripts/doc-drift-check.py`
reads only `git show` / `git ls-tree`, and the shell passes release and PR JSON
in. A finding is a correction or an addition. `mechanical: true` means the
fixer has a line-scoped edit; everything else stays on one rollup issue per repo.

Programs:

| Piece | Path |
|---|---|
| Checker | `scripts/doc-drift-check.py` |
| Scanner | `scripts/doc-drift-scanner.sh` |
| Fixer | `scripts/doc-drift-fixer.sh` |
| Precheck | `scripts/doc-drift-precheck.sh` |
| Run path | `doc-drift` row in `scripts/repoman-run.sh` |
| Standing order | `standing-orders/doc-drift.md` |

`repoman-run.sh --help` lists `doc-drift` because the program table is the
source of that list.

## What is checked

| Check | Claim |
|---|---|
| `paths` | A repo-relative path written in a doc is in the tree at that ref |
| `flags` | A flag or dispatcher subcommand written next to a script is accepted by that script |
| `enumerations` | Tokens captured from a `sources_of_truth` regex appear in the named docs |
| `retired_names` | A configured retired string is gone |
| `versions` | A line that states the project version matches the declared version. Changelog history and dependency pins are not version drift |
| `absence` | A "not implemented" or "lives in the automation repo" claim is false because the file is in the tree |
| `release_date` / `release_draft` | A changelog date matches GitHub `publishedAt`, and merged PRs after the newest release are drafted under `Unreleased` |

## Precision

These rules are what keep a scan of this repo from turning into a list of
false corrections.

**Flags bind to the nearest script before them on the same line.** A line
that names two scripts and then a flag is a claim about the second script
only. A flag that appears before any script, as in a shellcheck --severity
invocation that later names a test file, is not a claim about that test.
`--help`, `--version`, and the generic word `--flag` are not claims. Two
identical lines are two findings.

**Subcommands are the dispatcher.** The header list (`Subcommands:` /
`Commands:`) is compared to the arms of `case "$1"` / `$cmd` / `$command` /
`$subcommand` only. Arms of `case "$mode"` are not subcommands. A header line
contributes command tokens until prose starts, so
`init-config, add-repo, set-output. Each writes` is three commands, and
`Each writes a report` is none.

**A script citation names one file.** A path with a directory binds that
path. A bare basename binds only when it is unique. A script that sources
`program-lib.sh` also accepts the long options of that file and of the modules
it aggregates.

**Paths are locations in this repo.** A backtick or link is a path when it has
a repo-file extension (`.sh`, `.py`, `.md`, `.rst`, `.txt`, `.yml`, `.yaml`,
`.json`, `.toml`, `.env`) or a trailing slash. `gemini-2.5-flash`,
`discord.accounts`, and `rossoctl/agent-skills` are not paths. A bare
`link-health-fixer.sh` exists when any tree path has that basename. A slash
path is a miss only when its first directory is in the tree and the rest is
not. Before that miss is reported, the same citation is also tried relative
to the document, so a bundled skill doc that says `scripts/org.sh` matches
`skills/repoman/scripts/org.sh`. Host paths, bare runtime names, and a
relative link that resolves to a real file are not misses. Extensions outside
the list above (`.csv`, `.ini`, `.cfg`, `.conf`, `.js`, `.ts`, `.jsonl`) are
not checked. Path findings are `mechanical: false` because there is no
replacement text. `ignore_paths` drops path and flag findings in those
documents, not only version and retired-name findings.

## Config schema (`.repoman/doc-drift.yaml`)

The file is optional. When it is absent the checker uses these defaults, which
turn enumeration and retired-name checks off:

```yaml
boundary: main
stable_tag_pattern: '^v\d+\.\d+\.\d+$'
ignore_paths:
- CHANGELOG.md
- docs/history/**
sources_of_truth: []
retired_names: []
release:
changelog: CHANGELOG.md
```

| Key | Meaning |
|---|---|
| `boundary` | `main` (resolve `HEAD`) or `tags` (newest tag matching `stable_tag_pattern`) |
| `stable_tag_pattern` | Regex for release tags. An invalid pattern is a `config` finding, not a crash |
| `ignore_paths` | Glob-ish prefixes (a directory plus `/**`, or a trailing `*`) skipped by the version, retired-name, path, and flag checks |
| `sources_of_truth` | List of `{name, code: {file, pattern}, docs: [paths]}`. `pattern` is a regex; group 1 is the captured token list. Longer than 256 characters or an invalid regex is a `config` finding |
| `retired_names` | List of `{name, replace_with}`. `replace_with` may be empty |
| `release.changelog` | Keep-a-Changelog path, default `CHANGELOG.md` |

Example that checks the program allowlist against a doc:

```yaml
boundary: main
sources_of_truth:
- name: programs
code:
file: scripts/repoman-setup.sh
pattern: 'KNOWN_PROGRAMS="([^"]+)"'
docs:
- README.md
retired_names:
- name: kagenti
replace_with: ""
release:
changelog: CHANGELOG.md
```

An enumeration **correction** (a doc token that is not in the code capture)
is `mechanical: false`. An enumeration **addition** (a code token missing from
the doc) carries `draft` and the fixer appends it to
`.repoman/doc-drift-additions.md` on the rolling branch.

## Fixer

`versions` and `retired_names` replace only the flagged line, with word
boundaries, so `1.2.3` does not match inside `11.2.34`. A failed scan does not
close the rollup issue: the scanner keeps the previous findings. The rolling
additions branch pushes with `--force-with-lease`.
86 changes: 79 additions & 7 deletions scripts/automation-health-dashboard.sh
Original file line number Diff line number Diff line change
Expand Up @@ -114,16 +114,20 @@ display_name_for() {
case "$1" in
link-health) echo "Link Health" ;;
dep-bump) echo "Dependency Bumps" ;;
doc-drift) echo "Doc Drift" ;;
*) echo "" ;;
esac
}

LINK_SCAN_DIR=""
DEP_BUMP_DIR=""
DOC_DRIFT_DIR=""
HAS_LINK_HEALTH=false
HAS_DEP_BUMP=false
HAS_DOC_DRIFT=false
LINK_HEALTH_HEADING="Link Health"
DEP_BUMP_HEADING="Dependency Bumps"
DOC_DRIFT_HEADING="Doc Drift"

if [ -f "$INDEX_FILE" ]; then
if ! jq empty "$INDEX_FILE" >/dev/null 2>&1; then
Expand Down Expand Up @@ -159,12 +163,18 @@ if [ -f "$INDEX_FILE" ]; then
HAS_DEP_BUMP=true
DEP_BUMP_HEADING="$entry_display"
;;
doc-drift)
DOC_DRIFT_DIR="$entry_path"
HAS_DOC_DRIFT=true
DOC_DRIFT_HEADING="$entry_display"
;;
esac
done < <(jq -r 'keys[]' "$INDEX_FILE")
else
# Disk-derived fallback: no index, probe the known report subdirs directly.
LINK_SCAN_DIR="$REPORTS_DIR/link-health"
DEP_BUMP_DIR="$REPORTS_DIR/dep-bump"
DOC_DRIFT_DIR="$REPORTS_DIR/doc-drift"

if [ -f "$LINK_SCAN_DIR/latest.json" ] && [ -f "$LINK_SCAN_DIR/history.json" ]; then
HAS_LINK_HEALTH=true
Expand All @@ -174,17 +184,22 @@ else
HAS_DEP_BUMP=true
DEP_BUMP_HEADING=$(display_name_for "dep-bump")
fi
if [ -f "$DOC_DRIFT_DIR/latest.json" ] && [ -f "$DOC_DRIFT_DIR/history.json" ]; then
HAS_DOC_DRIFT=true
DOC_DRIFT_HEADING=$(display_name_for "doc-drift")
fi
fi

if [ "$HAS_LINK_HEALTH" = false ] && [ "$HAS_DEP_BUMP" = false ]; then
if [ "$HAS_LINK_HEALTH" = false ] && [ "$HAS_DEP_BUMP" = false ] && [ "$HAS_DOC_DRIFT" = false ]; then
echo "ERROR: No program reports found in $REPORTS_DIR"
echo "Expected: $REPORTS_DIR/link-health/latest.json and/or $REPORTS_DIR/dep-bump/latest.json"
echo "Expected: $REPORTS_DIR/link-health/, dep-bump/, and/or doc-drift/ latest.json"
exit 1
fi

PROGRAMS_ACTIVE=0
if [ "$HAS_LINK_HEALTH" = true ]; then PROGRAMS_ACTIVE=$((PROGRAMS_ACTIVE + 1)); fi
if [ "$HAS_DEP_BUMP" = true ]; then PROGRAMS_ACTIVE=$((PROGRAMS_ACTIVE + 1)); fi
if [ "$HAS_DOC_DRIFT" = true ]; then PROGRAMS_ACTIVE=$((PROGRAMS_ACTIVE + 1)); fi

# --- Setup ---
TMPDIR=$(mktemp -d)
Expand All @@ -211,6 +226,8 @@ LH_ISSUES_CLOSED=0
DB_ISSUES_CREATED=0
DB_ISSUES_CLOSED=0
DB_FIXER_CLOSED=0
DD_ISSUES_CREATED=0
DD_ISSUES_CLOSED=0
TOTAL_PRS_OPENED=0
LAST_SCAN_DATE="unknown"

Expand Down Expand Up @@ -241,14 +258,26 @@ if [ "$HAS_DEP_BUMP" = true ]; then
fi
fi

TOTAL_ISSUES_CREATED=$((LH_ISSUES_CREATED + DB_ISSUES_CREATED))
TOTAL_ISSUES_RESOLVED=$((LH_ISSUES_CLOSED + DB_ISSUES_CLOSED + DB_FIXER_CLOSED))
if [ "$HAS_DOC_DRIFT" = true ]; then
DD_ISSUES_CREATED=$(jq '[.[] | .issues_created // 0] | add // 0' "$DOC_DRIFT_DIR/history.json")
DD_ISSUES_CLOSED=$(jq '[.[] | .issues_closed // 0] | add // 0' "$DOC_DRIFT_DIR/history.json")
dd_last_date=$(jq -r '.date // ""' "$DOC_DRIFT_DIR/latest.json")
if [ -n "$dd_last_date" ]; then
dd_date="${dd_last_date%%T*}"
if [ "$LAST_SCAN_DATE" = "unknown" ] || [[ "$dd_date" > "$LAST_SCAN_DATE" ]]; then
LAST_SCAN_DATE="$dd_date"
fi
fi
fi

TOTAL_ISSUES_CREATED=$((LH_ISSUES_CREATED + DB_ISSUES_CREATED + DD_ISSUES_CREATED))
TOTAL_ISSUES_RESOLVED=$((LH_ISSUES_CLOSED + DB_ISSUES_CLOSED + DB_FIXER_CLOSED + DD_ISSUES_CLOSED))
HOURS_SAVED=$(echo "$TOTAL_ISSUES_RESOLVED" | awk '{printf "%.1f", $1 * 0.25}')

if [ "$VERBOSE" = true ]; then
echo "Executive metrics:"
echo " Issues created: $TOTAL_ISSUES_CREATED (LH: $LH_ISSUES_CREATED, DB: $DB_ISSUES_CREATED)"
echo " Issues resolved: $TOTAL_ISSUES_RESOLVED (LH: $LH_ISSUES_CLOSED, DB: $DB_ISSUES_CLOSED, fixer: $DB_FIXER_CLOSED)"
echo " Issues created: $TOTAL_ISSUES_CREATED (LH: $LH_ISSUES_CREATED, DB: $DB_ISSUES_CREATED, DD: $DD_ISSUES_CREATED)"
echo " Issues resolved: $TOTAL_ISSUES_RESOLVED (LH: $LH_ISSUES_CLOSED, DB: $DB_ISSUES_CLOSED, DD: $DD_ISSUES_CLOSED, fixer: $DB_FIXER_CLOSED)"
echo " PRs opened: $TOTAL_PRS_OPENED"
echo " Hours saved: $HOURS_SAVED"
echo ""
Expand Down Expand Up @@ -362,6 +391,28 @@ if [ "$HAS_DEP_BUMP" = true ]; then
' "$DEP_BUMP_DIR/history.json" 2>/dev/null || echo "| - | - | - | - |")
fi

# =============================================================================
# Step 4b: Compute doc-drift section
# =============================================================================

DD_REPOS_SCANNED=0
DD_CORRECTIONS=0
DD_ADDITIONS=0
DD_TREND_TABLE="| - | - | - | - |"

if [ "$HAS_DOC_DRIFT" = true ]; then
DD_REPOS_SCANNED=$(jq '.repos_scanned // 0' "$DOC_DRIFT_DIR/latest.json")
DD_CORRECTIONS=$(jq '.corrections // ([.findings[]? | select(.kind=="correction")] | length)' "$DOC_DRIFT_DIR/latest.json")
DD_ADDITIONS=$(jq '.additions // ([.findings[]? | select(.kind=="addition")] | length)' "$DOC_DRIFT_DIR/latest.json")
# Prefer .new/.fixed (emitted by doc-drift-scanner.sh); fall back to 0 so
# older history rows without those keys still render a numeric Delta.
DD_TREND_TABLE=$(jq -r '
.[-10:] | reverse | .[] |
(.new // 0) as $n | (.fixed // 0) as $f |
"| \(.date | split("T")[0]) | \(.corrections // 0) | \(.additions // 0) | \(if $n > $f then "+\($n - $f)" elif $n < $f then "\($n - $f)" else "0" end) |"
' "$DOC_DRIFT_DIR/history.json" 2>/dev/null || echo "| - | - | - | - |")
fi

# =============================================================================
# Step 5: Compute cross-program coverage
# =============================================================================
Expand Down Expand Up @@ -454,7 +505,7 @@ cat > "$TMPDIR/automation-health.md" << DASHBOARD_EOF

| Metric | Value |
|--------|-------|
| Total issues auto-created | $TOTAL_ISSUES_CREATED (link-health: $LH_ISSUES_CREATED, dep-bump: $DB_ISSUES_CREATED) |
| Total issues auto-created | $TOTAL_ISSUES_CREATED (link-health: $LH_ISSUES_CREATED, dep-bump: $DB_ISSUES_CREATED, doc-drift: $DD_ISSUES_CREATED) |
| Total issues auto-resolved | $TOTAL_ISSUES_RESOLVED |
| Total PRs auto-opened | $TOTAL_PRS_OPENED |
| Estimated hours saved | ${HOURS_SAVED} hrs (at 15 min/resolved issue) |
Expand Down Expand Up @@ -514,6 +565,27 @@ $DB_TREND_TABLE
DASHBOARD_EOF
fi

if [ "$HAS_DOC_DRIFT" = true ]; then
cat >> "$TMPDIR/automation-health.md" << DASHBOARD_EOF

## $DOC_DRIFT_HEADING

| Metric | Value |
|--------|-------|
| Repos scanned | $DD_REPOS_SCANNED |
| Open corrections | $DD_CORRECTIONS |
| Open additions | $DD_ADDITIONS |
| Issues created (cumulative) | $DD_ISSUES_CREATED |
| Issues closed (cumulative) | $DD_ISSUES_CLOSED |

### Trend (last 10 scans)

| Date | Corrections | Additions | Delta |
|------|-------------|-----------|-------|
$DD_TREND_TABLE
DASHBOARD_EOF
fi

# Tail: cross-cutting sections (always rendered).
cat >> "$TMPDIR/automation-health.md" << DASHBOARD_EOF

Expand Down
Loading
Loading