Repository navigation
Reconcile validate-counts vs drift checker stale-counts pattern across ecosystem #12
Copy link
Copy link
Closed
Labels
Description
Background
Discovered during Phase 2 Session D-0 audit of all 9 tool repos.
Two distinct patterns for handling aggregate counts (X skills, Y rules, Z tools) currently coexist in the ecosystem:
Pattern 1: Enforce in README.md via CI
- CFX-Developer-Tools and Unity-Developer-Tools have a
validate-countsjob in theirvalidate.yml. - It reads
.cursor-plugin/plugin.json, computes live counts, andgrepsREADME.mdfor the literal strings"{N} skills","{N} rules","{N} snippets","{N} templates". - CI fails if the strings do not match. README is forced to stay in sync.
Pattern 2: Flag as stale in agent files via drift checker
- Phase 2 Session B's
stale-countscheck flags any\d+\s+(skills?|rules?|MCP\s+tools?|tools?|commands?|hooks?)pattern in agent files (AGENTS.md, CLAUDE.md, SKILL.md, .mdc). - Per Q3 resolution: warn-only, suggest removal, do not sync.
- Smoke test surfaced 62 such warnings across 8 repos (mostly AGENTS.md / CLAUDE.md narratives like
177 skills, 71 rules).
The tension
The two patterns disagree on what aggregate counts are:
- Pattern 1 treats them as canonical surface that must stay accurate.
- Pattern 2 treats them as stale narrative that should be removed and replaced with a registry pointer.
Today they don't actually conflict at the file level (Pattern 1 only inspects README.md; Pattern 2 only inspects agent files). But the ecosystem has not picked a position. Tool-repo authors are getting two contradictory signals as we add the drift checker.
What needs deciding
- Do aggregate counts belong in agent files at all?
- If yes, should the drift checker stop flagging them? (Tighten
stale-countsto specific contexts; add a config knob; or remove the check entirely.) - If no, who is responsible for removing them? Mass-edit PR? Per-repo grooming?
- What about README.md? Keep the CFX/Unity validate-counts pattern, or migrate to "registry is source of truth, README links to it"?
Likely path
Probable resolution direction (to validate):
- Agent files: aggregate narratives are stale.
stale-countsstays as warn. Tool-repo authors should remove or rephrase assee registry.json for live counts. - README.md: keep the validate-counts pattern for now; reconsider in a future ecosystem-wide grooming session.
Acceptance criteria
- Documented decision in
standards/(likely a new section inversioning.mdor a new file likestandards/aggregate-counts.md). - If keeping
stale-countsas warn: a known-false-positive list or skip-pattern for legitimate technical counts (e.g., firewall rule counts in Home-Lab; see related issue). - If removing the check: bump checker minor and document rationale.
References
- Phase 2 Session B PR: feat: drift checker additional checks and policy data (Phase 2 Session B) #8
- Decision Q3 in the Phase 2 design document
Reactions are currently unavailable
Activity
TMHSDigital commentedon Apr 24, 2026
Cross-link: see #13 for the SKILL.md sub-question (technical-noun false positives).
fix: suppress stale-counts for cursor-plugin type to unblock D-1 (#16)
TMHSDigital commentedon Apr 25, 2026
Phase 2 Session D D-1 canary surfaced this tension concretely. Suppressing
stale-countsat thecursor-plugintype level viastandards/drift-checker.config.jsonto unblock the rollout (#16, v1.7.5).This is not a final answer to the policy question. The interim decision: agent files (
AGENTS.md,CLAUDE.md) carry narrative aggregates as descriptive prose; aggregate truth-bearing checks belong in CFX/Unity-stylevalidate-countsjobs againstREADME.md(the user-facing aggregate).Open question for future work: can
stale-countsbe made smarter to distinguish narrative ("theskills/directory contains 9 SKILL.md files") from truth claims ("CFX has 9 skills")? If so, re-enable it.Leaving this issue open. Updated by Phase 2 Session D D-1.
TMHSDigital commentedon Apr 25, 2026
DTD#13 investigation completed and produced a finding relevant to this policy discussion.
The hypothesis going in was that SKILL.md findings would mirror the AGENTS.md/CLAUDE.md narrative-aggregate case (suppressing them via type-level config like the existing cursor-plugin suppression).
The data refuted this hypothesis. Investigation of 5 SKILL.md findings showed:
One of the category A findings was a real off-by-one drift in
steam-api-reference/SKILL.md(claimed 7 tools, table had 8). Drift-check did its job. Extending type-level suppression to SKILL.md would have masked this bug.Starting position for the eventual DTD#12 policy session:
(N tools)headers and compare to the claim, rather than the current generic stale-counts pattern.DTD#13 closure: investigation complete, follow-ups filed (Steam content-fix PR TMHSDigital/Steam-Cursor-Plugin#13 merged, drift-check enhancement #37). Closing DTD#13 separately.
chore: bump standards-version signals and drift-check pin to 1.9.0
chore: bump standards-version signals and drift-check pin to 1.9.0 (#18)
chore: bump standards-version signals and drift-check pin to 1.9.0 (#6)
chore: bump standards-version signals and drift-check pin to 1.9.0 (#16)
chore: bump standards-version signals and drift-check pin to 1.9.0 (#9)
chore: bump standards-version signals and drift-check pin to 1.9.0 (#14)
chore: bump standards-version signals and drift-check pin to 1.9.0 (#13)
chore: bump standards-version signals and drift-check pin to 1.9.0 (#11)
chore: bump standards-version signals and drift-check pin to 1.9.0 (#32)
chore: bump standards-version signals and drift-check pin to 1.9.0 (#9)