Skip to content

stale-counts findings in SKILL.md files (Home-Lab, Steam) need investigation #13

Closed
Closed
stale-counts findings in SKILL.md files (Home-Lab, Steam) need investigation#13
Labels
enhancementNew feature or requeststandards

Description

@TMHSDigital
TMHSDigital
opened on Apr 24, 2026

Background

Phase 2 Session D-0 audit ran the drift checker against all 9 tool repos. stale-counts produced 62 warnings total.

61 of 62 warnings live in repo-root agent files (AGENTS.md, CLAUDE.md) where aggregate narratives like 177 skills, 71 rules are clearly stale ecosystem-summary boilerplate.

5 warnings are different: they are inside skills/<name>/SKILL.md files, where the aggregate count may be legitimate technical content rather than stale narrative.

Specific findings

Repo File Count
Home-Lab-Developer-Tools skills/secrets-management/SKILL.md 1
Steam-Cursor-Plugin skills/steam-api-reference/SKILL.md 4

Hypothesis

These may be matches against patterns like:

  • "3 firewall rules" (Home-Lab secrets-management referencing security configurations)
  • "5 API tools" or "12 commands" (Steam API reference describing Steam Web API surface)

If so, these are technical specifications, not stale ecosystem narratives. The drift checker's regex (\b\d+\s+(skills?|rules?|MCP\s+tools?|tools?|commands?|hooks?)\b) cannot distinguish between "3 firewall rules" (legitimate domain content) and "71 rules" (stale ecosystem summary).

What needs investigating

  1. Read each of the 5 flagged contexts and classify as legitimate-domain vs stale-narrative.
  2. If all are legitimate-domain: design a way for stale-counts to skip them. Options:
    • Add drift-ignore pragma in the SKILL.md frontmatter (workable today, but burdens tool authors with manual annotation).
    • Tighten the regex to exclude common technical-noun pairings (fragile).
    • Add a config knob like stale_counts_skip_patterns (most flexible).
  3. If any are actually stale: leave the warning, fix the content.

Related

Activity

  1. TMHSDigital
    added
    enhancementNew feature or request
    standards
    on Apr 24, 2026

  2. TMHSDigital commented on Apr 24, 2026

    @TMHSDigital
    TMHSDigital
    on Apr 24, 2026
    OwnerAuthor

    Cross-link: see #12 for the broader validate-counts vs stale-counts reconciliation.

  3. TMHSDigital
    added a commit that references this issue on Apr 25, 2026

    fix: correct stale tool count in steam-api-reference SKILL.md (#13)

    c9cea61

  4. TMHSDigital commented on Apr 25, 2026

    @TMHSDigital
    TMHSDigital
    on Apr 25, 2026
    OwnerAuthor

    Investigation complete. 5 findings categorized:

    • 4 category A (genuine count claims): 3 currently accurate, 1 stale
    • 1 category C (false positive from example dialogue)
    • 0 category B (narrative aggregates)

    Resolution:

    Full investigation written up in dtd13-investigation.md (workspace root).

    The drift-check stale-counts pass did its job: caught a real off-by-one bug that had drifted unnoticed. Validates the infrastructure investment from Phase 2.

  5. TMHSDigital
    closed this as completedon Apr 25, 2026
  6. TMHSDigital
    added a commit that references this issue on Apr 25, 2026

    fix: stale-counts policy correction and example-dialogue skip (#38)

    a163f1b

  7. TMHSDigital commented on Apr 25, 2026

    @TMHSDigital
    TMHSDigital
    on Apr 25, 2026
    OwnerAuthor

    Correction to my closing comment above.

    The framing "drift-check did its job. Validates the infrastructure investment from Phase 2." was incorrect.

    DTD#12 investigation (resolved in #38, shipped as v1.9.0) revealed that the cursor-plugin and mcp-server type-level skip_checks entries in standards/drift-checker.config.json were suppressing stale-counts entirely (whole-check skip, not file-scoped). The Steam off-by-one bug was found by manually running stale_counts.py during DTD#13 investigation — not by the production drift-check workflow, which had stale-counts suppressed for Steam-Cursor-Plugin.

    The bug fix in TMHSDigital/Steam-Cursor-Plugin#13 is still correct (the SKILL.md content matches reality now). But the verification narrative was wrong: production drift-check did not catch this; manual investigation did.

    DTD#12's resolution removes the over-broad suppression and encodes the AGENTS.md/CLAUDE.md narrative-aggregate policy in the check itself (plus DTD#37's example-dialogue skip). Empirical post-merge verification on Steam now surfaces 4 stale-counts findings — all accurate (because Steam#13 fixed the off-by-one), but the check is actually running against SKILL.md files for the first time since v1.7.5.

    Net: production drift-check now actually catches this class of bug across all tool repos. Closing my earlier overstatement of the verification chain.

  8. TMHSDigital
    added 2 commits that reference this issue on Apr 26, 2026

    fix: add drift-ignore pragma for legitimate aggregate counts in steam…

    72640b9

    chore: bump standards-version signals and drift-check pin to 1.9.0 (#14)

    bdd175e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requeststandards

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions