Skip to content

spec: validate numbering, edges, and links before writing artifacts - #217

Merged
joestump merged 2 commits into
mainfrom
feat/spec-authoring-validation
Aug 19, 2026
Merged

spec: validate numbering, edges, and links before writing artifacts#217
joestump merged 2 commits into
mainfrom
feat/spec-authoring-validation

Conversation

@joestump-agent

Copy link
Copy Markdown
Collaborator

First of five PRs hardening the authoring skills, based on a downstream audit: I reviewed every ADR and OpenSpec in a plugin-generated project (msgbrowse) and traced which defects came from the skills vs. the project. Three skill-side gaps showed up as real bugs downstream.

What changed (this PR)

Adds Step 4c — Consistency validation before writing to /sdd:spec, run after drafting and before the files hit disk:

  1. SPEC number match — the H1's SPEC-XXXX must equal the sequential number chosen in Step 3. Downstream evidence: msgbrowse's contact-merge spec was renumbered mid-draft to SPEC-0018 but its ten requirement headings kept the stale REQ-0015- prefix, and seven Go source comments inherited the wrong ID before anyone noticed.
  2. Frontmatter edge targets exist — every implements/requires/extends/supersedes ID must resolve to an artifact on disk; otherwise /sdd:graph validate fails the first time it runs.
  3. Relative links resolve — extract ](...) relative targets from both spec.md and design.md and stat each. Hand-finished specs accumulate silently-broken relative links.

Also extends the self-review/architect checklist with the same three checks plus: design.md must retain its ## Open Questions section (an explicit "None currently" is fine) — today nothing stops the section from being dropped, which loses the designated home for unresolved questions.

Verification

  • scripts/check-structure.sh passes
  • Skill-only change; no code paths affected

Companion PRs: ADR template hardening, Related Artifacts body section, security-baseline references, and a /sdd:status backfill mode.

🤖 This was posted autonomously by glm-5.3 using Crush.

@github-actions

Copy link
Copy Markdown

Skill Eval Results

Mode: quick | No eval results found.

ℹ️ High-cost scenarios exist (full-chain-with-review) but did not run in this build. They run only on release PRs or manual pipeline dispatch.

joestump added a commit that referenced this pull request Aug 19, 2026
…path

Two defects in the Related Artifacts section:

The heading collided with the one /sdd:docs already appends. Both
transform-openspecs.js and the vendored Docusaurus plugin emit a
generated '## Related Artifacts' mini-DAG at the foot of every spec and
design page, so an authored section under the same name gave every
rendered page two identical H2s -- duplicate anchors and a doubled TOC
entry. Renamed to '## Graph Edges', matching the repo's own edge
vocabulary.

The Implements example pointed at ../../../adr/, singular. The ADR
directory is docs/adrs/, so every spec seeded from this template shipped
a dead link -- and PR #217 adds a validation rule requiring exactly the
links this template breaks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Skill Eval Results

Mode: quick | No eval results found.

ℹ️ High-cost scenarios exist (full-chain-with-review) but did not run in this build. They run only on release PRs or manual pipeline dispatch.

@joestump
joestump force-pushed the feat/spec-authoring-validation branch from 453d6d8 to e56db3f Compare August 19, 2026 10:03
joestump added a commit that referenced this pull request Aug 19, 2026
…path

Two defects in the Related Artifacts section:

The heading collided with the one /sdd:docs already appends. Both
transform-openspecs.js and the vendored Docusaurus plugin emit a
generated '## Related Artifacts' mini-DAG at the foot of every spec and
design page, so an authored section under the same name gave every
rendered page two identical H2s -- duplicate anchors and a doubled TOC
entry. Renamed to '## Graph Edges', matching the repo's own edge
vocabulary.

The Implements example pointed at ../../../adr/, singular. The ADR
directory is docs/adrs/, so every spec seeded from this template shipped
a dead link -- and PR #217 adds a validation rule requiring exactly the
links this template breaks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Skill Eval Results

Mode: quick | No eval results found.

ℹ️ High-cost scenarios exist (full-chain-with-review) but did not run in this build. They run only on release PRs or manual pipeline dispatch.

joestump-agent and others added 2 commits August 19, 2026 11:26
Add a consistency-validation step between drafting and writing: the SPEC
number in the H1 must match the sequential number chosen in Step 3, every
frontmatter edge target must resolve to an existing artifact, and every
relative markdown link in both files must point at a real file. Found by
auditing downstream projects — a renumbered spec kept its stale
requirement-ID prefix and cross-links rotted silently. Also extends the
self-review checklist with these checks and requires design.md to keep
its Open Questions section.

💘 Generated with Crush

Assisted-by: Crush:glm-5.3
…tion

Step 4c's SPEC-number check asserted the H1 equals the *next* sequential
number from Step 3. Step 3 runs unconditionally, including on the Step 2
update path, so validating an update to an existing spec would have
demanded the file be renumbered — orphaning every tracker issue, graph
edge, and cross-spec reference that already cites the old number.

Scope the check to the number the run is actually authoring under.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@joestump
joestump force-pushed the feat/spec-authoring-validation branch from e56db3f to 530f94d Compare August 19, 2026 10:26
joestump added a commit that referenced this pull request Aug 19, 2026
…path

Two defects in the Related Artifacts section:

The heading collided with the one /sdd:docs already appends. Both
transform-openspecs.js and the vendored Docusaurus plugin emit a
generated '## Related Artifacts' mini-DAG at the foot of every spec and
design page, so an authored section under the same name gave every
rendered page two identical H2s -- duplicate anchors and a doubled TOC
entry. Renamed to '## Graph Edges', matching the repo's own edge
vocabulary.

The Implements example pointed at ../../../adr/, singular. The ADR
directory is docs/adrs/, so every spec seeded from this template shipped
a dead link -- and PR #217 adds a validation rule requiring exactly the
links this template breaks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@joestump

Copy link
Copy Markdown
Owner

Code review — /code-review --fix

Reviewed the diff and pushed one fix to this branch.

Finding: the SPEC-number check renumbers an existing spec on update

skills/spec/SKILL.md:150

SPEC number match: the SPEC number in the spec.md H1 MUST equal the next sequential number determined in Step 3.

Step 3 (Determine the next SPEC number) runs unconditionally — including on the Step 2 update path, where the user chose "update the existing spec". On that path the H1 legitimately holds the spec's existing number, which by construction is not the next sequential one. Validation as written would report a mismatch and demand the file be renumbered.

Concrete failure: /sdd:spec web-dashboard on an existing SPEC-0007 in a repo whose highest spec is SPEC-0019. Step 3 yields 0020; step 4c sees SPEC-0007 in the H1 and "reconciles" it to SPEC-0020. Every tracker issue filed by /sdd:plan SPEC-0007, every implements: [SPEC-0007] edge in another artifact's frontmatter, and every SPEC-0007 cross-reference on the docs site now points at nothing — and /sdd:graph validate, the very check item 2 invokes, starts failing on artifacts this run never touched.

Fixed by scoping the check to the number the run is actually authoring under: Step 3's next number on a new spec, the file's existing number on an update, with an explicit "an update MUST NOT renumber a spec" and the reason. Same correction applied to the matching Rules bullet.

Verified, no change needed

  • Bash is already in this skill's allowed-tools, so the "check this mechanically (…stat each)" instruction in item 3 is executable as written.
  • Step numbering: 4c slots cleanly between the existing 4b and 5.
  • Item 3's link check does not trip over the template's own ADR-XXXX placeholders — those are substituted before the file is written. (It would have tripped over the ../../../adr/ typo in spec: render frontmatter edges as a Related Artifacts body section #219; that PR's fix and this validation now agree.)
  • The ## Open Questions retention rule matches the design.md template, which has that section.

🤖 Posted on behalf of @joestump by claude-opus-5 using Claude Code.

@github-actions

Copy link
Copy Markdown

Skill Eval Results

Mode: quick | No eval results found.

ℹ️ High-cost scenarios exist (full-chain-with-review) but did not run in this build. They run only on release PRs or manual pipeline dispatch.

@joestump
joestump merged commit 86aa90d into main Aug 19, 2026
10 checks passed
joestump added a commit that referenced this pull request Aug 19, 2026
…path

Two defects in the Related Artifacts section:

The heading collided with the one /sdd:docs already appends. Both
transform-openspecs.js and the vendored Docusaurus plugin emit a
generated '## Related Artifacts' mini-DAG at the foot of every spec and
design page, so an authored section under the same name gave every
rendered page two identical H2s -- duplicate anchors and a doubled TOC
entry. Renamed to '## Graph Edges', matching the repo's own edge
vocabulary.

The Implements example pointed at ../../../adr/, singular. The ADR
directory is docs/adrs/, so every spec seeded from this template shipped
a dead link -- and PR #217 adds a validation rule requiring exactly the
links this template breaks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joestump added a commit that referenced this pull request Aug 19, 2026
)

* spec: render frontmatter edges as a Related Artifacts body section

Frontmatter edges are machine-readable only -- a reader of the raw
markdown or the rendered docs page never sees which ADRs a spec
implements or which capabilities it requires, so downstream projects
hand-write ad-hoc "Related ADRs" header lines in inconsistent shapes
(found in every audited downstream repo). Give the template a
canonical home for that information: a Related Artifacts section that
mirrors the frontmatter edges, one bullet per target, regenerated
whenever edges change.

💘 Generated with Crush

Assisted-by: Crush:glm-5.3

* fix(spec): avoid the generated mini-DAG heading and fix the ADR link path

Two defects in the Related Artifacts section:

The heading collided with the one /sdd:docs already appends. Both
transform-openspecs.js and the vendored Docusaurus plugin emit a
generated '## Related Artifacts' mini-DAG at the foot of every spec and
design page, so an authored section under the same name gave every
rendered page two identical H2s -- duplicate anchors and a doubled TOC
entry. Renamed to '## Graph Edges', matching the repo's own edge
vocabulary.

The Implements example pointed at ../../../adr/, singular. The ADR
directory is docs/adrs/, so every spec seeded from this template shipped
a dead link -- and PR #217 adds a validation rule requiring exactly the
links this template breaks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: joestump <joe@joestump.net>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants