pr-coverage: bug status, finding resolution, feature-scoped bugs, TC id collisions — reader and writer contracts - #89
Closed
TimothyHan wants to merge 3 commits into
Closed
TimothyHan wants to merge 3 commits into
TimothyHan wants to merge 3 commits into
Conversation
…, TC id collisions (0.2.0) Problem: four defects seen on the demo repo — (1) the author's to-do list ignored bug status (`**status:**` read case-sensitively, then never consulted); (2) findings were never checked for resolution (only the latest session read, every finding open for ever, closed issues re-created); (3) "also seen as" linked a finding to any feature's bug of the same number; (4) test case ids restarted at TC-001 per ticket, so one ticket's spec proved the other's rows — the mixed request-logging heatmap. Root cause: reader/writer contract gaps — no defined bug status, no finding identity across sessions, bug ids per feature not honoured, TC ids resolved by bare id repo-wide. Fix: `**Status:**` read case-insensitively with a closed resolved vocabulary; findings keyed by feature + title across every session, a later status wins, a bug-linked finding follows the bug; bugs matched within their feature; TC ids owned per test-case file, a collision reported and counted as no evidence, an id another feature also defines needs a spec that cites the feature; `issues --state all` closes resolved findings' issues and reopens closed ones that come back; the Korean finding block parses. Tests for each, badge 1887. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…line reads /qa-qa 0.3.8 writes `**Status:** open` on every bug file; /qa-verify-fix 0.4.3 (ko catches up from 0.4.1) sets `verified` / `reopened`; /qa-exploratory 0.4.7 re-checks earlier sessions' findings and re-lists them with a `**Status:**`; /qa-test-cases 0.6.3 numbers TC ids per feature, never per ticket (self-check 9). KB spec §6.5 (id uniqueness), §6.9 (finding identity), new §6.10 (bugs/) in en and ko; the kb and explore prompts carry the same rules; the pipeline README and the CI guides say what leaves the to-do list and why. CHANGELOG in both locales. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tion scope from the built index Another skill's SKILL.md is never loaded by this runner; a changed reference counts only when the section (by its qab id) is scoped to the skill or to all. The reason line now names the section, so a RUN verdict says what the A/B pays for. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four defects from the demo repo
Reported 2026-09-12; worked via
/qa-improveFix mode. The reader (bin/pr-coverage.js0.2.0, one commit) and the writers (the skills, prompts, KB spec — one commit) change together, because each defect was a contract the two sides never agreed on.**status:** open, the reader wanted**Status:**and then never used it**Status:**read case-insensitively; closed resolved vocabulary (fixed,verified,wont-fix,duplicate, …); absent = open; resolved bugs listed under "Resolved", never as a to-do/qa-qa0.3.8 writes**Status:** open;/qa-verify-fix0.4.3 setsverified/reopened; KB spec §6.10 defines the fileissueslisted open issues only, so a closed one was re-created**Status:**wins, a bug-linked finding follows the bug;issues --state all: a resolved finding closes its issue, a returning one reopens it/qa-exploratory0.4.7 re-checks earlier sessions and re-lists each finding with**Status:** still open | resolved | not reproduced; explore prompt says so; KB spec §6.9/qa-test-cases0.6.3 continues the feature's numbering (self-check 9); kb prompt; KB spec §6.5Also: the Korean finding block (
### 발견 N:,**상태:**) now parses — the ko build wrote findings the reader never saw.eval.js scopereads reference section scope from the built index and ignores other skills' files, so its RUN reason names the section (here: KB specfile-schemas, scoped to all).Verification
node test.js: 1887 passed, 0 failed — new checks for each defect (lowercase status → Resolved; later-session status resolves a finding and closes its issue; a returning finding reopens; two features withBUG-001link separately;TC-01defined by two tickets is a collision and proves nothing whileTC-02still counts; a shared id needs a citing spec).scopesays RUN for both calibrated skills because KB spec §6 (file-schemas, scoped to all) changed.eval.js abfor test-cases and exploratory is running; the per-criterion tables will be added here when it finishes.Left open
/qa-verify-fixko was at 0.4.1 while en was 0.4.2 (fix(verify-fix): invert the KB test cases that encoded the verified bug #86 skipped the twin); bumped to 0.4.3 here without back-porting fix(verify-fix): invert the KB test cases that encoded the verified bug #86's text — worth a look.🤖 Generated with Claude Code