mbeacom · mbeacom · Aug 14, 2026 · Aug 14, 2026
@@ -34,6 +34,18 @@ sync.
 | 6 — ARB queue | `specs/007-arb-queue/` | **landed / reference-verified** (kernel + `adr queue` CLI + queue Action; PR #22, `efef89b`); rung-2 maintainer isolated reference-repository validation met ([`adrkit-t018-dogfood`](https://github.com/mbeacom/adrkit-t018-dogfood)); **not** externally validated (rung 3 open, per [ADR-0014](docs/adr/0014-stage-phase-landing-evidence-across-a-three-rung-validation-ladder.md)) |
 | 7 — Spec Kit hook viability | `specs/008-spec-kit-hook-viability/` | **Remediated post-merge (PR #35 merge commit `35542a5`; this session): conformant, both blocking gates closed; verdict `no-go` (unchanged).** T001–T058: all 58 checked. **Historical record (through PR #35's merge):** T005, T012, and T057 were deliberately left unchecked (T005's rank-1 network-denial mechanism was available but not selected; T012 cannot certify a dependency set including an unchecked T005 — per PR review round 12, this is an explicit blocking-checkpoint violation, so the run was disclosed as out-of-contract, not fully gate-conformant, even though every applicable, non-short-circuited User Story task's own distinct action did run and remains independently evidenced; T044, T045, and T047 are separately recorded as correctly recognized and honored short-circuits per the `no-go` outcome's own contract-required rules, not as substantively evaluated — see `tasks.md`). Verdict trigger: `mutation`, driven independently by the `install`/`remove` lifecycle steps — unaffected by either gate issue. T024 (`hook-fire`) stayed checked but disclosed the same gap's FR-011-specific consequence (PR review round 13). An eighth, independently dispatched audit pass (PR review round 16) confirmed this gap met neither the isolation contract's strongest-mechanism rule nor FR-011's literal requirement; because that defect was disclosed rather than remediated, T057 itself was unchecked (PR review round 17). Per PR review round 18, T057's own text ("remediate any defect found before T058") plus the Dependency Graph's `T057 → T058` step named a second, distinct blocking-gate violation: T058 ran without that remediation. PR #35 nonetheless merged in that state. **Remediation (this session):** a coordinating session mandated a conformant rerun (not a waiver, and explicitly not proceeding to feature 009 under the merged state). A fresh probe of the actual execution host found rank 1 genuinely available via rootless Podman 5.4.2/5.8.5's `--network none` (kernel-enforced network namespace, confirmed via a direct outbound-connection-attempt probe); `/usr/bin/sandbox-exec (deny network*)` was also confirmed functional but not selected (weaker, egress-only guarantee); Docker was probed and rejected (client only, no daemon). Podman was wired into `scripts/probe.sh` step 5, scoped to the fixture/adrkit subprocess path only — the live-Copilot session's own parent model/API connectivity was never denied. A fresh, isolated live-Copilot session (clean mutation baseline, no concurrent feature 009 work) re-ran `hook-fire` under this genuine mechanism: exit 0, valid `adr queue` JSON, zero repo mutation. This closed T005; T012's dependency-set certification followed. A ninth independent audit pass (`claude-opus-4.8`, fresh context, never Opus 4.6) reviewed the complete two-run evidence against every relevant FR/SC/contract and **PASSED** (four cosmetic notes only), closing T057; T058 was re-performed. **The `no-go` verdict is unchanged** — it is driven by an axis (file-mutation comparison) wholly orthogonal to network-denial mechanism selection. Original first-run evidence is preserved unmodified as the historical record; new evidence and hashes are tracked in the evidence index below. **The 008→009 sequencing precondition is now genuinely satisfied.** This spike remains a disposable compatibility probe — not external validation/adoption, not a shipped adapter or integration — and does not change Phase 6's own landed/reference-verified status in either direction. Full task-by-task, audit-round, and remediation accounting: [`specs/008-spec-kit-hook-viability/tasks.md`](specs/008-spec-kit-hook-viability/tasks.md), [`specs/008-spec-kit-hook-viability/checklists/evidence-index.md`](specs/008-spec-kit-hook-viability/checklists/evidence-index.md) |
 | 8 — Catalog binding viability | `specs/009-catalog-binding-viability/` | **Executed** (this session, from `main` post-PR-#36 `8c06dc6`): T001–T086 + T014a (87 tasks) all **executed**, of which **85 are checked** in [`tasks.md`](specs/009-catalog-binding-viability/tasks.md); **T051 and T055 are honestly left `- [ ]`** (see the T051/T055 correction below — both real-corpus derivations deterministically fail-closed-rejected, so the populated `SnapshotEnvelope` each task requires was never producible). **Recorded verdict: `blocked`** (`blockedShortfall: "envelope-or-scale-evidence-incomplete"`) — precedence evaluated in the fixed order `no-go`(did not fire) → `go-explicit`(did not fire) → `blocked`(fired, exhaustive fallback); no unsafe/no-go condition fired (determinism, atomicity, repository-isolation, envelope-rejection, and network/credential gates all held). The shortfall is driven by a genuine, pre-existing `duplicate-canonical-id` defect in the real, frozen `community-plugins`/`rhdh-plugins` corpora (fourteen unsubstituted Nunjucks template skeletons sharing one identical literal `metadata.name` placeholder, `${{ values.name | dump }}` — five in `community-plugins`, nine in `rhdh-plugins`; two further `rhdh-plugins` skeletons, `bulk-import` and `orchestrator`, carry *different* unsubstituted placeholders, `${{ values.name }}` and `${{ values.entityName }}`, which canonicalize to distinct ids and are therefore not part of that collision cohort, giving sixteen unsubstituted skeleton files in total but fourteen colliding ones) that correctly, deterministically, fail-closed-rejects 2 of the 3 required real-corpus passes with zero partial output — this is the generator working exactly as designed, not a bug, and is not a shipping-code defect since this spike introduces no shipping code. The `synthetic` pass fully produced a populated 9-entity envelope; B/C measurement (via an independent per-descriptor heuristic, unaffected by the snapshot-pipeline shortfall) succeeded for both real corpora and synthetic data. A frozen, maintainer-authored reference oracle (six required case classes: positive/negative/overlap/absent-vs-explicit-empty/collision/repository-mismatch) was authored and independently audited **before** any generator output existed (T014a); the network-denial gate used rank-1 rootless Podman `--network none` (live-verified as genuinely blocking, not merely unobserved) across all 11 gated derivation/probe runs, with a structural + empirical zero-credential-forwarding guarantee (T069). A final independent fresh-context audit (Claude Opus 4.8, no prior authoring involvement, T085) reviewed the complete evidence bundle against every FR/SC/contract and returned **PASS with zero defects**. **PR #37 review remediation (this session, same execution):** two genuine, high-confidence findings from an automated Copilot review were corrected — (1) the T014a oracle audit had used Claude Sonnet 4.6 rather than the instructed Opus 4.8/GPT-5.6 Sol; a compliant fresh Claude Opus 4.8 sub-agent independently re-verified all input/oracle hashes and re-derived all six case classes from raw fixture bytes, returning **PASS** (`reference-oracle-audit.opus48.json`, now the authoritative T014a record; the original file is preserved unmodified as an honest historical record) and additionally disclosing one further minor, non-verdict-changing finding the first audit missed (the oracle's `positive` case records its derived-paths array in input rather than `compareCodeUnits`-sorted order — the case class, discriminator, and accept/reject outcome are unaffected, and this spike's own tooling never diffs against these fixtures by name, so the already-frozen oracle was left uncorrected rather than risk backfilling it after generator work had already run); and (2) the Phase 7 synthetic fixture's `duplicate-yaml-key` case, previously only disclosed as internally mislabeled `invalid-yaml-syntax`, was root-caused (the scratch harness's classifier regex never matched yaml@2.9.0's actual `"Map keys must be unique..."` error text) and **genuinely fixed** in the spike's own scratch tooling (not shipping code), then both affected probes were re-run under the same rank-1 Podman `--network none` mechanism with full before/after provenance — the fixture now reports its own correct, distinct `duplicate-yaml-key` triggerClass with no caveat. Two smaller items were also corrected for accuracy: T051/T055 (US6 `SnapshotEnvelope` production for the two real corpora) are now honestly `- [ ]`, since both real-corpus derivations correctly, deterministically fail-closed-rejected and therefore no populated envelope — only a differently-shaped rejection record — could ever be produced (T052/T056 scale-evidence and T074's own named `envelope-or-scale-evidence-incomplete` shortfall are unaffected); and T076 (`go-explicit`-only recommendation drafting) gained an explicit "correctly skipped in its entirety" annotation, matching feature008's own convention, with no change to its already-correct `null` recommendation. **None of these corrections change the recorded `blocked` verdict**, which remains driven solely by the pre-existing `duplicate-canonical-id` corpus defect and the resulting envelope-population shortfall — both orthogonal to every corrected item above. **Raw transcripts, corpus checkouts, and the full evidence bundle (`spike-009-evidence.{json,md}`) remain scratch/session-artifact-only per FR-019 and are not tracked in this repository.** A tracked, sanitized evidence index (`checklists/evidence-index.md`, linked below) is nonetheless included, matching feature008's own precedent of tracking this housekeeping/audit-trail artifact regardless of verdict outcome — its presence is not itself a "landed" or `go-explicit` claim; this run does not reach `go-explicit`, and any future landing PR, if ever pursued, would need its own separately-authorized scope per FR-019/ADR-0014 rung 2. **This spike introduces no shipping/production code** (no `packages/adapters/catalog-*/**` exists) and **does not** constitute, cause, or substitute for Phase 6 external/community validation or optional externally-validated maturity, which remain exactly as recorded above (landed/reference-verified, rung 3 open). Any resulting change to `spec.md`/`plan.md`/`tasks.md` that these findings might suggest is a separate, later, explicitly-scoped follow-up decision — not something this execution performed or decided unilaterally. **Carry-forward blocker for any future feature009 work:** the frozen `reference-oracle.json`'s `positive` case records `expectedOutcome.derivedPathPatterns` in input order rather than the `compareCodeUnits`-sorted order `owned-paths-annotation.md` §3 mandates. It was deliberately **not** corrected in this run (correcting a frozen, pre-generator-output artifact after generator work had already run against its existence as a gate would be exactly the backfilling FR-025/T014a exist to prevent), and it changes no case class, discriminator, accept/reject outcome, or the recorded `blocked` verdict. But it means the oracle is **not** a valid ground truth to reuse as-is: any future feature009 execution — and any landing PR — MUST begin from a corrected, re-frozen, re-audited oracle (a fresh T014/T014a cycle) before producing any generator output. Recorded as a named limitation in [`checklists/evidence-index.md`](specs/009-catalog-binding-viability/checklists/evidence-index.md) so it cannot be silently inherited. **PR #37 second-round review remediation (auto-triggered by the first remediation push):** two further "low confidence" findings were investigated with the same rigor. (1) A temporal-ordering question — whether T014a's compliant Opus 4.8 rerun, performed during PR review after T017+ generator output already existed, reopens the "before any generator-derived output" requirement — was resolved as a **clarifying note only, no evidence/verdict change**: the audited object (`reference-oracle.json`, unchanged hash `8f0e260f…`) was never touched after being frozen pre-generator, and the compliant sub-agent's own scope was independently confirmed to be restricted to only the ten frozen input fixtures and the oracle file, with zero exposure to any generator-derived artifact — the content-immutability and audit-scope-isolation that the ordering rule exists to protect were both intact, so only the reviewer-model-identity correction (not the oracle's own authoring/freezing/hashing/first-audit sequence) occurred later; documented in `tasks.md` T014a. (2) A structural question — whether `envelopes.communityPlugins`/`envelopes.rhdhPlugins` referencing differently-shaped rejection records (rather than §9 `SnapshotEnvelope`-shaped files) violates T071's completeness check — was resolved as a genuine documentation clarification (`data-model.md` §22's `envelopes` row, `tasks.md` T071) confirming this is the exact, anticipated mechanism behind `Verdict.blockedShortfall = "envelope-or-scale-evidence-incomplete"` per `contracts/evidence-bundle-and-verdict.md` §2 Step 3's own named case ("the envelope ... could not be fully populated from an actual run for any pass"), **plus one genuine, narrow data fix**: `envelopes` had been omitted from `verdict.drivingEvidence` despite the shortfall naming that field directly — `envelopes` has been added to `drivingEvidence` in `verdict.json`/`spike-009-evidence.json`/`.md` (all three re-hashed; `checklists/evidence-index.md` updated to the new hashes), documented in `tasks.md` T075. **The recorded `blocked` verdict, `blockedShortfall`, and every underlying evidentiary fact are unchanged by either fix** — only `drivingEvidence`'s completeness and the relevant documentation improved. Full task-by-task, PR-review-remediation, and evidence accounting: [`specs/009-catalog-binding-viability/tasks.md`](specs/009-catalog-binding-viability/tasks.md), [`specs/009-catalog-binding-viability/checklists/evidence-index.md`](specs/009-catalog-binding-viability/checklists/evidence-index.md) |
+| 9 — Evaluator calibration | `specs/012-evaluator-calibration/` | **scoped** (PR #141 merged). The frozen holdout, the release-precondition gate, and the absence statement that make [ADR-0027](docs/adr/0027-ratify-the-deterministic-evaluator-and-bind-calibration-reporting-to-the-first-probabilistic-pass.md) §3 satisfiable. Deterministic and model-free (Principle IV). 52 FRs, 53 SCs, 77 tasks, 36 `[OBSERVE-FAIL]`, six deliberate `[NEEDS CLARIFICATION]`. No code; nothing under `packages/`. |
+| 10 — Probabilistic evaluator passes | `specs/011-probabilistic-evaluator-passes/` | **scoped** (PR #140 merged). Rubric passes 1–3 plus the three triggers deferred by ADR-0027 §2 (`low-confidence`, `pass-disagreement`, `novel-no-precedent`). Architecture is harness-driven — adrkit emits a prompt bundle and never opens a socket — which keeps Principle II intact; the choice needs its own record before implementation. 29 FRs, 17 SCs, 42 tasks, four open `[NEEDS CLARIFICATION]`. No code; nothing under `packages/`. |
+
+**Phases 9 and 10 run opposite to their feature numbers, deliberately.** Feature
+012 (calibration) is the *lower* phase because it gates feature 011: no
+probabilistic pass may ship without a holdout frozen before that pass produced
+its first score ([ADR-0027](docs/adr/0027-ratify-the-deterministic-evaluator-and-bind-calibration-reporting-to-the-first-probabilistic-pass.md)
+§3). Numbering them this way makes the rule below — implementation of a phase
+MUST NOT begin until the phase beneath it has landed — enforce that dependency
+mechanically, rather than leaving it to prose in each spec that a reader may not
+connect. Both were scoped concurrently, which is why the feature directories are
+numbered in the other order.
 
 ## Adapters
 
@@ -304,10 +316,17 @@ Exit criteria:
 ### Phase 7+ — Deferred
 
 Import adapters for agent logs (blocked: needs a real sample to specify a
-deterministic split), LLM rubric passes, index
-and web UI, catalog adapters. (Phase 6's ARB queue has landed — see the
-Spec-kit realization table above. The Spec Kit extension has shipped — see the
-Adapters table above.)
+deterministic split), index and web UI, catalog adapters. (Phase 6's ARB queue
+has landed — see the Spec-kit realization table above. The Spec Kit extension
+has shipped — see the Adapters table above.)
+
+The **LLM rubric passes** are no longer in this list: they are now scoped as
+Phase 10 (`specs/011-probabilistic-evaluator-passes/`), gated behind Phase 9
+(`specs/012-evaluator-calibration/`). Scoped, not implemented — implementation
+is blocked on two Phase 0 gates that cannot be satisfied today: feature 012's
+frozen holdout with its precondition gate **observed failing**
+([ADR-0016](docs/adr/0016-require-every-check-to-be-observed-failing-before-it-counts-as-coverage.md)),
+and an accepted record ratifying the harness-driven architecture.
 
 ---