Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions contracts/dogfood-runner-unification-v1.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ metadata:
- 'scripts/check_verifier_pinning.sh — the gate: static scan + behavioural probe of both pins'
- 'scripts/check_dogfood_shim.sh — the gate: one runner, one prose, and the user-scope copy stays a gated shim'
- 'scripts/check_dogfood_row_order.sh — the guard (report mode until three green nights): long rows last, no early stop, a red cheap row keeps the verdict red, row times (aprender#4672)'
- 'scripts/check_dogfood_reads_ci.sh + scripts/release/sha_checks.sh — pre-publish reads fmt and test from the CI of the same commit and coverage from the receipt cut_tag requires (tag_coverage_gate.sh --resolve); no green receipt for that sha is red; clippy stays measured (aprender#4672)'
- 'scripts/.dogfood-shim-reference.sh — the reviewed text of the user-scope shim'
- '.claude/skills/dogfood/SKILL.md — the one canonical prose, with an explicit `name:` (#2332)'
- 'aprender#2361 — a user-scope skill shadowed the repo one; hardening it edited a file that never ran'
Expand Down Expand Up @@ -443,6 +444,37 @@ falsification_tests:
test: 'bash scripts/check_dogfood_row_order.sh --self-test # the five timing rows'
if_fails: 'a release-day check over its budget passes its own timing row, or a missing measurement reads as on time'
mutation: 'the six planted timing rows (ok, over, at budget, hole, absent, a budget of 15m) must give rc 0, 1, 0, 2, 2, 2 (observed 2026-10-04)'
- id: F-DOG1-032
rule: in --phase pre-publish, fmt and test read the commit's CI and coverage reads the receipt cut_tag requires; none re-runs; clippy stays measured
prediction: >
In scripts/dogfood.sh the fmt and test rows are ci_mark calls inside the pre-publish
branch (fmt reads `ci / gate`, test reads `workspace-test`), and the coverage row is
cov_mark, which PASSes only when `tag_coverage_gate.sh --resolve HEAD` exits 0: the
coverage-nightly receipt (#4734) that autopilot cut_tag already requires before the tag.
Their measuring commands sit only in that branch's else, so every other phase still
measures them. clippy is measured in every phase, because the `ci / gate` lint is
narrower (default features, unused-variables allowed, a lib-only fallback).
`workspace-test` is nextest --lib over the workspace; the root facade's doctests and
bin tests are not in it. Reviews and a C314 round on aprender#4886 settled these.
test: 'bash scripts/check_dogfood_reads_ci.sh # R2'
if_fails: 'the release dogfood re-runs work CI already did for the same commit, a row reads the wrong check, clippy reads a narrower lint, or pre-publish passes a coverage row it never read'
mutation: 'put `gate fmt` back in pre-publish (rerun), point test at `ci / gate` (wrongspec), read clippy from CI (clippyread), pass coverage unread (covpass), drop the coverage row (covmissing); each must turn RED (observed 2026-10-06 in --self-test)'
- id: F-DOG1-033
rule: no green receipt for the exact sha is RED, never a skip
prediction: >
The runner's own CI-read block, ci_mark() and cov_mark(), lifted and driven against
planted check rows and a planted coverage reader for a real commit, give PASS only
when each check, or the coverage receipt, is green for THAT sha, and FAIL on a red
check for the sha, a missing check, a green check on a different sha, a refused or
other-sha coverage receipt, a failed GitHub read, missing readers and a tree with
uncommitted tracked changes. scripts/release/sha_checks.sh holds the same rule on its
own (newest run wins, a queued rerun counts as newest, runs that disagree at one start
time are red, pending, cancelled and skipped are red, a same-named check in another
workflow does not count); tag_coverage_gate.sh --self-test holds it for coverage (no
receipt, other sha, below COV_FLOOR, gh failing).
test: 'bash scripts/check_dogfood_reads_ci.sh --self-test; bash scripts/release/sha_checks.sh --self-test; bash scripts/release/tag_coverage_gate.sh --self-test'
if_fails: 'a check or receipt that is red, absent, or for another commit lets the release dogfood say GO'
mutation: 'let a dirty tree read HEAD (nodirty), make the no-receipt fallback a PASS (skipread), pass a refused coverage receipt (covskip), resolve another sha (covwrongsha); each must turn RED; sha_checks.sh plants 21 rows (observed 2026-10-06)'
proof_obligations:
- id: DRU-INV-001
type: invariant
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
{
"$schema": "https://docs.oasis-open.org/sarif/sarif/v2.1.0/errata01/os/schemas/sarif-schema-2.1.0.json",
"version": "2.1.0",
"runs": [
{
"tool": {"driver": {"name": "pmat"}},
"invocations": [{"executionSuccessful": true, "toolExecutionNotifications": []}],
"results": [
{
"ruleId": "duplication_scan_no_hits",
"level": "note",
"message": {"text": "The lexical duplication scan over head, all 1540 unmerged sibling branches and the merge-base-to-main range (0 files) found 0 hits for the 13 symbols the diff adds. The diff changes no Rust file, so the pmat semantic index (Rust-only) was queried but cannot reach it."},
"properties": {
"grounding": "measured",
"command": ["bash", "scripts/pr_review_duplication_scan.sh", "--base", "88bf12c8710dd29617a28add806a37c3edac42d2", "--head", "6bac6be648bdcee2c8b8c0c7c640d1419d34703b", "--json", "<out>"],
"exit_code": 0,
"stdout_sha256": "35d6586b94bd0567dd5909f59e9a6a8041e90d521e3fdcd466b93441535574fc",
"failure_scenario": "A re-implementation under a different name is invisible to a name match; that residual is not covered.",
"precision_class": "advisory"
}
},
{
"ruleId": "diff_patch_id_run_by_parent",
"level": "note",
"message": {"text": "diff_patch_id 30be26cb97a09421c8df64ea78e5fa9bd2505a7d over 88bf12c871..6bac6be648 (evidence/pr-review/4886/ excluded) was computed by the PARENT session, not by this lane, under a cop ruling: subagent-lock.sh exempts the parent by design. Control by the parent: the same function over e95ae8280f..deb4f19d5e gives 6ff6a07990, equal to the A2 line CI printed. The stdout_sha256 here is the digest of the reported id plus a newline, taken by this lane from the value relayed to it; this lane did not observe that stdout."},
"properties": {
"grounding": "measured",
"command": ["bash", "-c", ". scripts/lib/pr_review_patch_id.sh && prpid_compute <worktree> 88bf12c8710dd29617a28add806a37c3edac42d2 6bac6be648bdcee2c8b8c0c7c640d1419d34703b 4886"],
"exit_code": 0,
"stdout_sha256": "4ea3e412e1fbb421c8243f3851823ac59632a5d513949ef8636ebfb92f6aa850",
"failure_scenario": "If the relayed id were wrong, Arm 4 would find no receipt bound to the subject diff and refuse the merge, so the error cannot arm a merge that should not land.",
"precision_class": "advisory"
}
},
{
"ruleId": "lane_hook_refusals",
"level": "note",
"message": {"text": "This lane was refused by the PreToolUse hook subagent-lock.sh (a review lane runs read-only git only) on two git subcommands: patch-id (via prpid_compute) and archive (the disposable tree for the agy lane). Neither was routed around: the patch-id was relayed by the parent; the disposable tree was built with tar from the clean worktree (tracked files, no .git, with evidence/, target/ and .pmat/ left out), a file copy and not a git operation."},
"properties": {
"grounding": "asserted",
"rationale": "The refusals are the hook output this lane received verbatim; no command ran, so there is no exit code or stdout to measure.",
"failure_scenario": "A reader of the receipt would otherwise take the tar-built agy workspace for a git-archive tree, and the patch-id for one this lane computed.",
"precision_class": "advisory"
}
},
{
"ruleId": "dead_elif_pre_publish_coverage",
"level": "warning",
"message": {"text": "scripts/dogfood.sh line 1857, `elif [ \"$DOGFOOD_PHASE\" = pre-publish ]`, sits inside the `else` (line 1819) of the new `if [ \"$DOGFOOD_PHASE\" = pre-publish ]` at line 1816, which closes at line 1865. It can never be true, so the #3957 F1b branch and its FAIL message are dead code; pre-publish coverage is now decided only by cov_mark."},
"properties": {
"grounding": "measured",
"command": ["sed", "-n", "1816p;1819p;1857p;1865p", "scripts/dogfood.sh"],
"exit_code": 0,
"stdout_sha256": "24dcf29e1f7cec413070882991cd914f534409f6da1bbcee4e21703639b01afa",
"failure_scenario": "A later edit trusts the F1b branch (coverage below floor in pre-publish is RED with its owed-by message) to run; it does not.",
"precision_class": "advisory"
}
},
{
"ruleId": "ci_read_narrower_than_measured_test",
"level": "note",
"message": {"text": "In pre-publish the test row reads `workspace-test` (nextest --lib over the workspace) where the replaced row ran `cargo test $FEATS` (doctests, integration targets, bin tests). The runner comment in the 2c block discloses this; the receipt row says only `read, not re-run`."},
"properties": {
"grounding": "asserted",
"rationale": "Taken from the diff comment in scripts/dogfood.sh and CLAUDE.md Testing (only a subset of integration targets is wired into CI). Not measured by running anything.",
"failure_scenario": "A doctest or integration-only failure that CI does not run passes the pre-publish test row.",
"precision_class": "advisory"
}
}
]
},
{
"tool": {"driver": {"name": "cargo-mutants"}},
"invocations": [{"executionSuccessful": true, "toolExecutionNotifications": []}],
"results": [
{
"ruleId": "committed_plants_killed",
"level": "note",
"message": {"text": "All 9 committed planted defects of check_dogfood_reads_ci.sh are killed (--self-test exits 0); sha_checks.sh --self-test is 21/21."},
"properties": {
"grounding": "measured",
"command": ["bash", "scripts/check_dogfood_reads_ci.sh", "--self-test"],
"exit_code": 0,
"stdout_sha256": "542142ef51e2f92a7d6370d1e99d028e0e81b1e4b2ef29ee09f04a2e63adb258",
"failure_scenario": "If a plant survived, a runner defect the guard claims to catch would pass unnoticed.",
"precision_class": "advisory"
}
},
{
"ruleId": "extra_mutant_cov_inverted_killed",
"level": "note",
"message": {"text": "Extra mutant: cov_mark records PASS when the reader's rc is non-zero (-eq 0 changed to -ne 0), in a scratch copy of the guard tree. Killed: the guard exits 1 with 8 FAIL lines."},
"properties": {
"grounding": "measured",
"command": ["bash", "<scratch copy, rc test inverted>/scripts/check_dogfood_reads_ci.sh"],
"exit_code": 1,
"stdout_sha256": "66bab24d53e48cbc40b95cf25f5836e858a2f4d0a88c6a5006da929a6059c8df",
"failure_scenario": "If it survived, an inverted coverage verdict would pass.",
"precision_class": "advisory"
}
},
{
"ruleId": "extra_mutant_git_dir_survives",
"level": "warning",
"message": {"text": "Extra mutant outside the committed set: cov_mark without the GIT_DIR prefix on the tag_coverage_gate.sh call (the commit message says the gate's git reads go through GIT_DIR, with no cd). The guard on a scratch copy with that edit still exits 0: R3 drives cov_mark against a planted reader that does not read git."},
"properties": {
"grounding": "measured",
"command": ["bash", "<scratch copy, GIT_DIR prefix removed>/scripts/check_dogfood_reads_ci.sh"],
"exit_code": 0,
"stdout_sha256": "bd261c161c3a1c825e81c5cbf508bf68441b1f404b40fdb1f83a8a0f48bcfe60",
"failure_scenario": "A later edit drops GIT_DIR and the real tag_coverage_gate.sh reads git from the caller's cwd; the guard stays green and the commit message's claim is untested.",
"precision_class": "advisory"
}
},
{
"ruleId": "extra_mutant_spec_substring_survives",
"level": "warning",
"message": {"text": "Extra mutant outside the committed set: ci_mark matching the sha_checks line by a substring of the spec instead of the ` <spec> -- ` field. The guard on a scratch copy still exits 0. Likely behaviourally equivalent today (the two specs share no text); that is not proven."},
"properties": {
"grounding": "measured",
"command": ["bash", "<scratch copy, grep -F widened to the bare spec>/scripts/check_dogfood_reads_ci.sh"],
"exit_code": 0,
"stdout_sha256": "bd261c161c3a1c825e81c5cbf508bf68441b1f404b40fdb1f83a8a0f48bcfe60",
"failure_scenario": "A future spec that is a substring of another check's name would read that check's line and could PASS on it.",
"precision_class": "advisory"
}
}
]
},
{
"tool": {"driver": {"name": "antigravity"}},
"invocations": [{"executionSuccessful": true, "toolExecutionNotifications": []}],
"results": []
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"_type":"https://in-toto.io/Statement/v1","subject":[{"name":"git+https://github.com/paiml/aprender","digest":{"sha1":"6bac6be648bdcee2c8b8c0c7c640d1419d34703b"}}],"predicateType":"https://paiml.dev/attestations/pr-review/v2","predicate":{"skill_version":"2.1.0","attestation_level":"L1-self","pr":4886,"base_sha":"88bf12c8710dd29617a28add806a37c3edac42d2","head_sha":"6bac6be648bdcee2c8b8c0c7c640d1419d34703b","author_actor":{"kind":"agent","id":"agent:claude-opus-5-5/build-kaizen-4672-2c"},"reviewer_actor":{"kind":"agent","id":"agent:claude-sonnet-5-5/pr-review-4886-lane"},"affected_crates":[],"verdict":"FINDINGS","consultations":{"pmat":{"status":"consulted","transport":"cli","transport_unavailable":["mcp: not attempted this run (cli answered)"],"index_commit":"6bac6be648bdcee2c8b8c0c7c640d1419d34703b","index_is_ancestor":true,"index_worktree_dirty":false,"complexity_delta":[],"tdg_delta":[],"satd_introduced":[],"cache_hits":0,"duplication_hits":[],"duplication_coverage":{"rust":"lexical","shell":"lexical","python":"lexical","config":"lexical","docs":"lexical","other":"lexical","sibling_branches":"lexical","merge_base_to_main":"lexical"},"duplication_horizon":["head=6bac6be648bdcee2c8b8c0c7c640d1419d34703b","siblings=refs/remotes/origin/* unmerged into origin/main","merge_base_to_main=88bf12c8710dd29617a28add806a37c3edac42d2..refs/remotes/origin/main"],"horizon_branches_total":1540,"horizon_branches_scanned":1540,"merge_base_to_main_files":0,"symbols_searched":13,"queries":[{"q":"pre-publish coverage reads receipt tag_coverage_gate","command":["pmat","query","pre-publish coverage reads receipt tag_coverage_gate","--limit","10"],"cwd":"the review worktree at head","results":10,"note":"The stale index was rebuilt at head (94427 functions) by this query. The semantic index is Rust-only and the diff changes no Rust. No TODO/FIXME/HACK/XXX on any added line.","stdout_sha256":"0c2f0cb4a5ba3de18e72e2138df7d266883a839a2f4b44fb0539dcf7e65726ad"}]},"cuda":{"status":"not-triggered","trigger_reason":"no changed path or commit message matches a section 3.B pattern (guard --match-path on all 4 paths and --match-message both exit 1) and the diff makes no device-behaviour claim: a shell runner, a guard, a helper and a contract"},"crux":{"status":"not-triggered","trigger_reason":"the guard --match-crux-surface predicate matches none of the diff added lines, and the diff changes no subcommand, flag, route, MCP tool, config key or output format"},"mutation":{"status":"consulted","scope":"guard","attempted":12,"killed":10,"survivors":[{"mutant":"cov_mark without the GIT_DIR prefix on the tag_coverage_gate.sh call","file":"scripts/dogfood.sh","line":697,"killed":false},{"mutant":"ci_mark matches the sha_checks line by a bare spec substring","file":"scripts/dogfood.sh","line":687,"killed":false}]},"antigravity":{"status":"consulted","attempted":1,"agy_version":"agy 1.3.0","binary_path":"/home/noah/.local/bin/agy","model_id":"gemini-3.1-pro-high","model_family":"google/gemini (requested id; unverifiable from the receipt)","exit_code":0,"duration_seconds":109.296312252,"agy_status":"SUCCESS","usage":{"input_tokens":108355,"output_tokens":8950,"thinking_tokens":7055,"cache_read_tokens":758376,"total_tokens":117305},"output_check":{"structured_output_present":true,"reviewed":true,"schema_valid":true},"reverified_by_primary":false,"divergence":{"agreed":0,"agy_only":0,"primary_only":2,"contradicted":0},"findings":[]}},"findings_ref":{"path":"findings.sarif","sha256":"3925c7448eb668ad6c20319a8a3757ac829d6be324761ab801eeafdd235fa900"},"cost":{"input_tokens":108355,"output_tokens":8950,"wall_seconds":1500},"diff_patch_id":"30be26cb97a09421c8df64ea78e5fa9bd2505a7d","diff_patch_id_algo":"git-patch-id-verbatim/pinned-diff-v1","notes":["diff_patch_id was computed by the parent session under a cop ruling (subagent-lock.sh exempts the parent); control e95ae8280f..deb4f19d5e gives 6ff6a07990, equal to the CI A2 line. This lane was refused by subagent-lock.sh on patch-id and on archive and did not route around either; see SARIF rules diff_patch_id_run_by_parent and lane_hook_refusals."]}}
Loading
Loading