diff --git a/.claude/skills/apr-dogfood/SKILL.md b/.claude/skills/apr-dogfood/SKILL.md index ee49969642..2a7d71197d 100644 --- a/.claude/skills/apr-dogfood/SKILL.md +++ b/.claude/skills/apr-dogfood/SKILL.md @@ -1594,7 +1594,7 @@ The nine clusters at zero, largest first: `http-orchestrate-banco` (95), `test-harness` (49), `rag-eval` (44), `qa-cgp` (37), `simulation` (18), `orchestrate-pacha-secrets` (17). -**Report both numbers or neither (T2).** "5 of 14 clusters gated (35.7%)" without "144 of 894 features gated (16.1%)" beside it is a proxy masquerading as coverage, and the gate refuses to emit it — *on this line too*. +**Report both numbers or neither (T2).** "5 of 14 clusters gated (35.7%)" without "144 of 904 features gated (15.9%)" beside it is a proxy masquerading as coverage, and the gate refuses to emit it — *on this line too*. The rule is about the NUMBER, not about a phrasing, and it is enforced on every surface that can emit one: the gate's own report, the receipt, the output of diff --git a/contracts/apr-dogfood-coverage-v1.yaml b/contracts/apr-dogfood-coverage-v1.yaml index 2697fbe38e..9e674338ef 100644 --- a/contracts/apr-dogfood-coverage-v1.yaml +++ b/contracts/apr-dogfood-coverage-v1.yaml @@ -3,7 +3,7 @@ metadata: created: '2026-08-22' author: PAIML Engineering references: - - 'docs/audits/surface_audit.csv — 142 of 830 features gated, across 28 binaries and 14 clusters, measured 2026-08-22 @ 4bbfeb07f; 142 of 832 features gated after the PERF-046 partial re-audit (PR #2742); 143 of 833 features gated after the PP-LLAMA-001 re-audit (GET /v1/effective-config added and swept by the dogfood route enumeration, router.rs rows re-pointed); 144 of 837 features gated after PP-066 L0-1b added the three `apr parity` flag rows (--per-op, --out, --threshold), which the dogfood sweep does not reach -- the denominator grew and the numerator did not; 144 of 838 features gated after #3726 added `apr tokenize encode` (the tokenizer-parity gate''s id source), which the dogfood sweep does not reach either; 144 of 839 features gated after ONT-10 S19 ledgered the bare `aprender-cbtop` root surface (interactive TUI / --headless), which the dogfood sweep does not reach; 144 of 851 features gated after ONT-4g (#4476) added the twelve pv/apr command rows the binary snapshot found unledgered, none of them gated; 144 of 853 features gated after the ONT-4g bin gate (surface_audit_bins_gate.sh) found `pv-sat` shipping with no row and its two surfaces were ledgered, neither gated; 144 of 857 features gated after ONT-10 S4 ledgered the 4 hf_* tools `aprender-orchestrate mcp` lists, none of them gated; 144 of 868 features gated after ONT-10 S9/S7 mirrored the eleven ONT-4g pv rows as `apr pv` rows (the apr node joins binary=apr), none of them gated; 144 of 894 features gated once main (with #4219''s 26 ` --version` rows, each run and observed printing its SHA, which the dogfood sweep does not reach) merged into ONT-10''s 868' + - 'docs/audits/surface_audit.csv — 142 of 830 features gated, across 28 binaries and 14 clusters, measured 2026-08-22 @ 4bbfeb07f; 142 of 832 features gated after the PERF-046 partial re-audit (PR #2742); 143 of 833 features gated after the PP-LLAMA-001 re-audit (GET /v1/effective-config added and swept by the dogfood route enumeration, router.rs rows re-pointed); 144 of 837 features gated after PP-066 L0-1b added the three `apr parity` flag rows (--per-op, --out, --threshold), which the dogfood sweep does not reach -- the denominator grew and the numerator did not; 144 of 838 features gated after #3726 added `apr tokenize encode` (the tokenizer-parity gate''s id source), which the dogfood sweep does not reach either; 144 of 839 features gated after ONT-10 S19 ledgered the bare `aprender-cbtop` root surface (interactive TUI / --headless), which the dogfood sweep does not reach; 144 of 851 features gated after ONT-4g (#4476) added the twelve pv/apr command rows the binary snapshot found unledgered, none of them gated; 144 of 853 features gated after the ONT-4g bin gate (surface_audit_bins_gate.sh) found `pv-sat` shipping with no row and its two surfaces were ledgered, neither gated; 144 of 857 features gated after ONT-10 S4 ledgered the 4 hf_* tools `aprender-orchestrate mcp` lists, none of them gated; 144 of 868 features gated after ONT-10 S9/S7 mirrored the eleven ONT-4g pv rows as `apr pv` rows (the apr node joins binary=apr), none of them gated; 144 of 894 features gated once main (with #4219''s 26 ` --version` rows, each run and observed printing its SHA, which the dogfood sweep does not reach) merged into ONT-10''s 868; 144 of 904 features gated after the bin gate found `aprender-ci-tools` (the C301 port of the CI helpers) shipping with no row and its ten surfaces (nine subcommands and --version) were ledgered, none of them gated (#5170)' - 'docs/audits/surface_audit_elbow.png — k-selection evidence: inertia knee and silhouette over k in [2,45]' - 'scripts/dogfood_cluster.py — the k-sweep that produced the elbow plot (k-selection ONLY; it does not write the columns)' - 'scripts/check_no_cluster_id_keys.sh — T1 enforcement: nothing may key on `cluster_id`' @@ -73,15 +73,15 @@ metadata: overall: command: "python3 scripts/dogfood_baseline.py # section: totals" - rows: 894 + rows: 904 covered: 144 - ratio: 0.1611 # 144/894 -- 16.1%, NOT "about 17%" - binaries: 29 - binaries_at_zero_coverage: 28 + ratio: 0.1593 # 144/904 -- 15.9%, NOT "about 16%" + binaries: 30 + binaries_at_zero_coverage: 29 clusters: 14 clusters_at_zero_coverage: 9 note: > - All 144 covered features are in `apr`. 28 of 29 binaries have no gate at + All 144 covered features are in `apr`. 29 of 30 binaries have no gate at all, including aprender-orchestrate (189 features), aprender-test-cli (40) and pv (49). @@ -119,6 +119,7 @@ metadata: aprender-qa-readme-sync: {total: 2, covered: 0} aprender-zram-generator: {total: 2, covered: 0} pv-sat: {total: 2, covered: 0} + aprender-ci-tools: {total: 10, covered: 0} per_cluster: command: "python3 scripts/dogfood_baseline.py # section: per_cluster" @@ -162,7 +163,7 @@ metadata: data-pipeline: {total: 77, covered: 0} # ZERO apr-lint-diag: {total: 71, covered: 55} # 77.5% of cluster, 38.2% of gate effort orchestrate-agent-stack: {total: 56, covered: 0} # ZERO - test-harness: {total: 51, covered: 0} # ZERO + test-harness: {total: 61, covered: 0} # ZERO http-apr-serve: {total: 45, covered: 40} # 88.9% of cluster, 27.8% of gate effort profiling: {total: 46, covered: 1} # 2.2% of cluster, 0.7% of gate effort rag-eval: {total: 46, covered: 0} # ZERO @@ -180,7 +181,7 @@ metadata: the 4.3% coverage of that band is why. q1_2: {total: 80, covered: 58} # 72.5% q3_4: {total: 49, covered: 27} # 55.1% - q5_6: {total: 706, covered: 29} # 4.1% + q5_6: {total: 716, covered: 29} # 4.1% q7_8: {total: 38, covered: 10} # 26.3% q9_10: {total: 21, covered: 20} # 95.2% @@ -291,7 +292,7 @@ equations: codomain: bool invariants: - "T2. Cluster coverage is a PROXY: one gate in a 95-member cluster is 1%, not covered" - - "5/14 clusters (35.7%) and 144/894 features (16.1%) are reported together, always" + - "5/14 clusters (35.7%) and 144/904 features (15.9%) are reported together, always" - "Enforced mechanically by enforce_pairing(); deleting the feature fraction from the emitter turns the gate RED" - "This is the single most important trap: reporting the proxy alone is how a coverage gate becomes theatre while looking stricter" @@ -680,7 +681,7 @@ falsification_tests: Every emitted line stating a CLUSTER-level ratio also states the underlying FEATURE-level ratio, on EVERY channel one can be emitted through: this gate's report, the receipt, scripts/dogfood_baseline.py and - .claude/skills/apr-dogfood/SKILL.md. 5 of 14 clusters (35.7%) is a proxy and 144 of 894 features (16.1%) is the thing. + .claude/skills/apr-dogfood/SKILL.md. 5 of 14 clusters (35.7%) is a proxy and 144 of 904 features (15.9%) is the thing. One gate in a 95-member cluster is 1%. The rule keys on the NUMBER -- a denominator equal to the cluster count -- not on a phrase, because a phrase can be reworded and a denominator cannot; diff --git a/contracts/contracts.nt b/contracts/contracts.nt index 0e356345be..f09e24b2be 100644 --- a/contracts/contracts.nt +++ b/contracts/contracts.nt @@ -840,19 +840,32 @@ . . . + "aprender-ci-tools --version"^^ . + "aprender-ci-tools coverage-report-scope"^^ . + "aprender-ci-tools dag-status"^^ . + "aprender-ci-tools git-patch-id"^^ . + "aprender-ci-tools llama-fit-verdict"^^ . + "aprender-ci-tools package-include-diff"^^ . + "aprender-ci-tools publishable-crates"^^ . + "aprender-ci-tools tarball-build-errors"^^ . + "aprender-ci-tools tarball-shrink-report"^^ . + "aprender-ci-tools tarball-workspace"^^ . "coverage-report-scope"^^ . "package-include-diff"^^ . "publishable-crates"^^ . "8b7d6a04c5f36e957c2a3a71d661e294c41fabed"^^ . "6fd04d873b6672953a624dd01eaeb3f310d983ab50e4e79994a38b690b6be057"^^ . + "dag-status"^^ . + "git-patch-id"^^ . + "llama-fit-verdict"^^ . + "tarball-build-errors"^^ . + "tarball-shrink-report"^^ . + "tarball-workspace"^^ . "aprender-ci-tools"^^ . "aprender-ci-tools"^^ . "0a9534e3416ab0acf2dc3c6523caecc317b645cbbe66ea60bb2af4b1ccd032f2"^^ . "crates/aprender-ci-tools/src/main.rs"^^ . "aprender-ci-tools"^^ . - "coverage-report-scope"^^ . - "package-include-diff"^^ . - "publishable-crates"^^ . "2ffcc380b81bf4779669ef921b1f2607d68308693adb5fe1cafbd5cc5beb2574"^^ . "aprender-ci-tools 0.70.0 (8b7d6a04c5)"^^ . . diff --git a/docs/audits/surface_audit.csv b/docs/audits/surface_audit.csv index 7f62ff9918..9e537d7804 100644 --- a/docs/audits/surface_audit.csv +++ b/docs/audits/surface_audit.csv @@ -893,3 +893,13 @@ aprender-train-shell,aprender-train-shell --version,7,x86_64-linux,NONE,no,12,te aprender-verify,aprender-verify --version,7,x86_64-linux,NONE,no,0,entrypoint-and-mcp,crates/aprender-verify-ml/src/bin/verificar.rs:20,high aprender-zram,aprender-zram --version,7,x86_64-linux,NONE,no,8,rag-eval,crates/aprender-zram-cli/src/lib.rs:23,high aprender-zram-generator,aprender-zram-generator --version,7,x86_64-linux,NONE,no,0,entrypoint-and-mcp,crates/aprender-zram-generator/src/main.rs:33,high +aprender-ci-tools,aprender-ci-tools publishable-crates,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:27,high +aprender-ci-tools,aprender-ci-tools package-include-diff,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:30,high +aprender-ci-tools,aprender-ci-tools coverage-report-scope,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:38,high +aprender-ci-tools,aprender-ci-tools tarball-shrink-report,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:45,high +aprender-ci-tools,aprender-ci-tools tarball-workspace,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:54,high +aprender-ci-tools,aprender-ci-tools tarball-build-errors,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:67,high +aprender-ci-tools,aprender-ci-tools dag-status,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:76,high +aprender-ci-tools,aprender-ci-tools git-patch-id,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:83,high +aprender-ci-tools,aprender-ci-tools llama-fit-verdict,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:92,high +aprender-ci-tools,aprender-ci-tools --version,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:15,high diff --git a/docs/roadmaps/entries/PMAT-5170.yaml b/docs/roadmaps/entries/PMAT-5170.yaml new file mode 100644 index 0000000000..befff67a30 --- /dev/null +++ b/docs/roadmaps/entries/PMAT-5170.yaml @@ -0,0 +1,18 @@ +- id: PMAT-5170 + github_issue: 5170 + item_type: task + title: 'surface ledger: aprender-ci-tools row (0.71.1 P1)' + status: planned + priority: high + assigned_to: null + created: 2026-10-12T02:38:04Z + updated: 2026-10-12T02:38:04Z + spec: null + acceptance_criteria: [] + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: null + epic: 3998 diff --git a/docs/roadmaps/roadmap.yaml b/docs/roadmaps/roadmap.yaml index 0694ad21b1..57ddce108c 100644 --- a/docs/roadmaps/roadmap.yaml +++ b/docs/roadmaps/roadmap.yaml @@ -24735,3 +24735,21 @@ Not done, pending the ruling: moving the 31 fixture files to an obviously synthe - kind:code notes: null epic: 3863 +- id: PMAT-5170 + github_issue: 5170 + item_type: task + title: 'surface ledger: aprender-ci-tools row (0.71.1 P1)' + status: planned + priority: high + assigned_to: null + created: 2026-10-12T02:38:04Z + updated: 2026-10-12T02:38:04Z + spec: null + acceptance_criteria: [] + phases: [] + subtasks: [] + estimated_effort: null + labels: + - kind:code + notes: null + epic: 3998