Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/apr-dogfood/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -1594,7 +1594,7 @@ The nine clusters at zero, largest first: `http-orchestrate-banco` (95),
`test-harness` (49), `rag-eval` (44), `qa-cgp` (37), `simulation` (18),
`orchestrate-pacha-secrets` (17).

**Report both numbers or neither (T2).** "5 of 14 clusters gated (35.7%)" without "144 of 894 features gated (16.1%)" beside it is a proxy masquerading as coverage, and the gate refuses to emit it — *on this line too*.
**Report both numbers or neither (T2).** "5 of 14 clusters gated (35.7%)" without "144 of 904 features gated (15.9%)" beside it is a proxy masquerading as coverage, and the gate refuses to emit it — *on this line too*.

The rule is about the NUMBER, not about a phrasing, and it is enforced on every
surface that can emit one: the gate's own report, the receipt, the output of
Expand Down
21 changes: 11 additions & 10 deletions contracts/apr-dogfood-coverage-v1.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ metadata:
created: '2026-08-22'
author: PAIML Engineering
references:
- 'docs/audits/surface_audit.csv — 142 of 830 features gated, across 28 binaries and 14 clusters, measured 2026-08-22 @ 4bbfeb07f; 142 of 832 features gated after the PERF-046 partial re-audit (PR #2742); 143 of 833 features gated after the PP-LLAMA-001 re-audit (GET /v1/effective-config added and swept by the dogfood route enumeration, router.rs rows re-pointed); 144 of 837 features gated after PP-066 L0-1b added the three `apr parity` flag rows (--per-op, --out, --threshold), which the dogfood sweep does not reach -- the denominator grew and the numerator did not; 144 of 838 features gated after #3726 added `apr tokenize encode` (the tokenizer-parity gate''s id source), which the dogfood sweep does not reach either; 144 of 839 features gated after ONT-10 S19 ledgered the bare `aprender-cbtop` root surface (interactive TUI / --headless), which the dogfood sweep does not reach; 144 of 851 features gated after ONT-4g (#4476) added the twelve pv/apr command rows the binary snapshot found unledgered, none of them gated; 144 of 853 features gated after the ONT-4g bin gate (surface_audit_bins_gate.sh) found `pv-sat` shipping with no row and its two surfaces were ledgered, neither gated; 144 of 857 features gated after ONT-10 S4 ledgered the 4 hf_* tools `aprender-orchestrate mcp` lists, none of them gated; 144 of 868 features gated after ONT-10 S9/S7 mirrored the eleven ONT-4g pv rows as `apr pv` rows (the apr node joins binary=apr), none of them gated; 144 of 894 features gated once main (with #4219''s 26 `<bin> --version` rows, each run and observed printing its SHA, which the dogfood sweep does not reach) merged into ONT-10''s 868'
- 'docs/audits/surface_audit.csv — 142 of 830 features gated, across 28 binaries and 14 clusters, measured 2026-08-22 @ 4bbfeb07f; 142 of 832 features gated after the PERF-046 partial re-audit (PR #2742); 143 of 833 features gated after the PP-LLAMA-001 re-audit (GET /v1/effective-config added and swept by the dogfood route enumeration, router.rs rows re-pointed); 144 of 837 features gated after PP-066 L0-1b added the three `apr parity` flag rows (--per-op, --out, --threshold), which the dogfood sweep does not reach -- the denominator grew and the numerator did not; 144 of 838 features gated after #3726 added `apr tokenize encode` (the tokenizer-parity gate''s id source), which the dogfood sweep does not reach either; 144 of 839 features gated after ONT-10 S19 ledgered the bare `aprender-cbtop` root surface (interactive TUI / --headless), which the dogfood sweep does not reach; 144 of 851 features gated after ONT-4g (#4476) added the twelve pv/apr command rows the binary snapshot found unledgered, none of them gated; 144 of 853 features gated after the ONT-4g bin gate (surface_audit_bins_gate.sh) found `pv-sat` shipping with no row and its two surfaces were ledgered, neither gated; 144 of 857 features gated after ONT-10 S4 ledgered the 4 hf_* tools `aprender-orchestrate mcp` lists, none of them gated; 144 of 868 features gated after ONT-10 S9/S7 mirrored the eleven ONT-4g pv rows as `apr pv` rows (the apr node joins binary=apr), none of them gated; 144 of 894 features gated once main (with #4219''s 26 `<bin> --version` rows, each run and observed printing its SHA, which the dogfood sweep does not reach) merged into ONT-10''s 868; 144 of 904 features gated after the bin gate found `aprender-ci-tools` (the C301 port of the CI helpers) shipping with no row and its ten surfaces (nine subcommands and --version) were ledgered, none of them gated (#5170)'
- 'docs/audits/surface_audit_elbow.png — k-selection evidence: inertia knee and silhouette over k in [2,45]'
- 'scripts/dogfood_cluster.py — the k-sweep that produced the elbow plot (k-selection ONLY; it does not write the columns)'
- 'scripts/check_no_cluster_id_keys.sh — T1 enforcement: nothing may key on `cluster_id`'
Expand Down Expand Up @@ -73,15 +73,15 @@ metadata:

overall:
command: "python3 scripts/dogfood_baseline.py # section: totals"
rows: 894
rows: 904
covered: 144
ratio: 0.1611 # 144/894 -- 16.1%, NOT "about 17%"
binaries: 29
binaries_at_zero_coverage: 28
ratio: 0.1593 # 144/904 -- 15.9%, NOT "about 16%"
binaries: 30
binaries_at_zero_coverage: 29
clusters: 14
clusters_at_zero_coverage: 9
note: >
All 144 covered features are in `apr`. 28 of 29 binaries have no gate at
All 144 covered features are in `apr`. 29 of 30 binaries have no gate at
all, including aprender-orchestrate (189 features), aprender-test-cli (40)
and pv (49).

Expand Down Expand Up @@ -119,6 +119,7 @@ metadata:
aprender-qa-readme-sync: {total: 2, covered: 0}
aprender-zram-generator: {total: 2, covered: 0}
pv-sat: {total: 2, covered: 0}
aprender-ci-tools: {total: 10, covered: 0}

per_cluster:
command: "python3 scripts/dogfood_baseline.py # section: per_cluster"
Expand Down Expand Up @@ -162,7 +163,7 @@ metadata:
data-pipeline: {total: 77, covered: 0} # ZERO
apr-lint-diag: {total: 71, covered: 55} # 77.5% of cluster, 38.2% of gate effort
orchestrate-agent-stack: {total: 56, covered: 0} # ZERO
test-harness: {total: 51, covered: 0} # ZERO
test-harness: {total: 61, covered: 0} # ZERO
http-apr-serve: {total: 45, covered: 40} # 88.9% of cluster, 27.8% of gate effort
profiling: {total: 46, covered: 1} # 2.2% of cluster, 0.7% of gate effort
rag-eval: {total: 46, covered: 0} # ZERO
Expand All @@ -180,7 +181,7 @@ metadata:
the 4.3% coverage of that band is why.
q1_2: {total: 80, covered: 58} # 72.5%
q3_4: {total: 49, covered: 27} # 55.1%
q5_6: {total: 706, covered: 29} # 4.1%
q5_6: {total: 716, covered: 29} # 4.1%
q7_8: {total: 38, covered: 10} # 26.3%
q9_10: {total: 21, covered: 20} # 95.2%

Expand Down Expand Up @@ -291,7 +292,7 @@ equations:
codomain: bool
invariants:
- "T2. Cluster coverage is a PROXY: one gate in a 95-member cluster is 1%, not covered"
- "5/14 clusters (35.7%) and 144/894 features (16.1%) are reported together, always"
- "5/14 clusters (35.7%) and 144/904 features (15.9%) are reported together, always"
- "Enforced mechanically by enforce_pairing(); deleting the feature fraction from the emitter turns the gate RED"
- "This is the single most important trap: reporting the proxy alone is how a coverage gate becomes theatre while looking stricter"

Expand Down Expand Up @@ -680,7 +681,7 @@ falsification_tests:
Every emitted line stating a CLUSTER-level ratio also states the underlying
FEATURE-level ratio, on EVERY channel one can be emitted through: this gate's
report, the receipt, scripts/dogfood_baseline.py and
.claude/skills/apr-dogfood/SKILL.md. 5 of 14 clusters (35.7%) is a proxy and 144 of 894 features (16.1%) is the thing.
.claude/skills/apr-dogfood/SKILL.md. 5 of 14 clusters (35.7%) is a proxy and 144 of 904 features (15.9%) is the thing.
One gate in a 95-member cluster is
1%. The rule keys on the NUMBER -- a denominator equal to the cluster count
-- not on a phrase, because a phrase can be reworded and a denominator cannot;
Expand Down
19 changes: 16 additions & 3 deletions contracts/contracts.nt
Original file line number Diff line number Diff line change
Expand Up @@ -840,19 +840,32 @@
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.w3.org/ns/prov#Entity> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://ont.paiml.dev/v1alpha1/Binary> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <https://ont.paiml.dev/v1alpha1/bin/target/aprender-ci-tools/aprender-ci-tools> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools --version"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools coverage-report-scope"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools dag-status"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools git-patch-id"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools llama-fit-verdict"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools package-include-diff"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools publishable-crates"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools tarball-build-errors"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools tarball-shrink-report"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/auditRow> "aprender-ci-tools tarball-workspace"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/command> "coverage-report-scope"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/command> "package-include-diff"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/command> "publishable-crates"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/gitSha> "8b7d6a04c5f36e957c2a3a71d661e294c41fabed"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/helpSha256> "6fd04d873b6672953a624dd01eaeb3f310d983ab50e4e79994a38b690b6be057"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/ledgerOrphan> "dag-status"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/ledgerOrphan> "git-patch-id"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/ledgerOrphan> "llama-fit-verdict"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/ledgerOrphan> "tarball-build-errors"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/ledgerOrphan> "tarball-shrink-report"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/ledgerOrphan> "tarball-workspace"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/name> "aprender-ci-tools"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/package> "aprender-ci-tools"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/sha256> "0a9534e3416ab0acf2dc3c6523caecc317b645cbbe66ea60bb2af4b1ccd032f2"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/srcPath> "crates/aprender-ci-tools/src/main.rs"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/target> "aprender-ci-tools"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/unledgeredCommand> "coverage-report-scope"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/unledgeredCommand> "package-include-diff"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/unledgeredCommand> "publishable-crates"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/versionSha256> "2ffcc380b81bf4779669ef921b1f2607d68308693adb5fe1cafbd5cc5beb2574"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-ci-tools/aprender-ci-tools> <https://ont.paiml.dev/v1alpha1/bin/versionString> "aprender-ci-tools 0.70.0 (8b7d6a04c5)"^^<http://www.w3.org/2001/XMLSchema#string> .
<https://ont.paiml.dev/v1alpha1/binary/aprender-compute-xtask/aprender-compute-xtask> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.w3.org/ns/prov#Entity> .
Expand Down
10 changes: 10 additions & 0 deletions docs/audits/surface_audit.csv
Original file line number Diff line number Diff line change
Expand Up @@ -893,3 +893,13 @@ aprender-train-shell,aprender-train-shell --version,7,x86_64-linux,NONE,no,12,te
aprender-verify,aprender-verify --version,7,x86_64-linux,NONE,no,0,entrypoint-and-mcp,crates/aprender-verify-ml/src/bin/verificar.rs:20,high
aprender-zram,aprender-zram --version,7,x86_64-linux,NONE,no,8,rag-eval,crates/aprender-zram-cli/src/lib.rs:23,high
aprender-zram-generator,aprender-zram-generator --version,7,x86_64-linux,NONE,no,0,entrypoint-and-mcp,crates/aprender-zram-generator/src/main.rs:33,high
aprender-ci-tools,aprender-ci-tools publishable-crates,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:27,high
aprender-ci-tools,aprender-ci-tools package-include-diff,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:30,high
aprender-ci-tools,aprender-ci-tools coverage-report-scope,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:38,high
aprender-ci-tools,aprender-ci-tools tarball-shrink-report,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:45,high
aprender-ci-tools,aprender-ci-tools tarball-workspace,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:54,high
aprender-ci-tools,aprender-ci-tools tarball-build-errors,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:67,high
aprender-ci-tools,aprender-ci-tools dag-status,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:76,high
aprender-ci-tools,aprender-ci-tools git-patch-id,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:83,high
aprender-ci-tools,aprender-ci-tools llama-fit-verdict,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:92,high
aprender-ci-tools,aprender-ci-tools --version,6,x86_64-linux,NONE,no,12,test-harness,crates/aprender-ci-tools/src/main.rs:15,high
18 changes: 18 additions & 0 deletions docs/roadmaps/entries/PMAT-5170.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
- id: PMAT-5170
github_issue: 5170
item_type: task
title: 'surface ledger: aprender-ci-tools row (0.71.1 P1)'
status: planned
priority: high
assigned_to: null
created: 2026-10-12T02:38:04Z
updated: 2026-10-12T02:38:04Z
spec: null
acceptance_criteria: []
phases: []
subtasks: []
estimated_effort: null
labels:
- kind:code
notes: null
epic: 3998
18 changes: 18 additions & 0 deletions docs/roadmaps/roadmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24735,3 +24735,21 @@ Not done, pending the ruling: moving the 31 fixture files to an obviously synthe
- kind:code
notes: null
epic: 3863
- id: PMAT-5170
github_issue: 5170
item_type: task
title: 'surface ledger: aprender-ci-tools row (0.71.1 P1)'
status: planned
priority: high
assigned_to: null
created: 2026-10-12T02:38:04Z
updated: 2026-10-12T02:38:04Z
spec: null
acceptance_criteria: []
phases: []
subtasks: []
estimated_effort: null
labels:
- kind:code
notes: null
epic: 3998
Loading