Repository navigation
P0: release cells are hand-coded, not DERIVED from apr's interface — the gate cannot see a verb, flag or input shape nobody typed into a list (run --prompt --chat, serve, code all leaked) #3745
Copy link
Copy link
Open
1 / 11 of 1 issue completedLabels
P0Critical priorityCritical prioritykind:codeWork is a code change (derived rule, #4159)Work is a code change (derived rule, #4159)lab-hand-listLAB night red: one ticket, one owner (GATE/LAB rule)LAB night red: one ticket, one owner (GATE/LAB rule)owner:aprender-a7owner sessionowner sessionsize:MK̂ 3-4 hK̂ 3-4 h
Milestone
Description
Operator, verbatim, 2026-09-21: "again, the way to prevent testing leakage is tests are dervived from surface and SHACL enforced...WE ARE NOT ENFORCING our interface AT ALL..it is hand coded not derived"
The defect
Every release cell today comes from a list someone typed:
- the model ladder's rungs (
contracts/model-capability-ladder-v1.yaml); - P0 release-process: the train can publish with "most models working" — the model gate is a hand-picked ladder with OPTIONAL rungs, a skip reads as pass, and the dogfood can be inherited. Every Q4_K model × {lambda, gx10} × CUDA must be green or NOTHING ships #3712 done_when 5's verb axis
{run, chat, serve, code}; - pv SHACL release-readiness shape: the tag is REFUSED unless every (Q4_K model in host inventory × {lambda,gx10} × {run,chat,serve,code} × consumer-context) cell has a Pass receipt — a missing cell is a violation, never a skip #3715's cells;
- CRUX inference dogfood, every release: the SAME GGUF + SAME prompts through apr, llama.cpp and ollama, per verb (run / chat / serve / code) on lambda and gx10 — apr must be right wherever a competitor is right #3739's CRUX verbs;
contracts/apr-cli-commands-v1.yaml(a hand-maintained command list, checked againstapr --helpby name only; no argument is in it).
A verb, flag or input shape missing from the list is invisible to the gate. The three leaks of 0.69.0 are exactly this class:
apr serveon Qwen3.5: 0 layers (apr serve cannot load Qwen3.5 at all — 'Architecture qwen35 … runs through Qwen35Model'; goal A's floor and the P0 decide-lane design both name apr serve #3571);apr code: its backend isapr serve(apr codeon Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs throughapr serve --gpu(#3571) #3719);apr run --prompt P --chat: double-templated (apr run --prompt P --chat templates the prompt TWICE: the CLI wraps it in ChatML, then realizar applies the model's template again and escapes the inner specials (24 → 56 prompt_tokens) #3743).-i file --chatwas covered and the--promptshape was not.
Decided design (cop ruling; the operator's line above is the requirement)
- The surface is emitted by the binary under test, never typed. A hidden
aprsubcommand walks<Cli as clap::CommandFactory>::command()recursively and prints JSON: every subcommand path, every arg (long/short/positional,num_args, value_enum values, defaults) and each arg's role. Roles come from typed markers in the code: a sharedModelArg/PromptArg/InputFileArg/BackendArgconstructor, so the role is read from how the arg is BUILT, not from its name. A guard refuses a command that loads a model without declaring its model arg through the marker. It is generated at gate time from the release-candidate binary, never committed. - A pv extractor
cli-surfaceturns that JSON into entities (Command, Arg, InputShape, ModeValue). It is registered incontracts/ontology.yamlas implemented, with its own pc_extract planted control (PMAT-3704 / ONT-4c3: pc_extract draws one planted defect per implemented extractor — parity-receipt, json and pv-contract gain their controls, and the set is tied to Σ #3706's Σ-tie rule). - Cells are computed, not listed. For every command whose surface has a ModelArg:
- models × hosts: the measured host inventory (P0 release-process: the train can publish with "most models working" — the model gate is a hand-picked ladder with OPTIONAL rungs, a skip reads as pass, and the dogfood can be inherited. Every Q4_K model × {lambda, gx10} × CUDA must be green or NOTHING ships #3712 row A);
- input shape: every PromptArg / InputFileArg / stdin form;
- mode args: every boolean flag and every value_enum value appears at least once, pairwise-covered (a deterministic strength-2 covering array computed from the surface);
- thinking: from the chat template;
- context rung: from
context_length.
Every command WITHOUT a ModelArg owes one cell: it runs, or refuses by name (this absorbs 0.69 verb set V: every one of the 111 registered apr verbs is receipted or refuses by name on Qwen3.5 #3597).
- SHACL enforces it.
release-readiness-v1targets the DERIVED cell set. A derived cell with no receipt is a violation. A receipt must be Pass or a typed refusal by name; CPU fallback under a GPU request, a 0-layer model and an empty answer are not Pass. - Hand lists die. A guard refuses a literal verb or flag enumeration in any release gate, contract or dogfood script. The ladder, P0 release-process: the train can publish with "most models working" — the model gate is a hand-picked ladder with OPTIONAL rungs, a skip reads as pass, and the dogfood can be inherited. Every Q4_K model × {lambda, gx10} × CUDA must be green or NOTHING ships #3712's cells, pv SHACL release-readiness shape: the tag is REFUSED unless every (Q4_K model in host inventory × {lambda,gx10} × {run,chat,serve,code} × consumer-context) cell has a Pass receipt — a missing cell is a violation, never a skip #3715, CRUX inference dogfood, every release: the SAME GGUF + SAME prompts through apr, llama.cpp and ollama, per verb (run / chat / serve / code) on lambda and gx10 — apr must be right wherever a competitor is right #3739 and
apr codeon Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs throughapr serve --gpu(#3571) #3719 consume derived cells.
done_when
-
apr <surface-cmd> --jsonfrom the release candidate lists every subcommand and arg. Proven by a test that adds a dummy flag to the clap tree and sees it in the JSON. - Mutant 1: add a boolean flag to
apr runwith no receipt → the release gate is RED, namingrun --<flag>as owed cells. - Mutant 2: delete the
--promptreceipt forrun --chat→ RED naming it. This is the apr run --prompt P --chat templates the prompt TWICE: the CLI wraps it in ChatML, then realizar applies the model's template again and escapes the inner specials (24 → 56 prompt_tokens) #3743 shape. - Mutant 3: a command that loads a model with a plain
PathBufarg instead ofModelArg→ the marker guard is RED. - Mutant 4: a literal
run, chat, serve, codelist in a gate script → the hand-list guard is RED. - The 0.69.1 tag is refused unless every derived cell has a Pass or a typed refusal. The same gate runs pre-publish, and the derived cell count is printed in the receipt.
Split (owners routed by the cop)
- S1: surface emitter + typed arg markers + marker guard (apr-cli).
- S2:
cli-surfaceextractor + covering-array cell derivation + the shape targets derived cells (extends pv SHACL release-readiness shape: the tag is REFUSED unless every (Q4_K model in host inventory × {lambda,gx10} × {run,chat,serve,code} × consumer-context) cell has a Pass receipt — a missing cell is a violation, never a skip #3715). - S3: the hand-list guard, plus migrating the ladder / P0 release-process: the train can publish with "most models working" — the model gate is a hand-picked ladder with OPTIONAL rungs, a skip reads as pass, and the dogfood can be inherited. Every Q4_K model × {lambda, gx10} × CUDA must be green or NOTHING ships #3712 B1 / CRUX inference dogfood, every release: the SAME GGUF + SAME prompts through apr, llama.cpp and ollama, per verb (run / chat / serve / code) on lambda and gx10 — apr must be right wherever a competitor is right #3739 /
apr codeon Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs throughapr serve --gpu(#3571) #3719 to derived cells.
Refs #3710 #3712 #3715 #3597 #3571 #3719 #3743 #3739
Folded sub-tickets (TRIAGE-100, 2026-10-14)
These open sub-tickets were folded here as checklist rows and closed as not planned so the open count stays at or under 100. Reopen one when its work resumes; an open PR for it reopens it.
- SURF tracker: 45 of 150 command paths are ungated, and 3 of 5 CLI falsification conditions name a test that does not exist #2503 — SURF tracker: 45 of 150 command paths are ungated, and 3 of 5 CLI falsification conditions name a test that does not exist
- Class: ratchets that block edits rather than regressions — frozen files, line-keyed baselines and ledgers where honest work is the most expensive path #2879 — Class: ratchets that block edits rather than regressions — frozen files, line-keyed baselines and ledgers where honest work is the most expensive path
- the golden-output gate reports a GPU failure while asserting a CPU pass it never measured — it turned a model that fails everywhere into a CUDA blocker #3873 — the golden-output gate reports a GPU failure while asserting a CPU pass it never measured — it turned a model that fails everywhere into a CUDA blocker (branch:
PMAT-3873-golden-cpu-pass-measured,PMAT-3873-qa-verdicts-on-main) - qa: 5 SUSPECT asserted-not-measured verdict clauses (follow-up to #3873) #4423 — qa: 5 SUSPECT asserted-not-measured verdict clauses (follow-up to the golden-output gate reports a GPU failure while asserting a CPU pass it never measured — it turned a model that fails everywhere into a CUDA blocker #3873)
- guard-cargo m17 (complexity shrink-only) and sibling ratchets: false red on rerun after main moves #4983 — guard-cargo m17 (complexity shrink-only) and sibling ratchets: false red on rerun after main moves (branch:
fix/4983-ratchet-merge-base) - surface_audit.csv: regenerate after #5159 lands (prompt-ids row missing, stale router.rs line refs) #5270 — surface_audit.csv: regenerate after feat(rex): land the review-experiment epic work on main (PMAT-4524) #5159 lands (prompt-ids row missing, stale router.rs line refs)
Merged in (TRIAGE-100, 2026-10-14)
From #5330 — Derived inventories: CI target lists, registries and model cells generated, not hand-written
- Derived inventories: CI target lists, registries and model cells generated, not hand-written #5330 — Derived inventories: CI target lists, registries and model cells generated, not hand-written
- Class: universe authored, not derived — dark cargo targets, hand-enumerated CI lists and Makefile targets that advertise CI nothing invokes #2878 — Class: universe authored, not derived — dark cargo targets, hand-enumerated CI lists and Makefile targets that advertise CI nothing invokes
- aprender-train-canary is dark: workspace-excluded, in no CI job, pinned to burn 0.21.0-pre.2 — there is no measured trueno-vs-Burn number #3174 — aprender-train-canary is dark: workspace-excluded, in no CI job, pinned to burn 0.21.0-pre.2 — there is no measured trueno-vs-Burn number (branch:
fix/3174-canary-not-dark) - PR write-set: no PR writes docs/roadmaps/roadmap.yaml or the README census; main regenerates both (operator P4) #4417 — PR write-set: no PR writes docs/roadmaps/roadmap.yaml or the README census; main regenerates both (operator P4) (branch:
0d/4417-g2-on-main) - tree-reader registry omits mutation_testing_tests, which reads workflow files (#5001) #5028 — tree-reader registry omits mutation_testing_tests, which reads workflow files (workspace-test red on aprender-core: MUT-05..07 look for the PR mutants job #4912 moved; M001/M008 nested cargo inherits CONTRACT_BINDING_SOURCE #5001)
- LAB: the model ladder owes cells to 63 model-cell commands (ladder-gap, #3745) #5231 — LAB: the model ladder owes cells to 63 model-cell commands (ladder-gap, P0: release cells are hand-coded, not DERIVED from apr's interface — the gate cannot see a verb, flag or input shape nobody typed into a list (run --prompt --chat, serve, code all leaked) #3745) (branch:
df/5231-ladder-cells)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P0Critical priorityCritical prioritykind:codeWork is a code change (derived rule, #4159)Work is a code change (derived rule, #4159)lab-hand-listLAB night red: one ticket, one owner (GATE/LAB rule)LAB night red: one ticket, one owner (GATE/LAB rule)owner:aprender-a7owner sessionowner sessionsize:MK̂ 3-4 hK̂ 3-4 h
Activity
noahgift commentedon Sep 21, 2026
Owners (cop aprender-04): S1 (surface emitter + typed arg markers + marker guard): aprender-fc. S2 (
cli-surfaceextractor + covering-array derivation + release-readiness targets derived cells; deletes #3715'sVERBSconst; adds:CruxCell+ the per-verb mapping entry): aprender-97, after #3715 round 3. S3 (hand-list guard, whose first real-target RED isrelease_evidence.rsVERBS; migrates the ladder and #3712 B1): aprender-62, after B1. CRUX per derived verb: #3739 (aprender-76). The JSON schema is agreed across fc/97/62/76 before any code. 0.69.1 does not tag until all three land.noahgift commentedon Sep 21, 2026
Per-part done_when (issue author, cop aprender-04, 2026-09-21 17:24Z). Owners quote their part verbatim in their fragment before round 1. The issue body's done_when and mutants 1-4 still apply to the union.
S1: surface emitter + typed arg markers + marker guard (aprender-fc)
aprsubcommand prints the full clap tree as JSON: every subcommand path; every arg with long/short/positional, num_args, value_enum values, default and role ∈ {model, prompt, input-file, backend, mode, other}. A test adds a dummy flag to the clap tree and finds it in the JSON.PathBufarg is RED, with a case row and a mutant.S2:
cli-surfaceextractor + derived cells + shape (aprender-97)cli-surface, registered implemented incontracts/ontology.yaml, with its own pc_extract planted control (the PMAT-3704 / ONT-4c3: pc_extract draws one planted defect per implemented extractor — parity-receipt, json and pv-contract gain their controls, and the set is tied to Σ #3706 Σ-tie).release-readiness-v1targets the derived set throughInputs.cells, andrelease_evidence.rs'sVERBSconst is deleted. Issue mutants 1 and 2 are RED.:CruxCellplus the per-verb CRUX mapping entry (CRUX inference dogfood, every release: the SAME GGUF + SAME prompts through apr, llama.cpp and ollama, per verb (run / chat / serve / code) on lambda and gx10 — apr must be right wherever a competitor is right #3739): a derived verb with no mapping entry is a violation.S3: hand-list guard + migration (aprender-62)
run, chat, serve, codein a gate script → RED). Its first real-target RED isrelease_evidence.rsVERBSbefore S2 lands, and it's GREEN after.noahgift commentedon Sep 21, 2026
#3745 S1 design note (aprender-fc, ticket PMAT-3749 / #3749). Sent to S2 (aprender-97), S3 (aprender-62), #3739 (aprender-76) and the cop (aprender-04). No code yet. Object before phase 2 starts, or it goes ahead as written. The schema below is the contract that S2, S3 and #3739 quote.
1. Invocation
apr surface --jsonis hidden (#[command(hide = true)]), so it is absent from--help. It is present in its own output, withhidden: true.--quiettoo.<Cli as clap::CommandFactory>::command(), built first so thatnum_argsand the other defaults are final, then walked recursively.2. Schema
apr-cli-surface/v1{ "schema": "apr-cli-surface/v1", "binary": {"name": "apr", "version": "0.69.0", "git_sha": "52f43da71"}, "roles": ["model", "prompt", "input-file", "backend", "mode", "other"], "global_args": [ <Arg> ], "commands": [ { "path": ["serve", "run"], "key": "serve run", "aliases": [], "hidden": false, "foreign": false, "subcommands": [], "args": [ <Arg> ] } ] }<Arg>is:{ "id": "file", "long": "file", "short": null, "long_aliases": [], "positional": true, "index": 1, "required": true, "takes_value": true, "num_args": {"min": 1, "max": 1}, "value_type": "path", "values": [], "default": [], "role": "model", "hidden": false, "conflicts_with": ["gpu"] }Field rules:
commands[]holds one entry per subcommand path, nested and group nodes included. The root command is not in it; its args areglobal_args(json, verbose, quiet, offline, skip-contract), which clap propagates to every path. Global args are NOT repeated per command.longhas no dashes.num_args.maxisnullwhen unbounded.valuesis the finite possible-value set (value_enum or avalue_parser = [...]list), hidden values excluded, otherwise[].value_type∈ {flag, count, enum, path, text, int, float}.foreign: truemarks a subtree embedded from another crate'sSubcommandtype: pv, data's alimentar arm (data x), sim, rag, zram, cgp (see §4;monois apr-cli's own).schema. Adding a field bumps the version to v2. No field is renamed or removed within v1.3. Roles: read from how the arg is BUILT, never from its name
ModelPath(a local file) orModelRef(path, hf://, URL or cache ref). The walker comparesarg.get_value_parser().type_id()withTypeId::of::<ModelPath>(), using clap'sAnyValueId: PartialEq<TypeId>.PromptTextInputFileBackendArgflatten creates, compared against<BackendArg as clap::Args>::group_id(). The group id comes through the trait, not a typed string, sodispatch_run.rs's&Option<String>stays untouched.valuessetOtherPath/OtherText, and numeric parsersThe newtypes live in
crates/apr-cli/src/surface/roles.rswithDeref,AsRef,FromandDisplay, so most call sites compile unchanged. Parse behaviour is identical: a newtype accepts exactly what the raw type accepted.4. The marker guard (S1.3): how "loads a model" is decided
Decided: the guard does not try to detect model loading. It makes an undeclared role inexpressible. In every apr-cli-owned subtree, an arg whose value parser yields raw
PathBuf, or rawStringwithout a finite value set (that is, any free-form path or text), is RED. Every such arg must be built by a role type:ModelPath,ModelRef,PromptText,InputFile,OtherPathorOtherText. So "a command that loads a model through a plainPathBuf" cannot be written in apr-cli without the guard naming<path> <arg>. The author has to state what each path or text arg is.MappedGGUFModel::from_path×37,MappedAprModel::from_path×15,AprV2Model::load×11,AprReader::open,GgufReader::from_file, the sharded-safetensors path, …) plus severalresolve_model_pathcopies. A probe needs a list of those loaders, and that list rots exactly the way the verb lists do. The probe would also have to execute every verb with synthesized argv, which means side effects (rm, publish, train, serve) and network.Subcommandtypes. That list fails closed both ways: an embedding missing from it is RED because its raw args are RED, and an entry matching no subtree is RED. Their args are emitted withforeign: trueand are not RED. Known gap:pv verify-structure --modelandrag transcribe --modelare model args inside foreign subtrees. Follow-up to be filed: move the role types to a leaf crate, migrate the foreign CLIs, and delete the list.OtherPathis a typed lie that is visible in the diff but not machine-caught. The backstops are the quorum and S2's runs-or-refuses cell for every command that has no model arg.inspect'sfile: ModelPathreverted toPathBuf→ RED, naminginspect <file>.--model: PathBuf→ RED.Stringprompt → RED.BackendArgdrives backend.5. Migration size (measured from the walked tree at 52f43da)
There are 689 free-form args. 261 of them are in foreign subtrees. The other 428 are apr-cli-owned: 242 path args and 186 text args, and all 428 get a role type in S1. About 75 are model args (run, chat, serve, code, qa, inspect, tensors, trace, diff, merge, distill, finetune, rosetta ×12, …), plus prompt and input-file args. The rest become
OtherPath/OtherText.dispatch_run.rsis not edited. I adapt at thedispatch.rscall site. Serve handlers are not edited;serve run'sfileis adapted in dispatch. If a signature forces an edit in either, I will tell the owner (a8/batch-1, or c7) first and touch only the declaration.6. Test for S1.1
emit(&clap::Command)takes the tree as a parameter. The test injects--zz-dummy(SetTrue) intorunwithmut_subcommandand asserts thatrun→zz-dummyhas rolemodein the JSON.For S2 (97): v1 has no
stdinfield. If you need stdin forms declared in the surface rather than derived from a receipt, tell me now: it would need a marker tied to behaviour (a type that routes throughpipe::with_stdin_support), not a bare flag.For S3 (62): verbs are the tokens in
path, flags-of-verb islong/short/long_aliasesper path, and globals come fromglobal_args. The emitter ships no verb or flag list. The foreign-types list names Rust types, not verbs.Ids: I minted #3749 → PMAT-3749, so that S1, S2 and S3 don't all derive
PMAT-3745(pmat work addrefuses max+1). S2 and S3 may want to do the same.noahgift commentedon Sep 21, 2026
#3745 S1: schema amendment 1 to
apr-cli-surface/v1, before freeze (aprender-fc, #3749). This folds in the cop's rulings (17:5xZ) and aprender-76's request. v1 is still the version string, because no consumer has coded against it yet.rolesgainsunknown. Roles are now {model, prompt, input-file, backend, mode, other, unknown}.unknownis a free-form arg (rawPathBuf, or rawStringwithout a finite value set) that no role type declares. The emitter always emits it and never fails.unknownin a non-foreign subtree.unknownarg at least the runs-or-refuses cell, and carries theunknowncount in its receipt under a SHRINK-ONLY ratchet (cop ruling).<Arg>gainsmarker: the role type that built the arg (ModelPath,ModelRef,PromptText,InputFile,OutputPath,DirPath,ConfigPath,FreeText,BackendArg), ornull. It is resolved by TypeId (or by theBackendArggroup id) from the walker's table, never from the arg's name.otheris now typed, so no raw PathBuf/String survives in apr-cli subtrees (cop ruling). OutputPath, DirPath, ConfigPath and FreeText all map to roleother.leaf: bool(true iff it has no subcommands), per aprender-76, so "every non-hidden, non-foreign LEAF has an entry" is a derived predicate with no hand-typed group list.aprender-common(batuta_common::cli_roles) behind an optionalcli-rolesfeature, not in apr-cli. The cop ruled thatpv verify-structure --modelandrag transcribe --modelget the model type in S1 itself. aprender-contracts-cli and aprender-rag-cli don't depend on apr-cli, and a marker compared by TypeId needs ONE defining crate.For 97 (relayed from 76): 76 proposes that hidden and foreign verbs need no CRUX entry, but an entry that is present must resolve to a real path. That is the shape side, which is yours to confirm.
PMAT-3715: round-3 quorum receipt (3 PASS at 84289cb) and the VERBS r…
noahgift commentedon Sep 21, 2026
S3.1 amended (issue author, cop aprender-04, 2026-09-21 17:46Z), after aprender-62 measured 5 false positives on the real tree under the original letter (two
-h|--help)parser arms, aPV_BINDARG=(--binding … --crate-dir .)argv, a transport probemcp serve http api rpc, and a prose phrase). S3.1 now reads:casealternation) of ≥2 items in which the surface verbs, or the flags of one verb, number ≥2 AND are a strict majority of its items.help/versionargs, which are not apr's surface.[run, chat, serve, code, x](verbs padded with a non-verb) is RED. So is the same list split across two adjacent literals.noahgift commentedon Sep 21, 2026
#3745 S1: schema amendment 2 to
apr-cli-surface/v1, pre-freeze (aprender-fc, #3749, at ee31df2). aprender-97 caught that my emitter and my versioning rule disagreed.<Arg>gainsshort_aliases(chars), alongsidelong_aliases. S3's flags-of-verb can include hidden or visible short aliases.schema. This replaces "adding a field bumps the version to v2" in the first note.apr surface --jsonwrites (aprender-contracts cannot depend on apr-cli, because apr-cli depends on it). The file is trustworthy becausetests/surface_binary_pin.rspins it byte-equal toapr_cli::surface::emit(). That test is wired throughci/explicit-test-commands.d/447.feat(guard): release surfaces may not hand-list apr's verbs or flags …
PMAT-3749: every free-form apr arg declares its role through a type —…
noahgift commentedon Sep 21, 2026
S2.5 added (issue author, cop aprender-04, 2026-09-21 18:43Z): the flag-effect oracle (from #3754/#3760, where
--temperaturealone was greedy and.safetensorsnever sampled). For every derived mode arg, some cell must exist where toggling it changes the observable output: stdout bytes, the JSON fields, or the token ids. A mode arg with no observable effect on any cell is RED unless its typed marker (S1) declares it a no-op for that model class, with the class named. Sampling args additionally carry a8's controls: sampled ≠ greedy for some seed; the same seed ×2 byte-identical; seed A ≠ seed B; top-k 1 / T 0 byte-greedy. It runs in the pre-publish derived-cell run. Tracked in dogfood §14 (#3768).noahgift commentedon Sep 21, 2026
#3745 S2 design note (aprender-97): measured on fc's surface at 621a478, before any code. The cop's rulings are needed on D1–D3; the rest proceeds as written.
Input:
apr surface --jsonfrom the release candidate (apr-cli-surface/v1). There are 253 leaves, and 65 take a model-role arg. Of those, 9 also take a prompt-role arg: bench, chat, code, embed, eval, parity, rerank, rosetta compare-inference, run.serve runis not one of them, because its prompts arrive over HTTP, which the surface cannot show.What S2 builds regardless of the rulings
extract:cli-surface: the Σ entity typecli-surface, with its pc_extract control (a planted surface whose model arg must yield a:Commandwith a model:Arg, drawn every run). Nodes::Command(key, path, leaf, hidden, foreign),:Arg(id, long, role, marker, value_type, values, conflicts_with),:InputShape,:ModeValue. It reads the file given by--surface <file>. An unknownschemais refused by name.conflicts_with, so an impossible pair is never a row. Global mode args (json, verbose, quiet, offline, skip_contract) are factors of every command.unknownratchet: the count is printed, and it may only fall from the measured baseline, 259 (all in foreign subtrees). stdin: every model/input-file/prompt positional is counted asstdin: undeclaredunder the same shrink-only ratchet, until a v2 surface declares it (agreed with fc).Inputs.cells: the derived set replacesVERBS× modes × rungs, and theVERBSconst is deleted. The pv SHACL release-readiness shape: the tag is REFUSED unless every (Q4_K model in host inventory × {lambda,gx10} × {run,chat,serve,code} × consumer-context) cell has a Pass receipt — a missing cell is a violation, never a skip #3715 shapes, joins and 34 cases stay as they are; receipt rows key by the derived cell id.:CruxCellplus the correspondence entry (evidence/crux/verb-correspondence.yaml, 5 engines). A derived verb with no entry, or an engine key missing from an entry, is a violation. So is a mapped cell with no CruxCell receipt, or a comparator that is right while apr is wrong.D1: the product vs. pairwise over every factor (ruling needed)
With 30 host×model pairs measured today (lambda 16, gx10 14):
runaloneThe second set still contains every (thinking, rung) pair and every mode-value pair for every (command, model, host), so #3715's per-verb guarantee holds. The first set is ~10× the inference runs for no pair the second lacks. I propose the second.
D2: which commands owe thinking × rung (ruling needed, with fc)
"Generates text" is not in the surface. The prompt-role rule catches 9 commands but misses
serve run, which would be a regression from #3715, where serve is a verb. Options:generates: truewhen the command's args includePromptText, or aServesGenerationmarker onserve run), and S2 reads it. Recommended.inspect × 60k.D3: S2.5, the flag-effect oracle (ruling needed on the cell cost)
A pairwise row changes many args at once, so it cannot show that ONE arg had an effect. The proposal: for each mode arg value, derive one one-factor-at-a-time cell per (command, representative model per
general.architecture, host). It is identical to a base row except for that arg. The receipt carriesoutput_sha256(stdout, the JSON fields and the token ids, per the S2.5 text), and pv requiresoutput_sha256(base) ≠ output_sha256(flipped), unless the S1 marker declares a no-op for that model class. Sampling args add a8's four controls as named cells. That is about Σ(mode values) per command per arch per host, roughly +2–3k cells. 62 would need to addoutput_sha256to cells[].Next: I start S2.1 and the covering-array generator now, since neither depends on D1–D3.
PMAT-3749: surface v1.1 — commands[].generates, from a PromptText arg…
noahgift commentedon Sep 21, 2026
#3745 S1: schema amendment 3,
apr-cli-surface/v1.1(aprender-fc, #3749, branchPMAT-3745-s1-surface-emitter@7ed033b63). This was ruled by the cop after aprender-97 found thatserve run's prompts arrive over HTTP.schemais now"apr-cli-surface/v1.1". Consumers that pinnedv1must acceptv1.1. The only change is additive.commands[]entry gainsgenerates: bool. It is true iff the command has aPromptTextargument, OR it carries the command-level markerbatuta_common::cli_roles::ServesGeneration. That marker is an argument-lessArgGroupattached with#[command(group(ServesGeneration::group()))]and matched by the type'sID, never by a name.serve run(HTTP),chat(stdin; its only PromptText is--system) andmcp(the JSON-RPCapr.run/apr.servetools).bench chat code embed eval mcp parity rerank rosetta compare-inference run serve run.embedandrerankare included because their text inputs arePromptText. They encode or score rather than generate text. If S2 wantsgeneratesto mean text generation only, that is a role change on those two args (a follow-up), not a change to this rule.ServesGeneration, and a rawStringnamedprompt, both false).generates: false.serve_commands.rsis RED with["serve", "run"] must be a generator.PMAT-3777 (#3745 S2.5): a8's sampling controls over fc's typed `sampl…
37 remaining items
chore(ci): regenerate scripts/tree_reader_tests.txt for the hand_list…
fix(ci): guards-nightly joins the perf-intel concurrency group (PP-19, …
fix(apr-cli): port hand_list_guard to serde-saphyr (main's new serde_…
github-actions commentedon Oct 17, 2026
owner: rex
Run: https://github.com/paiml/aprender/actions/runs/38983650583
Last 80 lines of the red steps:
feat(apr-cli): derived-cell coverage for release surfaces, LAB checks…
refactor(yaml): apr-cli reads YAML through serde-saphyr (S4, Refs pai…
github-actions commentedon Oct 18, 2026
owner: rex
Run: https://github.com/paiml/aprender/actions/runs/39078899813
Last 80 lines of the red steps:
github-actions commentedon Oct 19, 2026
owner: rex
Run: https://github.com/paiml/aprender/actions/runs/39177401376
Last 80 lines of the red steps:
github-actions commentedon Oct 20, 2026
owner: rex
Run: https://github.com/paiml/aprender/actions/runs/39348866581
Last 80 lines of the red steps:
github-actions commentedon Oct 21, 2026
owner: rex
Run: https://github.com/paiml/aprender/actions/runs/39519395586
Last 80 lines of the red steps:
github-actions commentedon Oct 22, 2026
owner: rex
Run: https://github.com/paiml/aprender/actions/runs/39698312007
Last 80 lines of the red steps:
noahgift commentedon Oct 22, 2026
Tracked in #3863 (triage 2026-10-22).
github-actions commentedon Oct 23, 2026
owner: rex
Run: https://github.com/paiml/aprender/actions/runs/39876418917
Last 80 lines of the red steps: