EPIC E5 0.73: Train What You Serve  #4000

owner: aprender-w5382 (set 2026-10-19 by aprender-triage-worker at the cop's word; picks up children only as batch constituents after its current timed task)

Spec: APR-EPIC-001 §2 · Epic: E5 · Train: 0.72 · Successor at cut: E6

Theme

Train What You Serve (was: Agent Ready; operator ruling 2026-09-27, APR-LOOKAHEAD-001 §2a)

done_when

Current exit criteria (APR-LOOKAHEAD-001 §2a): T1 apr finetune (QLoRA), apr distill, apr merge, apr quantize complete end to end on Qwen 3.5 with 0 refusals (the qwen35 refusals are removed) · T2 fine-tune throughput ≥ 0.8× Unsloth on the same GPU and model, with a committed measurement command · T3 Prometheus B2: one challenger (local Qwen3.5-27B teacher → 4B student, gold labels only, 0 sealed-test hashes in training data) judged by PRM-001 §5.4 · T4 first improved dogfood model published to Hugging Face as an rc with receipts (EXT-001) · T5 trained weights round-trip gguf ↔ safetensors ↔ .apr at cosine ≥ 0.98

Rows from the old theme that do not fit move to milestone backlog (cop 2026-09-27).

Superseded done_when (old theme): /v1/chat/completions and /v1/messages pass one conformance suite; tool calls; --json-schema; used_gpu provenance on every response.

Budget

At most 5 open children (docs/roadmaps/epics.yaml). Repo-wide cap 100 open issues; andon at 95.

Absorbs

—

Prior body

Theme: Agent Ready (operator 2026-09-23): one gateway serves all agents

apr serve is the single gateway every agent talks to: OpenAI-compatible (/v1/chat/completions: tools, streaming, seeds) plus an Anthropic Messages adapter (/v1/messages), which the operator approved ("yes to 1"). It is one per host and pinned by binary sha. Fleet consumers (quorum local lane, arbiter ask/decide, cookbook recipes, external harnesses) use ONE client contract, never CLI invocations. CRUX measurement uses a dedicated instance, never the shared gateway.

Exit bar. One conformance suite (OpenAI + Anthropic shapes, tool calls, streaming finish/[DONE], seeded reproducibility) passes on every model × backend that 0.71 certifies. An agent harness (Claude Code via /v1/messages; Alfredo's via OpenAI) completes a tool-calling task end to end against it.

Must-carry

Open decision

• Consumer migration (infra quorum lane, arbiter, paiml-implement) in 0.72, or aprender ships the gateway and conformance suite and the other repos migrate after?

Members not listed above (62, every other open 0.72.0 issue)

Rule (operator 2026-09-24, applied by the cop aprender-cf): a milestone is an epic only if this issue lists its members, and tickets without an epic go to backlog. This list was derived from the milestone's open issues on 2026-09-24 by aprender-dd (#4159).

• apr run: 7 defects incl. --stream emitting empty text and --trace-output always writing an empty event list #2378
• R13: apr serve has three different HTTP surfaces and which one you get depends on the format of the file you passed #2507
• Epic: apr-dogfood v3.0 — coverage is the organizing gate (17.1% of 830 features, 27 of 28 binaries at zero) #2582
• POST /api/generate and POST /generate enforce opposite unknown-field policies in one process #2610
• The README contract count is a serialization point: any two contract-adding PRs collide by construction (4 occurrences today) #2630
• apr gpu reports unified memory on GB10 but not on Apple silicon, and steers macOS users to --device cpu #2661
• RCPT-1: a NO-GO receipt cannot explain its own RED — the note is the first error-ish line anywhere, and WORKLOG is trap-deleted #2663
• Six untracked audit findings from #2644: two shim-check universe gaps and four run-idempotency defects #2664
• perf(serve): prefill is 3.64x behind llama.cpp — 2,860 vs 10,399 tok/s, same GGUF, same GPU #2693
• F-PROVE-007 asserts 297 compile-time proofs against a tree that generates 578, and never executes #2732
• clippy reaches lib targets only: 1,782 tests/examples/benches/bins across 78 members are outside every lint gate #2751
• apr run --gpu on the 7B silently falls back to CPU after the F2 parity gate rejects at cosine 0.9278: any tok/s from that path is a CPU number with a GPU label #2790
• Merge-queue runs are cancelled by their own concurrency key, and dequeuing does not stop them — ~101 wasted runner-minutes measured #2812
• apr mis-tokenizes <|im_end|> after sentence punctuation: 4 junk tokens are fed to the model instead of the turn delimiter #2815
• BLOCKER: no W1 cell is producible on the comparator lane in either mode — streaming reports prompt_tokens=0, non-streaming reports 534 against a 512 ± 8 band #2817
• CI jobs fail with cargo exit 101: target/debug/deps files vanish MID-BUILD (ENOENT, not ENOSPC — 884G free) #2822
• workspace-test and guard-runner-labels share one cargo target dir concurrently with different feature sets — observed producing the ENOENT signature #2828
• arm_a_scaling emits nothing on a c=1-only receipt and the gate still returns VERDICT PASS #2830
• PARITY: the 10.34x gap was measured on a build with batching compiled out — re-baseline first #2844
• SafeTensors serve route drops ignore_eos and seed: apr-cli's ChatCompletionRequest has no such fields #2853
• EPIC S2 standing: Provable Contracts (PVL-001) #2870
• Class: a gate that answers empty or degenerate input with PASS (or a traceback) instead of a named refusal #2876
• Class: ratchets that block edits rather than regressions — frozen files, line-keyed baselines and ledgers where honest work is the most expensive path #2879
• W-F-attach: apr run attaches to a running apr serve; W5 resident arm #2944
• CUDA Q5_K gemv/gemm read a non-ggml block layout (qh bit order; gemm also qh/qs offsets and scale unpacking) #3111
• gpu-quick on gx10 (GB10 unified memory): test_alloc_oversize_100gb asserts discrete-GPU refusal — the only red in 2604 cuda tests #3129
• aprender-train-canary is dark: workspace-excluded, in no CI job, pinned to burn 0.21.0-pre.2 — there is no measured trueno-vs-Burn number #3174
• P0 triage: 30 remote branches carry unmerged content and never had a PR — five families, not thirty items #3252
• gpu-quick: aprender-gpu test binary SIGSEGVs on gx10 on identical code — passes 14:19, crashes 15:13 #3328
• qwen3_moe_argmax_parity has been broken at every llama.cpp pin — llama-cli never accepted -no-cnv, and both obvious fixes return an empty string #3332
• P1: Multi-layer stack ensembling with a leakage guard (num_stack_levels, auto_stack, dynamic_stacking) — CRUX-O-07 #3377
• apr run: the --no-gpu arm prints no Backend: line, so the CPU arm cannot be proved (post-publish 0.68.2) #3541
• prefix_cache_hits is counted and unit-tested but exposed nowhere: /metrics has no prefix/cache counter, so a prefill measurement cannot exclude cache hits #3553
• ARB-APR-3: parity receipt for the decide lane's cell on the agent's host (Qwen3.5-9B-Q4_K_M @ lambda-labs) #3574
• 0.69.1 CONSUMER REQUIREMENTS: every project that uses apr states its user-interface needs (verbs/API, models, context, output shape, latency) — the release gate is derived from these answers #3716
• apr code on Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs through apr serve --gpu (#3571) #3719
• apr response contract for consumers (#3716): weights digest + seed determinism, empty completion = error, refused vs did-not-run distinguishable — on run --json and serve #3720
• apr inspect --json drops every qwen35.* metadata key (context_length 262144, block_count, full_attention_interval…): 46 header KVs, 24 shown #3733
• Replace or remove crates/aprender-serve/src/grammar once #3568's TokenConstraint lands — an unwired, char-level grammar engine beside the real one #3734
• Schema-constrained decoding with a thinking template: constrain only after </think> (0.70; 0.69.1 refuses SchemaWithThinking) #3735
• #3568 PR 1/4: the TokenConstraint hook and the llguidance engine behind it — no generation loop wired #3747
• apr code --emit-trace never records a tool call (4 records, one text block; code.rs calls it 'M29+') — the trace cannot evidence agent behaviour #3776
• #3568 PR 2/4: apr run --json-schema / --grammar on the CPU paths (dense GGUF + Qwen3.5) — the named refusals, a second-reader validator, finish_reason + prompt_tokens #3793
• P0: agy budget exhausted — wire aprender's own models as quorum review lanes (prah lanes local), and measure whether they can actually judge our diffs #3819
• apr run --format json reports "fell_back": false on a run whose own stderr says "attempting fallback" #3826
• CRUX slice (#3739): the code verb — apr code vs Claude Code (and agy), judged on an edit-and-verify task, not on token parity #3830
• 0.70.0: apr serve outlives its launcher and keeps the port — a leaked listener on a shared box (found via #3828) #3838
• apr run --chat is INERT: the chat template is applied whether or not the flag is passed, and there is no way to send a prompt raw #3848
• model_ladder.sh: serve_rc is a dead variable — every serve route can 503/500 and the row is still green #3886
• apr serve reports HTTP 200 for a forced-accelerator request that silently fell back to CPU — it has no after_generation equivalent #3889
• P0: chat/code judged by rc and serve by http — a verb emitting 'zombie zombie zombie' is recorded as working, and a GREEN row proves it #3921
• apr chat --json emits the backend, not the reply — the transcript is the only place the model's text exists #3927
• aprender-mcp: a generic McpServer over a caller-supplied ToolSet, apr tools behind a default feature (blocks paiml/infra#917) #3954
• False provenance: chat banner claims 'fused Q4K, F2-validated' on BF16 with F2 SKIPPED; chat envelope requested:"default" under --gpu #3955
• Batch serving F2 GPU validation has never run (empty probe) — needs a representative probe at batch-server init #3980
• apr serve advertises dead GGUF generation routes: /v1/batch/completions and /stream|/realize/generate 503; /generate, /batch/generate and /realize/batch echo the prompt #3991
• apr run labels a mixed UD-IQ2_XXS model 'quant=Q5_K' — it prints the tied lm_head's qtype, not the body's #4006
• apr serve picks the chat template from the CLIENT's model string — an unknown name gets a plain prompt and the reply runs into a fabricated Human: turn #4007
• CRUX vLLM driver: thinking-ON serve rows refused HTTP 400 — max_model_len = context (4096) < prompt + ON budget (4096) #4029
• binary-debt #6: aprender-train-* → apr (distill / finetune --plan / inspect / bench); delete train-shell and the excluded train-canary #4063
• apr serve (dense GGUF route) reports no used_gpu on /v1/completions or /v1/chat/completions — no provenance for which backend ran #4146
• clippy debt ratchet 3/5 (0.72): qa-runner, second half (~559) #4155

Parked

Closed over budget (APR-EPIC-001 rule 5). Reopen = pull (rule 7).

• Schema-constrained decoding with a thinking template: constrain only after </think> (0.70; 0.69.1 refuses SchemaWithThinking) #3735 Schema-constrained decoding with a thinking template: constrain only after </thi — over E5 budget (rank 6/14)
• apr code --emit-trace never records a tool call (4 records, one text block; code.rs calls it 'M29+') — the trace cannot evidence agent behaviour #3776 apr code --emit-trace never records a tool call (4 records, one text block; code — over E5 budget (rank 12/14)
• #3568 PR 2/4: apr run --json-schema / --grammar on the CPU paths (dense GGUF + Qwen3.5) — the named refusals, a second-reader validator, finish_reason + prompt_tokens #3793 Schema-constrained decoding in apr (--json-schema or grammar) — three customers: arbiter's quorum lane, tool calling in verb set V, and the ruchy language #3568 PR 2/4: apr run --json-schema / --grammar on the CPU paths (dense GGUF + Q — over E5 budget (rank 10/14)
• 0.70.0: apr code has no backend selector — it spawns its own apr serve, so "code on cpu" vs "code on cuda" is not expressible #3845 0.70.0: apr code has no backend selector — it spawns its own apr serve, so " — over E5 budget (rank 7/14)
• aprender-mcp: a generic McpServer over a caller-supplied ToolSet, apr tools behind a default feature (blocks paiml/infra#917) #3954 aprender-mcp: a generic McpServer over a caller-supplied ToolSet, apr tools behi — over E5 budget (rank 11/14)
• apr run: payload trace / --logprobs must record per-step top-k logits (no CLI path today; found on #4004) #4026 apr run: payload trace / --logprobs must record per-step top-k logits (no CLI pa — over E5 budget (rank 8/14)
• Load-time CRUX certification status for every model (certified / not certified / known issue) #4031 Load-time CRUX certification status for every model (certified / not certified / — over E5 budget (rank 13/14)
• apr serve (dense GGUF route) reports no used_gpu on /v1/completions or /v1/chat/completions — no provenance for which backend ran #4146 apr serve (dense GGUF route) reports no used_gpu on /v1/completions or /v1/chat/ — over E5 budget (rank 14/14)
• apr serve --ollama-compat /api/chat drops client tools and tool_calls #4182 apr serve --ollama-compat /api/chat drops client tools and tool_calls — over E5 budget (rank 9/14)

0.7x schedule (release-schedule-07x, 2026-10-08)

Train 0.72.0: plan cut 2026-10-26, tag 2026-10-27 (plan dates, not measured). Report: docs/reports/0.7x-epic-reports.md.

Open children kept in 0.72.0 (owners are proposals; the cop assigns):

• TR-09: one trace schema (= APR-OBS OBS-17) (TRACE-001) #4564 · P2 · ticket · size M · proposed owner la-0.72
• TR-11: new CRUX contracts (define) (TRACE-001) #4566 · P2 · ticket · size M · proposed owner la-0.72
• TR-14: CRUX coverage for tracing (TRACE-001) #4570 · P2 · ticket · size M · proposed owner la-0.72
• T1-R1: aprender-train refuses hybrid/GDN (qwen3.5) arch until modelled; fix qwen3_5 head_dim override #4552 · P1 · ticket · size M · proposed owner la-0.72
• TR-06: golden gate made real (TRACE-001) #4561 · P2 · ticket · size M · proposed owner la-0.72
• TR-07: external serve witness (= APR-OBS OBS-16) (TRACE-001) #4562 · P2 · ticket · size M · proposed owner la-0.72
• TR-08: consumer registry (TRACE-001) #4563 · P2 · ticket · size M · proposed owner la-0.72
• TR-15: CRUX first records (TRACE-001) #4567 · P2 · ticket · size M · proposed owner la-0.72
• TR-13: K-08 unblocked (TRACE-001) #4568 · P2 · ticket · size M · proposed owner la-0.72
• TR-12: dogfood renacer in CRUX (TRACE-001) #4569 · P2 · ticket · size M · proposed owner la-0.72

Children tracked here (REDUCE-1022)

• Schema-constrained decoding in apr (--json-schema or grammar) — three customers: arbiter's quorum lane, tool calling in verb set V, and the ruchy language #3568 — Schema-constrained decoding in apr (--json-schema or grammar) — three customers: arbiter's quorum lane, tool calling in verb set V, and the ruchy language (open)
• apr response contract for consumers (#3716): weights digest + seed determinism, empty completion = error, refused vs did-not-run distinguishable — on run --json and serve #3720 — apr response contract for consumers (0.69.1 CONSUMER REQUIREMENTS: every project that uses apr states its user-interface needs (verbs/API, models, context, output shape, latency) — the release gate is derived from these answers #3716): weights digest + seed determinism, empty completion = error, refused vs did-not-run distinguishable — on run --json and serve (open)
• aprender-train: InstructPipeline::from_pretrained only warns when the adapter cannot be read, then trains fresh adapters #4698 — aprender-train: InstructPipeline::from_pretrained only warns when the adapter cannot be read, then trains fresh adapters (open)
• TRACE-001: one trace schema from training to serving, with CRUX records #5314 — TRACE-001: one trace schema from training to serving, with CRUX records (closed: not planned)
• apr finetune and LoRA adapters: correct loading, merging and CUDA training #5341 — apr finetune and LoRA adapters: correct loading, merging and CUDA training (closed: not planned)
• apr code eval: task suite, sandbox, metrics and nightly receipts (APR-CODE-EVAL-001) #5342 — apr code eval: task suite, sandbox, metrics and nightly receipts (APR-CODE-EVAL-001) (closed: not planned)
• LAB: kfocus-shadow-lab.yml — dispatch-only x86-main shadow on kfocus #5889 — LAB: kfocus-shadow-lab.yml — dispatch-only x86-main shadow on kfocus (closed: not planned)