[noahgift]

CRUX inference dogfood, slice 1: the run verb, apr vs llama.cpp@pin vs ollama on the same GGUF

Refs #3739. This is slice 1 of the issue's plan (design note: #3739 (comment), addendum: #3739 (comment)).

keep-open: #3739 has six done_when items and the 17:19Z scope amendment. This PR is only the run verb, the golden prompts, thinking OFF and lambda, so the issue stays open for chat/serve/code, thinking ON, the consumer rungs, the ollama upgrade, gx10, the correspondence file and the T-1 wiring. The other issues cited here are context, not closed by this PR.

What it adds

• scripts/crux_inference_dogfood.sh <version> --model <gguf>… [--backend gpu|cpu] is the driver.
  • It pins apr with apr_bin.sh and declines unless apr --version reports <version>.
  • It resolves llama.cpp only through llama_bin.sh, and ollama by its declared path, recording the server's /api/version.
  • Temperature, seed and context come from llama_pin.toml [protocol].
  • ollama imports the exact file (FROM <path>), and the receipt records the input sha256 and ollama's stored blob digest.
  • On the GPU lane, one prompt takes /tmp/apr-gpu.lock for all of its engine calls (fd 9, closed in every engine). ollama runs with --keepalive 0 and must leave VRAM before the lock drops. Every engine runs under choom -n 1000.
  • It reads no clock and computes no rate (check_no_competing_harnesses.sh stays at count=0).
• scripts/lib/crux_inference_judge.py is the judge. It reads raw outputs only, and writes evidence/crux/<version>/<host>-<backend>.{json,md}.
  • The rule: a cell is RED when llama.cpp or ollama is correct and apr isn't. That includes apr exiting non-zero, falling back off the lane's backend, or having no row at all.
  • An UNJUDGED cell (no comparator answered) declines the run, per the amendment's "a missing cell is a NO-GO".
  • Every engine wrong (ALL_WRONG) is named and counted per model, never a pass. Each model must have a measured positive-control prompt (golden-2plus2); if the control is ALL_WRONG, missing or undeclared, the run declines as a broken harness. These are the cop's rulings.
  • Token counts and rates are copied as each engine reported them, and never judged.
• scripts/check_crux_inference_judge.sh is a hermetic case table of 21 rows that guard_tree runs bare.
  • It includes the pv SHACL release-readiness shape: the tag is REFUSED unless every (Q4_K model in host inventory × {lambda,gx10} × {run,chat,serve,code} × consumer-context) cell has a Pass receipt — a missing cell is a violation, never a skip #3715 falsifier (llama.cpp right, apr wrong → RED) and a drift check against golden_output.rs::golden_test_cases.
  • 11 judge mutants and 1 prompt-set mutant each turn it red.
• Also: scripts/crux_inference_prompts.json, and the fragment docs/roadmaps/entries/PMAT-3739.yaml.

Measured (lambda RTX 4090, qwen2.5-coder-1.5b Q4_K_M sha256 cc324af0…)

Measured by apr 0.69.0 and the harness at afb05a3de. Receipts: evidence/crux/0.69.0/lambda-{cpu,gpu}.{json,md}.

lane	verdict	cells
cpu	PASS	3 GREEN: apr and llama.cpp@d1d3c3396 both correct
gpu	RED (rc 1)	2 RED, 1 GREEN: on golden-2plus2 (the positive control) and golden-paris, apr run --gpu exits 14 (CUDA diverges from CPU, cosine 0.8795; wgpu rejected; CPU fallback refused) while llama.cpp answers on the same RTX 4090. This is the open #3602, seen from outside; the findings are named on #3739.

The harness is supposed to go RED on the GPU lane today. It is merged as a measuring instrument, and the defect it names is #3602's to fix.

Findings this harness surfaces (it doesn't fix them)

• apr applies the chat template twice (apr run applies the chat template TWICE: the prompt reaches the model as an escaped user turn inside a user turn (52 tokens vs llama.cpp's 21; U+200B encodes as three token-0s) #3672). Every cell shows 46 prompt ids against llama.cpp's 36 for "What is 2+2?", first diverging at id 1, and the U+200B escapes encode as id 0 (apr's byte-level BPE tokenizer is non-canonical: every non-ASCII byte → id 0, ASCII segmented unlike llama.cpp (2 of 10 ids agree on a path) — corrupts copy tasks and every non-ASCII user; apr parity is blind (shared tokenizer) #3726). The golden answer is still right, so only the token-parity field catches it.
• ollama 0.5.7 on lambda imports a GGUF with TEMPLATE {{ .Prompt }}, meaning no chat template. Its cells are refused by name rather than run as a raw completion. gx10's 0.33.2 imports the GGUF's own template. The lambda upgrade through forjar is slice 2.

Verification

• bash scripts/check_crux_inference_judge.sh: 21 ok, 0 broke.
• bash scripts/guard_tree.sh --no-cargo: 77 checks, 0 failed (on c77c1f45f).
• check_no_competing_harnesses.sh: count=0. check_no_pipe_into_grep_q.sh: rc 0. Roadmap aggregate --check, fragment-required, sorted and additive: all rc 0.

🤖 Generated with Claude Code

[github-actions]

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=3767 head=c77c1f45f7ab8f0cd30fb704d6cd98e4d5e260a5 verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

[noahgift]

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3739",
 "head": "c77c1f45f7ab8f0cd30fb704d6cd98e4d5e260a5",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 15
  },
  {
   "lane": 2,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 0
  }
 ]
}

[noahgift]

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3739",
 "head": "c77c1f45f7ab8f0cd30fb704d6cd98e4d5e260a5",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 11
  },
  {
   "lane": 2,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "NO-VERDICT",
   "findings": 0
  }
 ]
}

[noahgift]

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3739",
 "head": "c77c1f45f7ab8f0cd30fb704d6cd98e4d5e260a5",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 2,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "NO-VERDICT",
   "findings": 0
  }
 ]
}

[noahgift]

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3739",
 "head": "c77c1f45f7ab8f0cd30fb704d6cd98e4d5e260a5",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 2,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "NO-VERDICT",
   "findings": 0
  }
 ]
}

[noahgift]

Every commit is already in release/0.69.1-batch-2 (git cherry: 0 unfolded). It reaches main through the release batch PR #3835. Closing as folded, not rejected.