[noahgift]

CRUX: the verb correspondence, and the judge + driver side of the hf and llamafile engines

Closes #3774. Stacked on #3767 (the CRUX run slice of #3739), so this diff is crux-3739-inference-dogfood...HEAD.

keep-open: #3739 stays open (chat/serve/code cells, thinking ON, consumer rungs, T-1 wiring). #3782 is the Rust apr qa copy of a hole this PR closes on the CRUX side. #3715/#3745/#3726/#3672/#3602 are cited as context.

evidence/crux/verb-correspondence.yaml

The file is derived from apr surface --json (apr-cli-surface/v1.1, sha256 207c3297…, #3745 S1). It has one entry per leaf (253), and every entry states all four engines (llama.cpp, ollama, hf, llamafile), each as a mapping or {none: <reason>}. That's aprender-97's per-engine explicitness: a missing key is a shape violation.

• 16 verbs map at least one engine: run, chat, serve run, code, bench, quantize, pull, list, rm, tokenize encode-corpus, eval, inspect, tensors, devices, modelfile parse, import.
• 237 are none, each with a reason. Every generating leaf that maps nothing has a specific reason.
• Measured at authoring:
  • 7/7 llama.cpp binaries and 10/10 flags exist in the pinned build (gx10, d1d3c3396);
  • 7/7 ollama subcommands plus create --quantize exist in 0.33.2;
  • every mapped apr flag is in its verb's surface args;
  • the negative controls (--frobnicate, bin llama-frobnicate, ollama frobnicate) FAIL.

evidence/crux/hf-sources.yaml

It maps each GGUF (by sha256) to the HF source weights at a pinned revision. The one entry, qwen2.5-coder-1.5b Q4_K_M, has its provenance checked through the HF API: the repo's LFS sha256 equals the file's.

Judge (scripts/lib/crux_inference_judge.py), per row contract v1 (#3739 issuecomment-5765991210)

• hf and llamafile are comparators, so hf or llamafile right with apr wrong is RED. Output that isn't the contract's JSON, or JSON without text, is no answer.
• tok (raw-text ids) and tmpl (chat rendering) are byte-equal or RED, with the first differing index or byte named. An engine row with no apr row beside it is RED, and a reference-refused row declines.
• greedy is REPORTED: the first divergence, plus the logit cosine there, read from .npy with the stdlib.
• A degenerate answer is no answer, from any engine: 8 or more non-space characters, 90% or more of them one character.
• Plugin engines are held to their lane: an hf/llamafile row must report device, and it must match its lane. On the integration run, HF reached the 4090 on the cpu lane, outside gpu-q; infra-3c fixed their side in 9c806f0.
• Every cell carries positive_control (additive, for pv's shape, pv SHACL release-readiness shape: the tag is REFUSED unless every (Q4_K model in host inventory × {lambda,gx10} × {run,chat,serve,code} × consumer-context) cell has a Pass receipt — a missing cell is a violation, never a skip #3715).

Driver (scripts/crux_inference_dogfood.sh)

• Each cell (one prompt, every engine) runs as ONE gpu-q command (GPUQ_WAIT=<bound> gpu-q --prio ${CRUX_GPU_PRIO:-3} -- bash <cell>; the train's T-1 run passes 1), falling back to a bounded flock recorded as "may have jumped the queue". ollama's unload wait runs inside the cell, and exit 75 makes every engine of that cell a refused row.
• Plugin engines come from scripts/crux_engine_<hf|llamafile>.{sh,py} (infra-3c, PMAT-3778). probe is recorded verbatim, and gen is called with the pin's sampling; llamafile uses --interface server, hf gets --source-repo/--source-revision/--dtype. A missing script, a failing probe, or a gen that appends no row is a refused row naming why.

What the integration run found

It ran on lambda's cpu lane with infra-3c's real drivers (copied in, not committed):

• The plumbing works end to end. The hf probe was recorded verbatim (transformers 5.17.0, torch 2.14.0+cu130), and llamafile was refused by name because it isn't installed yet (infra#906).
• HF answered ! × 64 on every prompt. transformers left lm_head untied; I reported it to infra-3c.
• The golden greeting pattern "!" scored that garbage correct. This PR adds the degenerate rule, and apr qa golden_output: the greeting case's "!" pattern scores degenerate output ("!!!!", token id 0) as correct #3782 files the same hole in apr qa.

Verification

• bash scripts/check_crux_inference_judge.sh: 38 ok, 0 broke. A mutant of each new rule turns it RED; the counts are in the fragment.
• bash scripts/guard_tree.sh --no-cargo: 77 checks, 0 failed on a330f0104. The follow-up commit only changes the gpu-q priority default (5 → 3), and the shell-lint ratchet, pipe-into-grep-q, competing-harness and case-table checks were re-run on it.
• check_no_competing_harnesses.sh: count=0. The driver reads no clock.

🤖 Generated with Claude Code

[noahgift]

Every commit is already in release/0.69.1-batch-2 (git cherry: 0 unfolded). It reaches main through the release batch PR #3835. Closing as folded, not rejected.