Skip to content

apr: llama.cpp logit-compare verb as the PRM parity.oracle (PRM-001 v2 §4.1) #4444

Open

Description

@noahgift

Operator ruling, 2026-09-25 (PRM-001 v2, item 3a): add an apr verb that compares logits against llama.cpp@d1d3c3396, to serve as the declared parity.oracle in rex-cell-admission-v1.

Today apr parity compares GPU vs CPU only. The REX-04 receipt found that.

Acceptance:

  • The verb exists and emits a comparison receipt, per position, against llama.cpp logits.
  • A planted-divergence fixture turns it RED.
  • It can be declared as parity.oracle in a cell.

Until it is declared, cells use apr-parity-gpu-cpu plus the H1 verdict check.

Parent: #4354 (PRM-001).

Activity

  1. added this to the 0.70.1 milestone on Sep 25, 2026
  2. noahgift commented on Sep 25, 2026

    @noahgift
    MemberAuthor

    Branch feat/4444-llamacpp-logit-oracle @ 300ec86e4, pushed only (fold owner: 59).

    • The verb exists: apr parity-oracle --reference <llama.cpp APRRAWLG> --subject <apr APRRAWLG> --threshold F --threshold-basis TEXT -o receipt.json. It writes a per-position receipt (cosine, argmax ref/subject, max |diff|). With --json it prints the rex-cell-admission-v1 Parity block: {oracle: "llama.cpp@d1d3c3396", cosine, threshold, threshold_basis, receipt_sha256}. aprender-84 confirmed the field set.
    • The planted divergence turns it RED: reversing one row makes it exit 13 and name that position. A mutation that neutered the comparison was killed (2 of 11 tests failed). Token-id, shape or non-finite mismatches are refused (exit 4) before any receipt is written.
    • Real data: on the d1d3c3396-vs-apr qwen35 pair (sha 1d5eaf12… / f6f79264…), the verb reproduces compare_raw_logits.py exactly: min cosine 0.995905 at pos 4, argmax mismatches [2, 28, 36, 73]. See evidence/parity/oracle-4444/CROSSCHECK.md.
    • Still open before a cell can be admitted on it: the qwen3.5-0.8b@cpu-vs-llama.cpp row in thresholds.yaml has no measured basis. The verb requires --threshold and --threshold-basis and supplies no default. Also, apr has no shipped producer for the subject APRRAWLG yet; the pair above came from the feat(qwen35): Qwen3.5 / Qwen3.8 hybrid GGUFs run on the CPU — apr run + apr chat (#3091) #3114-era example.
  3. noahgift commented on Sep 25, 2026

    @noahgift
    MemberAuthor

    Follow-up landed on branch feat/4444-llamacpp-logit-oracle @03579caa9, for 59's fold. No new PR.

    apr parity-oracle --model <qwen3.5.gguf> [--subject-out S] now produces the subject in-process. It runs realizar's Qwen3.5 CPU forward (forward_single_qwen35, one token per call) over the reference's own token ids, so the ids agree by construction. The receipt's subject.producer records the forward and the model's sha256.

    Measured on lambda with the model file from REFERENCE.md (bd258782…) and reference 1d5eaf12… (llama.cpp d1d3c3396):

    • The produced subject is byte-identical to the intel harness's apr-intel-run1.bin (f6f79264…).
    • min cos 0.995905 at pos 4, argmax mismatches [2,28,36,73]: GREEN at 0.99.
    • The strict control at 0.9999 exits 13.
    • 7.3 s wall.

    Checks:

    • 16/16 unit tests and cli_commands 15/15 pass. Clippy on apr-cli is clean, and fmt is clean.
    • A mutant that drops the vocab guard is killed by the out-of-vocab case.

    Still open: the qwen3.5-0.8b@cpu-vs-llama.cpp row in thresholds.yaml is unmeasured. The 0.99 above is a crosscheck value, not a basis.

  4. modified the milestones: 0.70.1, 0.71.0 on Sep 25, 2026
  5. added
    P0Critical priority
    owner:la-0.71Owned by the 0.71 look-ahead slot
    on Oct 4, 2026
  6. added
    P1High priority
    and removed
    P0Critical priority
    on Oct 14, 2026
  7. noahgift commented on Oct 14, 2026

    @noahgift
    MemberAuthor

    P0 to P1 (2026-10-14 P0 triage): P0 now means it blocks the next release or carries the operator's own words on it; this issue does neither, so it stays on the 0.71.0 milestone at P1.

  8. noahgift commented on Oct 14, 2026

    @noahgift
    MemberAuthor

    merged into #5317 (master ticket for epic #4381; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.

  9. modified the milestones: 0.71.0, 0.73.0 on Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priorityowner:la-0.71Owned by the 0.71 look-ahead slotsize:MK̂ 3-4 h

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions