Skip to content

PP-QUANT-001 M2: reconciliation gate — GgmlType vs the vendored ggml.h id list at a pinned sha; exhaustive matches, no _ => #3431

Closed as not planned

Description

@noahgift

Parent: #3421 · Slot: 0.69.0 must-carry, after M1.
Mechanism: nothing compares the tree to the spec, so the enum lags silently (NVFP4=40, Q1_0=41, Q2_0=42 appeared upstream between 2026-01 and 2026-08-29 with no signal here).
Fix: vendored id list [X] with source sha in the header; a test reconciling it to GgmlType both ways; clippy::wildcard_enum_match_arm denied where GgmlType is matched.
Falsifier — three mutations, each RED and each naming its own gate: delete a TRAITS row · add a fake id to the vendored list · add _ => to a GgmlType match.

Ĵ: 24 turns [A] — borrowed; estimate.sh aprender pools 0 of 47 ledger rows today, so there is no measured basis.
Plan + evidence: #3427 · design discussion: #3418 · spec PR: #3420

🤖 Generated with Claude Code


First task (operator decision 2026-09-17): classify the 558 bare-integer qtype lines (regex upper bound, 10+ crates, [U]) into qtype | not-qtype, with unknown = 0 or listed, and ship the pattern with a must-match / must-not-match case table. The reconciliation gate needs that classification as its baseline; only 2 bare-integer sites inside the 33-file union are classified today (PMAT-3427 receipt, Table G).

Activity

  1. added this to the 0.69.0 milestone on Sep 17, 2026
  2. modified the milestones: 0.69.0, 0.70.0 on Sep 20, 2026
  3. noahgift commented on Sep 20, 2026

    @noahgift
    MemberAuthor

    RULING (operator, 2026-09-20) — PMAT-3430 M1

    Recorded verbatim. Lane A is unblocked; no fifth quorum round.

    1. Settled, no further quorum: target (one #[repr(u32)] GgmlType, 43 rows: 35 live + 8 Removed), home crate aprender-quant, Table E, X1 ordering. Undisputed in rounds 1-4; they are closed.

    2. Oracle: upstream ggml at the pinned llama.cpp sha (3173a5647) is the reference [X]. In-tree enums are the defendant, never the reference. Where they disagree, in-tree is a defect.

    3. Falsifier, re-worded. Old wording (agreement with the existing core/compute/serve values) is unsatisfiable: core Q4_1 = 18 bytes/block vs upstream 20 (2xf16 + 16); Q8_1 upstream is 36 by static_assert vs 40 in-tree (pre-f16 layout).
      New: for every live row i, TRAITS[i].{blck_size,type_size} equals the value extracted from ggml at the pinned sha by a committed command; the extraction output is a checked-in fixture; mutation of any one row turns the test RED (shown in the PR body).

    4. Every in-tree disagreement is a row in the reconciliation table (PMAT-3431), each with: upstream value, in-tree value, call sites, and a verdict of NEVER-WORKED or REGRESSION, measured on the v0.68.1 binary - not inferred.

    5. Fix-or-refuse per type: a type whose in-tree size was wrong ships in M1 with the upstream value AND a one-line refusal + exit code on load, until a round-trip fixture for that type is green. No silent behaviour change for Q4_1/Q8_1 consumers.

    6. Remaining disputed D-rows: one amendment round, then HRQ. No fifth open round.

    One clause needs a correction before it can be executed — clause 2's sha

    The ruling makes the pinned llama.cpp sha the oracle, so every number in M1's fixture is extracted from whatever that sha resolves to. It does not resolve here. Measured on the release tree:

    sha status
    d1d3c3396 the pin of record — scripts/llama_pin.toml build_commit, pinned_on = 2026-09-15, pin_expiry = 2026-12-01, 216 references in-tree
    39173bcac an older comparator pin, still referenced 124 times (e.g. the CPU-leg parity basis in evidence/parity/thresholds.yaml)
    3173a5647 (the ruling's) appears nowhere in the tree — grep -rl over the repo returns nothing

    I am not substituting a sha into the ruling by inference: the whole point of clause 2 is that the oracle is named, and an oracle resolved by guessing is not an oracle. The implementer needs one line back: is the oracle d1d3c3396 (the pin of record), or a deliberate new pin? If it is a new pin, note that bumping the comparator invalidates every ratio measured against the old one — llama_pin.toml says so itself, and that is a decision with a named cost rather than a bump.

    Second-order finding, independent of the answer: two comparator pins are live in-tree at once — 216 references to d1d3c3396 and 124 to 39173bcac. Some committed artifacts are therefore measured against a superseded oracle. That deserves its own row regardless of how clause 2 resolves.

    What proceeds now, unblocked

    Refs #3080, #3430, #3431, #3421, #3562

  4. noahgift commented on Sep 20, 2026

    @noahgift
    MemberAuthor

    AMENDMENT (operator, 2026-09-20) — clause 2 of the M1 ruling, recorded verbatim

    2'. The oracle is scripts/llama_pin.toml build_commit, by reference, never a literal sha. Today that resolves to d1d3c3396. No pin bump is ordered by this ruling. "3173a5647" in the original clause is struck: it was the checkout the 43/35/8 count was first measured at, not a pin.

    2a. The TRAITS fixture records the sha it was extracted from. A test asserts fixture.sha == llama_pin.toml build_commit. A pin bump therefore turns the fixture RED until it is regenerated by the committed extraction command — the bump's cost is visible in the bump PR, not discovered later.

    2b. Row counts are derived, not ruled. Re-extract at d1d3c3396; if live/Removed differs from 35/8, the extraction wins and the PMAT-3430 title is corrected. "43" is [U] until then.

    2b is now measured — the ticket's numbers hold. [U] cleared.

    Extracted from the pinned checkout at /home/noah/src/llama.cpp-d1d3c3396 (git rev-parse HEAD → d1d3c3396aa1…, matching llama_pin.toml build_commit), from ggml/include/ggml.h:

    count ids
    live 35 0–3, 6–30, 34, 35, 39–42
    Removed (commented out) 8 4 Q4_2, 5 Q4_3, 31 Q4_0_4_4, 32 Q4_0_4_8, 33 Q4_0_8_8, 36 IQ4_NL_4_4, 37 IQ4_NL_4_8, 38 IQ4_NL_8_8
    total slots 43 GGML_TYPE_COUNT = 43, ids 0…42, and 35 + 8 = 43 ✓

    So 43 rows: 35 live + 8 Removed is confirmed at the pin. No title correction is needed and the count can stop being [U].

    Method note, because it nearly went the other way. A first pass with a one-line grep returned 44/41/3 and looked like a disagreement. It was wrong twice: it counted GGML_TYPE_COUNT as a type, and it classified a commented row as live whenever the comment did not contain the word "removed" (// GGML_TYPE_Q4_0_4_8 = 32, says nothing). Reading the enum settled it. Worth stating because clause 3's fixture will be generated by a command, and a sloppy extractor produces a fixture that is confidently wrong on exactly these rows — the commented block at 31–33 and 36–38 is the trap.

    Four live types are recent — MXFP4 (39), NVFP4 (40), Q1_0 (41), Q2_0 (42). Whether in-tree carries them at all is part of clause 4's reconciliation, and a NEVER-WORKED verdict is the likely shape there rather than REGRESSION.

    What clause 3's committed extraction command must do

    1. Parse enum ggml_type from ggml/include/ggml.h at the resolved pin, taking commented rows as Removed by position, not by comment wording.
    2. Take blck_size / type_size from ggml's own type-traits table, not from the header comment, and not from any in-tree value.
    3. Emit the fixture with the sha it read, so 2a's assertion has something to compare.
    4. Prove it can fail: mutate one row, expect RED (in the PR body, per clause 3).

    2a generalises, and there is already a ticket for the general case

    2a — an artifact derived from the pin records the pin, and a test asserts they match, so a bump's cost surfaces in the bump PR — is exactly the mechanism #3563 asks for repo-wide, where two comparator pins are live at once (216 references to d1d3c3396, 124 to the earlier 39173bcac). The TRAITS fixture should be the first artifact built to this shape, and #3563's guard should be written as the general form of it rather than as a separate idea.

    Refs #3080, #3430, #3431, #3452, #3563

  5. modified the milestones: 0.70.0, 0.74.0 on Sep 23, 2026
  6. noahgift commented on Sep 23, 2026

    @noahgift
    MemberAuthor

    milestone → 0.74.0 per docs/roadmaps/milestone-triage-2026-09-23.md (#4024), operator approval 2026-09-23 (relayed by aprender-cf). Reason: quant/tensor/arch dispatch consolidation: Any Model core (#4001)

  7. added 2 commits that reference this issue on Sep 25, 2026
  8. noahgift commented on Sep 25, 2026

    @noahgift
    MemberAuthor

    Landed on stage/0.70.1 (7a5a906, b8edaf1):

    • First task: bare-integer qtype lines classified. 1176 candidates → 170 qtype · 1006 not-qtype · 0 unknown. Ratchet gate scripts/classify_bare_int_qtype.py (31-case table) + baseline docs/audits/bare-int-qtype-baseline.tsv, CI fragment 480.
    • Falsifiers, each RED naming its own gate: TRAITS row drift → traits_table_equals_the_upstream_fixture_row_for_row; fake fixture id 43 → "live id 43 (FAKE) is not constructible"; planted _ => → clippy wildcard_enum_match_arm rc 101.
    • clippy measured 3 wildcard arms on GgmlType workspace-wide (all compute inference/model.rs, all zero-fill). Now named + fn-level deny.
    • Open follow-up: those sites still load unsupported dtypes as zeros; they should refuse.
  9. added a commit that references this issue on Oct 11, 2026
  10. noahgift commented on Oct 14, 2026

    @noahgift
    MemberAuthor

    merged into #5315 (master ticket for epic #4001; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.

  11. added a commit that references this issue on Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions