[BlivionIaG]

What changes

Serve LFM3-1.7B from the official MLX 4/6/8-bit checkpoints through the native Metal lane engine. Add decoder operators, per-stream convolution/KV state, verification, snapshots and prompt-cache integration. Paired FFN/KV projections, shared convolution and post-add/next-norm fusion preserve their retained rounding boundaries. The range-safe attention correction repairs underflow/overflow without relaxing numerical bounds.

Refs #1491. Depends on admission #1513; vision is stacked above this in #1502. This text slice refuses VL checkpoints. The PRs target upstream main until their foundations land; review the text-only difference.

flowchart LR
    A[Admission: 1513] --> T[Text: this PR]
    T --> V[VL: 1502]

Loading

Code pointers

• Native loader and serving integration.
• Model loading.
• Decoder forward pass.
• Shared lane adapter and prefill lifecycle.
• KV, convolution state and scratch ownership.
• Full/sliding attention kernels.
• Metal contract checks.
• State and logit exactness checks.

Current long-context checks

Current M5 long-context checks: all 26 requests completed: VL6/8 text prompts at 32k/64k, text4/6/8 at 131072/262128 prompt tokens, and VL4/6/8 at 127984 prompt tokens, each repeated in a fresh process. All return 16 tokens with zero cached input; all 13 repeated output-token hashes match. All 16 servers exit zero, all three packet leases are released, and the controller exits zero. These are native text-context checks, including VL checkpoints; they do not establish image-context, FP64 or comparative speed acceptance.

Validation and numerical evidence

Recipe and receipts pin source, executable and input hashes. The extracted text range-correction artifact passes its native/contract/host build: 67/67 steps, formatting passed, and zero new lean findings over 428 inherited. Its separate receipt does not claim a separate text-binary GPU run.

The matching combined candidate passed locked local M4 qualification across all six text/VL checkpoints: 408/408 state checks, 303/303 serving checks, including sampled/cancellation/image cases, and 394,752,000 logit bytes identical to the previous public runtime on retained fixtures. The combined range receipt identifies that artifact separately. Registered original-fail/new-pass attention cases retain the same FP64 bounds; skip-repair and clear-risk controls fail as intended. Guarded M5 testing separately passed the current 70-kernel combined LFM3 contract set and normal limits. Live CI is shown by PR checks.

Independent prompt reports retain all 5,581 rows, changed top tokens and both margins. Named production-rounding replay resolves disjoint maximum sets; 19 lowest-index differences remain overlapping maxima. All six genuine one-row decode captures after 63 identical history tokens have zero top differences. These diagnostics do not invent a tie policy or erase ideal-FP64 evidence.

The earlier frozen combined full-backend M5 run at 8955faee0 reports 309/310 kernels passing, one inherited Qwen4 HC failure, two known findings, all 317 controls red and zero limit problems. The failure reproduces on pinned base b30d5b9d. This is not a global pass or a complete-backend check of the extracted text binary.

Measurements and qualification

Extracted text artifact: 15 focused state checks, 120 serving checks and 18 cancellation checks passed across text 4/6/8-bit. Six prompt/decode comparisons match 49,920,000 bytes against the combined artifact. Public native/forward/exact build: 65/65 steps.

M5 text four-bit code decode, solo / four streams: 315.75 / 861.85 tok/s; standard 344.20 / 995.0. Requests produce 256 tokens. Cold requests produce 16 tokens with zero cached input and a 70k request budget. Cache allocation differs; standard reply hashes are unavailable.

Four-bit M5 measurements, 6/8-bit M5 measurements, M4 measurements, 32k/64k checks.

Prefill lifecycle and scratch reuse retain their tested source/binary scope. Scratch reuse removes one allocation: 1,572,864 requested buffer bytes at 128 rows. No throughput or physical-footprint change is asserted for that allocation reduction. Final text 6/8-bit checks: 136 state checks and 80 serving checks passed for the text checkpoints on the combined Metal artifact; current prompt/decode outputs match retained native outputs. These are not device runs of the extracted text-slice binary. Long-context checks are recorded below.

Remaining checks: shared prompt/decode path requirements, merged-chain integration, and maintainer review of recorded FP64 differences and performance comparisons. No optimization experiments are scheduled.

Weights: LFM Open License v1.0. Checked machines: M4 Pro 48 GB and M5 Ultra 256 GB; these measurements do not establish smaller-memory qualification. ATC issue 1491 remains held.

Prefill optimization status

Performance qualification remains open. Private replay-splitting passed 35 synthetic attention cases on M4 and M5 with identical intermediate buffers, final BF16 output and replay decisions, plus planted controls. Five balanced M5 timing rounds found no useful gain: the complete 64-row attention operation measured 5.686 ms public versus 5.682 ms split at a 2k history, and 3.701 ms versus 3.766 ms at 8k. This experiment remains disabled.

An intentionally unsafe diagnostic that suppresses range-risk predicates and replay measured 15.93x and 2.72x public throughput on the same two synthetic fixtures. It is not a production candidate or an end-to-end speedup; the measurement combines predicate, replay and compiler effects. The next candidate preserves the original risk decisions while reducing their common-case cost. No new prefill/decode/TTFT/ITL serving result against MLX-VLM is claimed yet.

The disabled risk-predicate experiment is now committed in 1543227fc. The native, kernel-contract and developer-tool builds passed, the CPU predicate-equivalence proof passed, and the public M5 targeted contract passed both full/sliding kernels with zero failures or limit problems. The receipt retains the negative M5 timing result: 5.707 to 6.417 ms at 2k and 3.708 to 6.385 ms at 8k. -Dlfm3-risk-fast=true is a developer opt-in; the default remains unchanged. This is a draft experiment, not performance qualification. Current optimization work prioritizes prompt FFN projections; no new matched serving result is claimed.

[ashhart]

Thanks @BlivionIaG for the depth of evidence here. The FP64 oracle tests and the byte-equality runs are careful work.

Before this can leave draft:

1. Please add a PR description: what it serves, which formats, what is still unqualified, and that it stacks on lfm3: admit official MLX text and VL checkpoints #1513.
2. Several receipts contain local temp paths (/private/tmp/...) in attention-range-text.json, attention-range.json, current-public-long-context.json and local-standard-phase1.json. Please replace them with relative or placeholder paths.
3. The receipts are about 8.6 MB. The cold-prompt bodies and ids are synthetic and can be regenerated. Please commit the generator and their hashes instead of the files, and trim the large raw captures (kv-pair.json, greedy-dual.json and others) to summaries plus hashes.
4. The threaded contract reference (affine.zig, reference_threads.zig) and the GLM test changes touch other families. Please move them to their own PR.
5. The receipts note that the global contract run is not on the final source, and that the model-fidelity gate is still open. We'll agree that gate on Add native Metal support for LFM3-1.7B and LFM3-VL-2B MLX 4/6/8-bit checkpoints #1491 before serving lfm3 by default.