jschmied · Oct 30, 2026 · Oct 30, 2026 · Oct 30, 2026 · Oct 30, 2026 · Oct 30, 2026
diff --git a/build.zig b/build.zig
@@ -103,6 +103,7 @@ pub fn build(b: *std.Build) void {
     @import("zig/build/digest.zig").steps(b, test_step, draft_ids, build_options);
     @import("zig/build/flashnext_budget.zig").steps(b, test_step);
     @import("zig/build/flashnext_batch_host.zig").tests(b, test_step);
+    @import("zig/build/lfm3_host.zig").steps(b, test_step);
     if (b.graph.host.result.os.tag == .macos) {
         const budget_mods = metalEngineModules(b, b.graph.host, .Debug, draft_ids);
         const budget_empty = b.addWriteFiles().add("budget-compile-only.metallib", "");

diff --git a/docs/recipes/lfm3-native.md b/docs/recipes/lfm3-native.md
@@ -0,0 +1,72 @@
+# LFM3 native Metal bring-up
+
+Tracking issue: [#1491](https://github.com/ashhart/TensorFold/issues/1491).
+
+This branch adds host checks for LFM3 checkpoint configuration and text tensor layouts.
+It does not register a serving engine. Text generation and image input are not implemented or qualified here.
+
+## Inspect a checkpoint
+
+Use Python 3.11 or newer. The inspector uses the standard library. It resolves a full Hugging Face commit,
+reads the config and index, and requests only the safetensors headers. It refuses a server that returns a
+complete weight file instead of the requested byte range.
+
+```sh
+python3 tools/lfm3/inspect_checkpoint.py LiquidAI/LFM3-1.7B-MLX-4bit --output text4.json
+python3 tools/lfm3/inspect_checkpoint.py LiquidAI/LFM3-VL-2B-MLX-4bit --output vl4.json
+```
+
+Replace `4bit` with `6bit` or `8bit` to inspect those exports. Pass `--revision COMMIT` to repeat a pinned inspection.
+Use `--local-dir MODEL_DIR` instead of a repository name to inspect files already on disk.
+
+## Check the native text layout
+
+Use the Zig version in `.zig-version`.
+
+```sh
+zig build test-lfm3-host tf-lfm3-check
+./zig-out/bin/tf-lfm3-check text4.json
+./zig-out/bin/tf-lfm3-check vl4.json
+python3 -m unittest discover -s tools/lfm3 -p 'test_*.py' -v
+```
+
+The native checker reads the saved metadata, validates shard header geometry, and checks the LFM3 text decoder's
+required names, dtypes, shapes, and byte counts. It performs no projection, GPU allocation, or model inference.
+For VL, it checks the text decoder only. A passing result does not validate vision weights or image processing.
+
+The config reader preserves full-attention versus sliding-attention layers, distinct rotary settings, sandwich norms,
+and each module's affine width and group size. Unsupported geometry and conflicting quantization declarations fail.
+
+## Checkpoint details
+
+All six inspected MLX exports store the decoder under `language_model.model.*`, including the text-only model.
+Each decoder has 594 tensors. Convolution kernels are BF16 with shape `[2048, 7, 1]`.
+
+Text 4/6/8-bit and VL 6/8-bit use affine groups of 64. VL 4-bit uses groups of 32 and an 8-bit embedding override.
+The 138 additional VL vision/projector tensors are BF16 in all three exports. Global quantization metadata must not
+cause these dense tensors to be treated as packed affine weights.
+
+The header inspections used these revisions:
+
+- Text 4-bit: `d0ca73dcfa70ce18733754a492cdc1fb0123262d`.
+- Text 6-bit: `26be898c36b545d625d84ac0761c6c3eb664aef3`.
+- Text 8-bit: `ee7667f3cfccb2a41725f91820531823fcabc6b7`.
+- VL 4-bit: `c823fade98e35a3d6942ad4d38b1ea15f675cd12`.
+- VL 6-bit: `6d71bfae121abedc75f6d531fc7f6ceb4f5e3501`.
+- VL 8-bit: `83336d5be3874a2bdd3297bde77b2114dbd3a816`.
+
+## Next gate
+
+Implement the text operators against an independent reference before advertising generation.
+Check the 7-tap convolution, per-head attention gate, sandwich normalization, sliding window, and partial-RoPE formula.
+Then exercise actual 1.7B 4-bit weights through the shared lane engine, followed by 6/8-bit checkpoints.
+The VL encoder and projector need their own native Metal operators and reference checks after text bring-up.
+
+Agree model-fidelity and performance acceptance criteria on the tracking issue before claiming release readiness.
+
+## Weight licence and checked hardware
+
+The official text and VL MLX exports retain the [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM3-1.7B-MLX-4bit/blob/main/LICENSE).
+The VL export carries its own [copy of that licence](https://huggingface.co/LiquidAI/LFM3-VL-2B-MLX-4bit/blob/main/LICENSE).
+TensorFold code remains Apache-2.0; admission does not change the weights’ licence.
+This slice validates metadata on the host and does not register inference or qualify a GPU.
diff --git a/docs/recipes/xpu-arc.md b/docs/recipes/xpu-arc.md
@@ -50,6 +50,23 @@ Intel Arc Pro B70 Graphics (32.5 GB maximum single allocation), Linux 7.0.0 (`xe
 Nemotron MLX 4-bit checkpoint (`TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit`, revision `d9d758fb83953437f7263256b0d96157e2a348b8`,
 729 tensors, 17.27 GiB of weights). The commit under test is the head of the pull request.
 
+LFM3-1.7B (`LiquidAI/LFM3-1.7B-MLX-4bit`, MLX affine 4-bit, group 64) on an Intel Iris Xe laptop GPU (Gen12LP, `tgllp`, 96 EUs, no
+DPAS) under WSL2 (Ubuntu 24.04, 8 GB given to WSL), Ubuntu's compute runtime 23.43.27642 and Level Zero loader 1.16, Zig 0.17.0,
+`zig build xpu-native -Dxpu-device=tgllp` (DPAS off, so the server is built without the Qwen3.8-27B runner):
+
+| Setup | Decode, c=1, `--no-drafts` | Decode, c=1, copy drafts | Decode, c=4, aggregate | TTFT, 1,024 tokens | TTFT, 8,192 tokens | Memory (`serve --context 16384`) |
+| --- | ---: | ---: | ---: | ---: | ---: | --- |
+| Iris Xe, LFM3-1.7B MLX 4-bit | 15.6 to 16.8 tok/s | 15.9 (chat) to 26.3 (prose) tok/s | 25 to 32 tok/s | 11.7 to 12.2 s | 116.3 to 117.7 s | 1.33 GB device allocation, 448 MB host resident |
+
+Decode, c=1: two short chats (160 tokens), a 1,001-token prose prompt and a 601-token code prompt (128 tokens), greedy, three runs
+each, with copy drafts (the default) and with `--no-drafts`; c=4: four concurrent 128-token completions of distinct 32-token prompts,
+six runs a mode, with or without drafts; TTFT: prompts of numbered item sentences, one token generated; three runs each, through
+`/v1/completions`. Copy-draft gains depend on how repetitive the text is: prose and code that repeat their own context decode at 26
+and 22 tok/s, the chats, with few copies to draft, at the speed without drafts. A `/v1/chat/completions` request with the
+checkpoint's chat template answers (thinking on, `finish_reason` stop).
+With `--parallel N` each stream has its own sequence (about 196 MiB of KV at 16,384 tokens); a second chat turn over an
+8,037-token document resumes from the prompt cache in 0.80 s instead of 122.6 s, with the same reply.
+
 ## Measurements
 
 Decode, 128 greedy tokens from a 5-token prompt: 5.72 ms/token, 175 tok/s (`tensorfold-xpu run`, five runs on different

diff --git a/tools/lfm3/REF64.md b/tools/lfm3/REF64.md
@@ -0,0 +1,94 @@
+# Float64 reference receipt: LFM3-1.7B NVFP4 on CUDA
+
+- Checkpoint: [LiquidAI/LFM3-1.7B-NVFP4](https://huggingface.co/LiquidAI/LFM3-1.7B-NVFP4) at revision
+  `b30025bbbbcbf62f6c110b40d51d36af6579f6a2`.
+- Engine: `9265a0f5f`, which is the PR head `08b338c7c` before its rebase (range-diff: the same patches except one
+  test-only line in `contract_limits.zig`). Served on a GB10 (sm_121), greedy, drafts off,
+  `--parallel 8 --context 32768`. The `collect` step was repeated on 2026-10-28 and returned the same 957 tokens.
+- Reference: `ref64.py` on the CPU in float64 (Python 3.12, torch 2.14, safetensors 0.8, tokenizers 0.23), 2026-10-28.
+
+Reproduce (the reference pass takes about a minute and 20 GB of host memory):
+
+```
+zig-out/native/bin/tensorfold-native serve MODEL_DIR --port 8080 --parallel 8 --context 32768
+python -I tools/lfm3/ref64.py collect http://127.0.0.1:8080 MODEL_DIR engine.json
+python -I tools/lfm3/ref64.py MODEL_DIR engine.json --bf16
+```
+
+Rows. A prompt row is the engine's one-token greedy pick after the prefix [0, t] of a prompt, one request per prefix.
+A reply row is a token of the engine's 32-token greedy reply, teacher-forced on that reply. `t` is the logits row after
+token t, so prompt row P - 1 and reply token 0 are the same position. The prompts (`prompts()` in the script) are
+capitals (20 tokens), code (40), a chat turn (19) and items (750, a list of 60 items and a question; it crosses the
+256-key sliding window).
+
+The float64 pass dequantizes the NVFP4 weights exactly. The `--bf16` pass adds the engine's bf16 storage points (see
+the script's docstring). It shows which side of a miss rounding alone lands on; it is not bit-exact.
+
+## Result
+
+- f64: prompt rows 825/829, reply rows 127/128
+- bf16: prompt rows 827/829, reply rows 126/128
+
+Float64 top-1 misses (margin = f64 logit of its top minus f64 logit of the engine's token; bf16 margin = the same difference in the bf16-points pass, < 0 means it prefers the engine's token):
+
+| case | row | t | engine | float64 | f64 margin | bf16 margin | bf16 picks |
+|---|---|---|---|---|---|---|---|
+| capitals | reply | 24 | 11097 " Spain" | 3888 " Japan" | 0.1176 | +0.1250 (pre-round +0.1802) | f64's |
+| chat | prompt | 12 | 593 " per" | 22 "." | 0.1530 | -0.0625 (pre-round -0.0671) | engine's |
+| items | prompt | 5 | 1008 " first" | 1579 " number" | 0.0996 | +0.0000 (pre-round +0.1009) | engine's |
+| items | prompt | 9 | 29 "5" | 654 "10" | 0.0033 | +0.0000 (pre-round +0.0093) | engine's |
+| items | prompt | 180 | 28 "4" | 30 "6" | 0.0674 | +0.0000 (pre-round +0.0076) | engine's |
+
+bf16-points misses where float64 agrees with the engine (margin = bf16 logit of its pick minus the engine's token; f64 margin = how far float64 prefers the engine's token):
+
+| case | row | t | engine | bf16 pick | bf16 margin | f64 margin |
+|---|---|---|---|---|---|---|
+| items | prompt | 4 | 440 " The" | 229 " " | 0.0000 (pre-round -0.0985) | 0.0881 |
+| items | prompt | 631 | 32 "8" | 25 "1" | 0.0000 (pre-round -0.0467) | 0.0636 |
+| items | reply | 772 | 41501 " Item" | 1672 " So" | 0.0000 (pre-round -0.0814) | 0.1797 |
+
+## FP8 and MLX-4bit (2026-10-29)
+
+- Checkpoints: [LiquidAI/LFM3-1.7B-FP8](https://huggingface.co/LiquidAI/LFM3-1.7B-FP8) at
+  `243955da95b1cd68026f293b29de4b4c13f3638f` (block FP8, dequantized exactly: e4m3 times its block's bf16 scale) and
+  [LiquidAI/LFM3-1.7B-MLX-4bit](https://huggingface.co/LiquidAI/LFM3-1.7B-MLX-4bit) at
+  `d0ca73dcfa70ce18733754a492cdc1fb0123262d` (MLX affine, dequantized exactly).
+- Served the same way with `--no-drafts`; the same prompts and rows. Only the float64 pass was run: the `--bf16`
+  pass models the NVFP4 engine's rounding points.
+
+FP8: prompt rows 824/829, reply rows 128/128.
+
+| case | row | t | engine | float64 | f64 margin |
+|---|---|---|---|---|---|
+| items | prompt | 131 | 30 "6" | 1325 "24" | 0.0348 |
+| items | prompt | 582 | 654 "10" | 1035 "14" | 0.0756 |
+| items | prompt | 619 | 25 "1" | 29 "5" | 0.0066 |
+| items | prompt | 631 | 27 "3" | 32 "8" | 0.0290 |
+| items | prompt | 693 | 9402 " stone" | 17937 " stones" | 0.0773 |
+
+MLX-4bit: prompt rows 825/829, reply rows 128/128.
+
+| case | row | t | engine | float64 | f64 margin |
+|---|---|---|---|---|---|
+| items | prompt | 10 | 6802 " items" | 93266 " marbles" | 0.0466 |
+| items | prompt | 61 | 41501 " Item" | 29155 " Question" | 0.0042 |
+| items | prompt | 595 | 27 "3" | 1035 "14" | 0.0300 |
+| items | prompt | 748 | 34 ":" | 10494 " holds" | 0.0848 |
+
+## Without torch: the numpy path and the XPU's bf16 points
+
+`ref64.py` runs on numpy when torch is not installed (or with `--numpy`): MLX affine and bf16 weights are dequantized
+exactly as on the torch path; NVFP4 and FP8 exports are refused by name and need torch. `--bf16-points` replaces the
+`--bf16` pass with the Intel GPU (XPU) engine's storage points: the embedding rows and RoPE cos/sin rounded to bf16 too,
+attention P and the logits never. `dump` writes the float64 hidden state after every layer for the XPU forward's
+layer-by-layer check, and `check` replays an engine's greedy replies teacher-forced:
+
+```
+python -I tools/lfm3/ref64.py dump MODEL_DIR OUT_DIR
+python -I tools/lfm3/ref64.py check MODEL_DIR tokens.json      # {"name": {"prompt": [...], "reply": [...]}}
+python -I tools/lfm3/ref64.py MODEL_DIR engine.json --bf16-points
+```
+
+On LiquidAI/LFM3-1.7B-MLX-4bit (Python 3.12, numpy 2.5, no torch; 3 min 14 s and 4.1 GB for `dump`): the three
+prompts' bf16-points gap reaches 0.041 (capitals), 0.025 (chat) and 0.111 (items, worst row of any layer), and the
+XPU engine's 64-token greedy replies agree with the float64 top-1 at 192 of 192 positions.