train 1107140231: #1941 #2439 (CI only)

CHANGELOG.md

@@ -7,6 +7,7 @@ Each release's page on GitHub has its notes and measurements. See [the 1.0.0 rel
 - Docs: the optimization ledger's cells for #2011, #2162 and #2282 say they landed (#2431).
 
 - Lane host: a stream paid for at load (`free_streams`, the engine's own sequence) is a standing lease in the KV ledger, so a family that reserves per request admits as many full-window requests as the window plan, not one fewer, and its prompt cache sits beside the same streams; the job on it promises only what it adds, and the lease is not reported at stop. No family on main reserves per request yet, so every current load is unchanged (#1943 step 2). Thanks to @laudney for finding the gap.
+- Nemotron Parse on HIP: NVIDIA's `nvidia/Nemotron-3.5-Parse` is served natively from its FP32 safetensors on a Radeon AI PRO R9700 (gfx1201), with its RADIO image tower, convolution neck and mBART decoder. The token arrays match the published example's on three pages, plain and drafted, solo and concurrent (#1941, part of #1927). Thanks to @laudney.
 
 - Developers: `test-glm-snapshots-host` runs under `test-cuda-host`, as GLM's other host steps do, so CI builds it and runs its 21 tests; its host-cache test sets `TF_GLM_HOST_CACHE` through `sys.env`, so it links on Windows too (#2426, refs #2031). Thanks to @Bizuayeu.
 - Conversation states on disk try one shorter compatible state after a refused read before prefilling from the start. Failed reads still name their error in the log.
@@ -23,6 +24,8 @@ Each release's page on GitHub has its notes and measurements. See [the 1.0.0 rel
   streams and copy drafts; there is no drafter or prompt cache yet (#1492, part of #1488). Thanks to @jschmied.
 - HIP: capture a drafter's requested target-layer taps in each stream's ring, with graph-on/off and planted kernel controls (#2314).
 
+- Docs: `/health`'s `memory` on Metal is the process footprint from the families with a reader, not `{}`, and `/memory`'s `prompt_cache_plan` row names every Mac family that reports it, with Qwen3.5-2B under `--prompt-cache-gib` (#2439). Thanks to @Whq02.
+
 ## 1.2.4
 
 - Docs: four subcommand-flag fixes checked against the code: `slide audit --threshold`'s range, `benchmark --context` and `--kv-cache`, `info` on Flash Next, and `run --device` and `--carveout` on CUDA builds (#2348).

LICENSES/NVIDIA-OpenMDW-1.1.txt

@@ -0,0 +1,57 @@
+Unless separately indicated, the model weights and NVIDIA-provided source code
+and documentation are licensed under the OpenMDW-1.1 License.
+The tokenizer included with this model is separately licensed under the
+MIT license. See TOKENIZER_ATTRIBUTION.md for the license text,
+original copyright notice, source, and NVIDIA modification history.
+
+------------
+Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+
+OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)
+
+By exercising rights granted to you under this agreement, you accept and agree
+to its terms.
+
+As used in this agreement, "Model Materials" means the materials provided to
+you under this agreement, consisting of: (1) one or more machine learning
+models (including architecture and parameters); and (2) all related artifacts
+(including associated data, documentation and software) that are provided to
+you hereunder.
+
+Subject to your compliance with this agreement, permission is hereby granted,
+free of charge, to deal in the Model Materials without restriction, including
+under all copyright, patent, database, and trade secret rights included or
+embodied therein.
+
+If you distribute any portion of the Model Materials, you shall retain in your
+distribution (1) a copy of this agreement, and (2) all copyright notices and
+other notices of origin included in the Model Materials that are applicable to
+your distribution.
+If you file, maintain, or voluntarily participate in a lawsuit against any
+person or entity asserting that the Model Materials directly or indirectly
+infringe any patent or copyright, then all rights and grants made to you
+hereunder are terminated, unless that lawsuit was in response to a
+corresponding lawsuit first brought against you.
+
+This agreement does not impose any restrictions or obligations with respect to
+any use, modification, or sharing of any outputs generated by using the Model
+Materials.
+
+THE MODEL MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE
+ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL
+TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+
+YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY
+APPLY TO THE MODEL MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION
+ANY PERSON'S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MODEL
+MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS
+REQUIRED FOR ANY USE OF THE MODEL MATERIALS; OR (3) PERFORMING ANY DUE
+DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MODEL MATERIALS OR
+ANYTHING INCORPORATED OR EMBODIED THEREIN.
+IN NO EVENT SHALL THE PROVIDERS OF THE MODEL MATERIALS BE LIABLE FOR ANY CLAIM,
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL MATERIALS, THE
+USE THEREOF OR OTHER DEALINGS THEREIN.

THIRD_PARTY_NOTICES.md

@@ -35,6 +35,23 @@ The native engine installs and links neither. It reads checkpoints in their form
 `zig/kernels/metal/` repeat the arithmetic of MLX's kernels and mlx-lm's model math, many of them bit for bit.
 The Python engine, frozen at 0.6.6, installs MLX and mlx-lm as dependencies.
 
+## Nemotron 3.5 Parse on HIP
+
+The reference tool `tools/zig/nemotron_parse_reference.py` adapts `run_transformers_inference.py` from NVIDIA's
+[nvidia/Nemotron-3.5-Parse](https://huggingface.co/nvidia/Nemotron-3.5-Parse) at revision
+`4035ffb1d8f8bafcdb867dfaf6f7514f101464f9`, and keeps that file's SPDX header. The native family follows the same
+release's code for its math, rounding and resizing; each file is written for TensorFold, and the file named is the
+reference:
+
+- `zig/src/families/nemotron_parse/processor.zig`: `NemotronParseImageProcessor` in `hf_nemotron_parse_processor.py`
+  (the aspect-preserving resize, the NaViT white padding, the normalization and the patch order).
+- `zig/src/families/nemotron_parse/tower.zig` and `decoder.zig`: `RadioWithNeck`, `_VisionTransformer`,
+  `NemotronParseDecoder`, `_DecoderLayer`, `_GQAAttention` and `_ScaledEmbedding` in `hf_nemotron_parse_modeling.py`.
+- `zig/kernels/hip/ops/vl_position.hpp`: `_normalized_2d_position_pages` in `hf_prefix_geometry.py`.
+
+These release files are Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES, under the NVIDIA OpenMDW License 1.1; see
+[the license text](LICENSES/NVIDIA-OpenMDW-1.1.txt).
+
 ## MLX and mlx-lm adaptations
 
 In the Python engine, frozen at 0.6.6, the DeltaNet implementations in

build.zig

@@ -205,6 +205,7 @@ pub fn build(b: *std.Build) void {
     const with_hip = b.option(bool, "hip", "Enable the opt-in HIP backend and its host/device targets") orelse false;
     const hip_options = hip_build.options(b); // declared on every target (#1165), used only with -Dhip=true
     hip_build.loaderTests(b, test_step);
+    @import("zig/build/nemotron_parse.zig").hostTests(b, test_step);
     if (with_hip) hip_build.steps(b, target, optimize, build_options, test_step, hip_options);
     const with_vulkan = b.option(bool, "vulkan", "Enable the opt-in Vulkan backend and its host/device targets") orelse false;
     const vulkan_options = vulkan_build.options(b); // declared on every target, used only with -Dvulkan=true

docs/api.md

@@ -963,7 +963,7 @@ configured the body is only `{"status": "ok"}`. An engine that reports unhealthy
 | `max_batch_size` | integer | The lanes the engine serves at once |
 | `policy` | string | The engine's policy line; absent when the engine reports none |
 | `warming` | boolean | Always `false` on the native engines |
-| `memory` | object | CUDA: `active`, `cache` and `peak` in bytes (`cache` is 0), plus `limit` when the engine reports one. Metal: `{}` |
+| `memory` | object | `active`, `cache` and `peak` in bytes (`cache` is 0), plus `limit` when the engine reports one: CUDA's allocator counts; on Metal the process footprint, from Nemotron, Qwen3.5-2B, Qwen3.8-27B, Qwen3.5-MoE, Flash Next, GLM-5.3-Flash and GLM-5.3 (EXLR8); Flash Next and both GLMs add `limit`, their load limit. `{}` from the other Metal families |
 | `live` | object | `connections`, `waiting`, `decode_tokens_per_second`, `prefill_tokens_per_second` |
 | `version` | string | The release this server was built from |
 | `backend` | string | The backend built in: `metal`, `cuda`, `hip` or `xpu`; absent from a build without one |

docs/memory-diagnostics.md

@@ -17,7 +17,7 @@ zero means a measured or configured zero. The endpoint publishes no paths, UUIDs
 | `device.active_bytes`, `device.cache_bytes` | Existing backend allocator counters, where implemented. |
 | `device.peak_bytes` | Backend peak since startup or the last `/health?reset_peak=1`. |
 | `device.peak_scope` | `since_start_or_health_reset`; distinct from the process lifetime peak. |
-| `prompt_cache_plan` | Immutable retained-prefix budget applied at startup, reported by Nemotron, Flash Next and Qwen3.8-27B on Macs. |
+| `prompt_cache_plan` | Immutable retained-prefix budget applied at startup, reported by Nemotron, Flash Next, Qwen3.8-27B, Qwen3.5-MoE, Gemma 4, LFM2.5-24B-A2B and LFM3-1.7B on Macs, and by Qwen3.5-2B with `--prompt-cache-gib`. |
 
 The cache plan contains `source`, `budget_bytes`, and `explicit_budget`. With OS readings,
 `source` is `physical_footprint` and the plan also includes the sizing inputs: `ram_bytes`,
@@ -30,11 +30,11 @@ rather than merely the presence of an override flag.
 
 Without both OS readings, the existing fallback uses an explicit budget (`source: explicit`) or the spare
 Metal working set up to 16 GiB (`source: metal_working_set`). Its OS inputs and unknown `over_cap` remain omitted.
-Nemotron and Flash Next report a disabled retained-prefix cache as a known `budget_bytes: 0`; Qwen3.8-27B omits
-`prompt_cache_plan` when its budget is 0. GLM-5.3-Flash and the CUDA server omit it too: GLM keeps
-`--prompt-cache-gib` or 16 GiB inside its load limit, and the CUDA server keeps `--prompt-cache-gib` or 16 GiB inside
-what its streams leave and names that size in its startup line. The 2B keeps prompt states only with
-`--prompt-cache-gib`, and then reports the plan.
+Nemotron, Flash Next, LFM2.5-24B-A2B and LFM3-1.7B report a disabled retained-prefix cache as a known `budget_bytes: 0`;
+Qwen3.8-27B, Qwen3.5-MoE and Gemma 4 omit `prompt_cache_plan` when their budget is 0. GLM-5.3-Flash and the CUDA server
+omit it too: GLM keeps `--prompt-cache-gib` or 16 GiB inside its load limit, and the CUDA server keeps `--prompt-cache-gib`
+or 16 GiB inside what its streams leave and names that size in its startup line. The 2B keeps prompt states only with
+`--prompt-cache-gib`, and then reports the plan, `budget_bytes: 0` included.
 This plan is a cache allowance, not a process limit, current cache occupancy, or an admission guarantee.
 
 `/metrics` keeps `tensorfold:process_footprint_bytes` and adds `tensorfold:process_footprint_peak_bytes`

docs/recipes/README.md

@@ -10,6 +10,7 @@ For agent-assisted contributions, read the [TensorFold contributor skill](SKILL.
 | dots.mocr | [Native HIP OCR, BF16, one GPU](dots-mocr-hip.md) |
 | Kimi K3 / K3.1 | [Native Metal bring-up, synthetic only, resident budget refusal](kimi-k3-metal.md) |
 | Nemotron 3.5 Lightning | [MLX](nemotron-3.5.md), [Intel Arc, experimental](xpu-arc.md) |
+| Nemotron 3.5 Parse | [Native HIP OCR, validation and qualification limits](nemotron-parse.md) |
 | Nemotron 3.5 Super VL | [Native CUDA on one or two GB10s, images](nemotron-3.5-super-vl.md) |
 | Qwen3.8-27B | [MLX, quantization and CUDA](qwen3.8-27b.md) |
 | Qwen3.5-2B | [Native Metal preview, tied affine 4-bit](qwen3.5-2b-native.md) |

docs/recipes/nemotron-parse.md

@@ -0,0 +1,141 @@
+# Nemotron 3.5 Parse on native HIP
+
+This port runs `nvidia/Nemotron-3.5-Parse` on one HIP rank. The
+validated checkpoint revision is
+`4035ffb1d8f8bafcdb867dfaf6f7514f101464f9`. The family is
+`nemotron_parse`; it is separate from Nemotron-H and Super VL.
+
+The original FP32 safetensors stay unchanged. The loader validates all
+510 tensors, then owns BF16 device copies of the 509 floating tensors.
+Activations and KV use BF16 with FP32 sums. The image tower, horizontal
+neck, causal mBART decoder and trained one-step proposal run in native HIP
+kernels. Python is used only for independent development references.
+
+## Build and serve
+
+Use the repository Zig version. On ROCm installations with a different
+layout, give the actual compiler and device-library paths:
+
+```sh
+zig build native-hip -Dhip=true -Dhip-arch=gfx1201 \
+  -Dhipcc=/opt/rocm/core/bin/hipcc \
+  -Dhip-device-lib=/opt/rocm/core/lib/llvm/amdgcn/bitcode
+
+zig-out/native-hip/bin/tensorfold-native serve "$PARSE_MODEL" \
+  --backend rocm --device 0 --vision --name parse --parallel 2 --context 4096 \
+  --mtp-drafts 1 --prompt-cache-gib 0.25 --host 127.0.0.1 --port 8000
+```
+
+`PARSE_MODEL` names the already cached snapshot. Select the intended
+visible device and verify its PCI identity; management and HIP ordinals
+can differ. The engine checks driver free memory at load and admission.
+The developer checks require at least six GiB free before a bounded
+two-lane run.
+
+Send one user message with one PNG `image_url` data URL to
+`/v1/chat/completions`. For example, with an ordinary OpenAI client:
+
+```python
+import base64
+from pathlib import Path
+from openai import OpenAI
+
+image = base64.b64encode(Path("page.png").read_bytes()).decode()
+client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="local")
+reply = client.chat.completions.create(
+    model="parse",
+    messages=[{"role": "user", "content": [
+        {"type": "image_url", "image_url": {"url": "data:image/png;base64," + image}}
+    ]}],
+    temperature=0,
+    max_tokens=1024,
+)
+print(reply.choices[0].message.content)
+```
+
+Use the served model name reported by `/health`, or set `--name` explicitly.
+The built-in media template puts the visual prefix before the example's
+six task tokens. Optional text parts follow those tokens. Generated OCR,
+including its location/class markers, is returned in `content`. A custom
+`--chat-template` still overrides the built-in bridge.
+
+The image processor shrinks into 2048-by-1664 without enlargement, pads
+bottom/right to 16/64, and preserves the visible content size for RADIO
+positions. A full page uses 3329 visual context rows. Visual and text rows
+share the 16384-row model context; `--context` may lower this limit. One
+image is required. Multi-turn chat, video, tensor parallelism, lossy KV,
+Sliding Weights and multiple proposal heads are not part of this port.
+
+Default generation is greedy with repetition penalty 1.1. The shared
+sampler widens raw logits to FP32 before applying penalties, and forces
+EOS at the configured final output position. `"draft": false` selects
+plain target decoding; normal drafted requests use the trained head and
+the existing target verifier. Shared rounds batch dense work across all
+admitted streams. Prompt snapshots use image content and frame identities.
+
+## Reproduce the checks
+
+Create offline BF16 references from the checkpoint's own example workflow:
+
+```sh
+python tools/zig/nemotron_parse_reference.py --model "$PARSE_MODEL" \
+  --image page.png --output "$REFERENCE"
+python tools/zig/nemotron_parse_reference.py positions --model "$PARSE_MODEL" \
+  --output "$POSITION_REFERENCE"
+
+zig build test-nemotron-parse-host -Dparse-model-dir="$PARSE_MODEL" \
+  -Dparse-reference-dir="$REFERENCE"
+
+TF_PARSE_DIR="$PARSE_MODEL" TF_PARSE_POSITION_DIR="$POSITION_REFERENCE" \
+  zig build hip-parse-ops-test -Dhip=true -Dhip-arch=gfx1201 \
+  -Dhipcc=/opt/rocm/core/bin/hipcc \
+  -Dhip-device-lib=/opt/rocm/core/lib/llvm/amdgcn/bitcode
+```
+
+The reference environment used CPU Torch 2.11.0, TorchVision 0.26.0,
+Transformers 5.14.1 and timm 1.0.22. It loads only local assets. The tool
+retains NVIDIA's OpenMDW notice; the license is
+[LICENSES/NVIDIA-OpenMDW-1.1.txt](../../LICENSES/NVIDIA-OpenMDW-1.1.txt).
+Reference directories must be new; large tensors are not checked into Git.
+`nemotron_parse_reference.py processor` prints the model-free
+`zig/src/families/nemotron_parse/fixtures/processor.json`.
+
+`tf-nemotron-parse-test` builds a developer probe. Give it the model,
+reference and output directories. It compares intermediate seams and
+teacher-forced logits, then checks full/chunked and solo/shared KV/hidden
+bytes and suffix rollback. `nemotron_parse_reference.py compare
+--reference "$REFERENCE" --native "$PROBE_OUTPUT"` applies the original HF
+logits processor to saved native rows.
+
+The kernel contract registers twelve new operator cases, which
+`tf-hip-kernel-contract` runs with no argument, beside the rest of the
+contract and its planted controls. `hip-kernel-contract-objects` builds
+the code objects for gfx1030, gfx1100, gfx1151 and gfx1201, and
+`hip-kernel-contract-limits` checks every launch record against them
+without a GPU.
+
+## Evidence and limits
+
+Native HTTP token arrays matched three PNG references (two 512-by-256
+pages, one 1664-by-2048 page), plain/drafted and solo/concurrent. The
+full page uses 3335 prompt rows including the six text tokens. Prompt KV
+and final-hidden bytes agree across full/chunked and solo/shared forwards
+and after suffix rollback. No speed target is claimed.
+
+All twelve new operator cases pass arithmetic gates 1–4, including five
+repeat runs, guards, non-finites, tile edges and accumulator stress.
+Their seven launch records match the code objects of all four recorded
+chips and the live gfx1201 device. The complete contract reports only
+the limit problems that main reports too: two stale records on the live
+gfx1201 (`tf_sum_partials`, `tf_sum_partials_act`) and 13 across the four
+chips' code objects.
+
+BF16 intermediate tensors differ from HF's CPU GEMM rounding; saved
+seam errors are diagnostic, while actual output token arrays are checked
+separately. Runtime model checks cover gfx1201 only. CUDA, Metal, other
+AMD hardware have not been qualified here. Sampled plain/drafted equality
+passes at two small-page seeds, one with presence and frequency penalties;
+this does not compare TensorFold's keyed sampler with HF's random draws.
+PNG is tested. JPEG uses the repository's pinned libjpeg-turbo decoder
+and has no qualification receipt yet. Structured output needs the
+existing grammar library and has no real-model receipt yet.