Repository navigation
Conversation
Nemotron 3.5 Parse needs its own family before any device work, so this validates the Parse config graph and every stored tensor of the checkpoint, and ports its image processor with the same resize, white padding, normalization and patch order. A host test checks the processor against independent Pillow and NumPy fingerprints that the new nemotron_parse_reference.py processor subcommand prints, and the script keeps NVIDIA's OpenMDW notice with its license under LICENSES.
Image towers on native HIP need dense operators that the text kernels do not provide, so the vision set that dots.mocr put in ops/vision.hpp gains embedding, biased linear, layer norm, pointwise, layout, resize and content-normalized position kernels, hip.vision a handle that loads them, and the vision records their seven launches. Parse keeps its own LayerNorm: it takes 1 / sqrtf where dots' takes rsqrtf, and about one BF16 output in 100,000 differs, so each family keeps the norm its receipts were made on. The full kernel contract runs the twelve new cases, and gate 4 now names FP32 sums that pass within a derived bound instead of calling them exact.
An OCR family on native HIP needs serving hooks that the text families never asked for, so the host draw can apply a checkpoint repetition penalty over prompt and reply tokens, OpenAI penalties and a forced final EOS on FP32 scores without rounding back to BF16, and images carry their visible size before padding for content-normalized positions. The prompt-cache capacity types move to engine_status.zig, which keeps engine_api.zig clear of the line limit.
Nemotron 3.5 Parse now serves on one native HIP rank: its RADIO tower, horizontal neck, causal mBART decoder and trained one-step proposal head run in native kernels behind the shared lane verifier, a built-in media template bridges chat image parts to the checkpoint's visual prefix, and prompt reuse opts in to keyed repeats so a page served again resumes from its image state. The family gives the host its image tower and prompt cache in Loaded.offer and its flag refusals in refusal(o), declares replies without reasoning, so all OCR text is answer content, and cancelling one request leaves the other admitted streams running. Host tests, focused HIP operator and position checks, a model-seam probe, the reference script's reference, positions and compare subcommands, and a recipe cover the port.
The registry test from #1933 checks the hook on two anonymous namespaces, so it does not name Nemotron Parse. A new registry test asserts that the registered Parse family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The oracle hashed only hf_nemotron_parse_modeling.py and accepted any checkpoint or package set. It now holds the SHA-256 of the model, processor and example code, configuration, tokenizer and model card at 4035ffb1d, and the reference versions of Torch, TorchVision, Transformers and timm, whose Block the encoder imports. The reference and positions commands refuse a difference before running checkpoint code. A new sources command checks a checkpoint and prints the pins.
The Parse oracle adapts run_transformers_inference.py from NVIDIA's release, and the native processor, tower, decoder and position kernel follow that release's processor, modeling and prefix-geometry files, so the notices name each file at the pinned revision with its OpenMDW-1.1 licence, and the timm Block the encoder's blocks follow.
The encoder block followed timm's Block, and library code stays out of TensorFold even with a notice. A writer that did not see timm's source or the old block wrote it again from the ViT paper (Eq. 2-3) and the checkpoint's tensor names, shapes and config, and checked the dataflow on CPU against NVIDIA's model as a black box. It makes the same helper calls in the same order, so the device work is unchanged. The timm lines leave THIRD_PARTY_NOTICES.md.
…text.zig stays at 600 lines Main's DeepSeek template handling and this branch's Parse bridge together took hf_text.zig to 606 lines. The bridge moves to its own file unchanged: with no --chat-template, a nemotron_parse checkpoint loads the bundled template, and every other model type is as before.
…he_plan
docs/api.md said /health's `memory` is `{}` on Metal. The Metal families with a memory reader (Nemotron, Qwen3.5-2B,
Qwen3.8-27B and Qwen3.5-MoE through qwen27_served.zig, Flash Next, GLM-5.3-Flash and GLM-5.3 (EXLR8)) answer `active`,
`cache` (0) and `peak` from the process footprint, as status_routes writes whatever the engine reads, and Flash Next and
both GLMs add `limit`, their load limit; only the families without a reader answer `{}`. docs/memory-diagnostics.md
listed Nemotron, Flash Next and Qwen3.8-27B as the families that report `prompt_cache_plan`: Qwen3.5-MoE (the 27B's
host), Gemma 4 (the plan when its budget is positive), LFM2.5-24B-A2B and LFM3-1.7B (the plan at any budget) and the 2B
with an explicit `--prompt-cache-gib` (the plan at any budget, `budget_bytes: 0` included) report it too. The row and
the zero-budget sentences now say what the hosts do.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #1941 #2439 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.