Repository navigation
Conversation
|
The #1938 review asked dots.mocr for pinned sources, a device receipt and a per-family reasoning assertion. This PR gets the same now, in two new commits on top (
Pinned sources at 4035ffb1d (SHA-256)
Device receipt
Hashes: ascii = sha256 of ids joined by
All 49 matrix cells and 6 supplementary sampled cells passed; all three server runs exited 0 after SIGTERM. Note: responses report The replies report Kernel contract and limits (the two new commits change no kernel)
Ordering: #1929, #1931 and #1933 are now on main as your versions, so this branch shows one conflict with main, in |
|
Thanks @laudney. One change, plus small points. The change: |
|
Thanks, Ash. Fixed in |
|
Thanks @laudney: the global |
Nemotron 3.5 Parse needs its own family before any device work, so this validates the Parse config graph and every stored tensor of the checkpoint, and ports its image processor with the same resize, white padding, normalization and patch order. A host test checks the processor against independent Pillow and NumPy fingerprints that the new nemotron_parse_reference.py processor subcommand prints, and the script keeps NVIDIA's OpenMDW notice with its license under LICENSES.
Image towers on native HIP need dense operators that the text kernels do not provide, so the ops image gains embedding, biased linear, layer norm, pointwise, layout, resize and content-normalized position kernels, with a vision handle that loads them. The kernel contract registers twelve new cases for gates 1 to 4, a record step compiles the ops image for every served RDNA chip so their launch records come from actual objects, and gate 4 now names FP32 sums that pass within a derived bound instead of calling them exact.
An OCR family on native HIP needs serving hooks that the text families never asked for, so the load options move into hip_serve with an image field, and the host draw can apply a checkpoint repetition penalty over prompt and reply tokens, OpenAI penalties and a forced final EOS on FP32 scores without rounding back to BF16. Images carry their visible size before padding for content-normalized positions, and the native host takes a family's penalties, image hook, prompt cache and raw-text capability when it declares them.
Nemotron 3.5 Parse now serves on one native HIP rank: its RADIO tower, horizontal neck, causal mBART decoder and trained one-step proposal head run in native kernels behind the shared lane verifier, a built-in media template bridges chat image parts to the checkpoint's visual prefix, and prompt reuse opts in to keyed repeats so a page served again resumes from its image state. The family declares replies without reasoning, so all OCR text is answer content, and cancelling one request leaves the other admitted streams running. Host tests, focused HIP operator and position checks, a model-seam probe, the reference script's reference, positions and compare subcommands, and a recipe cover the port.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name Nemotron Parse. A new registry test asserts that the registered Parse family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The oracle hashed only hf_nemotron_parse_modeling.py and accepted any checkpoint or package set. It now holds the SHA-256 of the model, processor and example code, configuration, tokenizer and model card at 4035ffb1d, and the reference versions of Torch, TorchVision, Transformers and timm, whose Block the encoder imports. The reference and positions commands refuse a difference before running checkpoint code. A new sources command checks a checkpoint and prints the pins.
The family flag check refused --vision for every HIP family without its own refusal, so Qwen 3.5, Qwen 3.5 MoE, Flash Next and Nemotron stopped at start where main served them text-only. Only Nemotron Parse's own check runs now; a family without an image tower starts text-only and logs that image requests are refused.
The note a text family prints under --vision had no test, and no test showed that the host then offers no image tower, which is what makes the server refuse image parts with a 400. The note now comes from one function that returns the exact line openWith prints, and a host test checks that line for Qwen 3.5, Qwen 3.5 MoE, Flash Next and Nemotron, that no note is printed without --vision or for GLM-OCR and Nemotron Parse, and that each text family's loaded engine gives the lane host a null vision offer.
f191e17 to
d08e659
Compare
|
Rebased on main
Resolutions against #1940:
Your small point: Checks on Device record (gfx1201, Radeon AI PRO R9700), binary sha256
One note for the order: #1938 and this PR both add |
Part of #1927.
This serves NVIDIA's
nvidia/Nemotron-3.5-Parse(revision4035ffb1d8f8bafcdb867dfaf6f7514f101464f9) natively on HIP. The checkpoint's original FP32 safetensors stay unchanged. The loader validates all 510 tensors and owns BF16 device copies of the 509 floating tensors. The RADIO image tower, the horizontal neck, the causal mBART decoder and the trained one-step proposal run in Zig and HIP, with no Python in the served engine. It is the checkpoint other engines publish numbers for and the one users download first, so serving the publisher's weights matters for benchmarking and adoption.nvidia/Nemotron-3.5-Parseon HIPtools/zig/, adapted from the model repository's examplePrerequisites #1929, #1931 (NFKC tokenizer) and #1933 (no-reasoning replies) are merged. This branch still carries their earlier copies at its base, so a rebase onto main leaves only the Parse commits,
7fb39aaed^..e29b3e2f3.Commits
tools/zig/nemotron_parse_reference.pywith itsprocessorcommand. The oracle carries NVIDIA's OpenMDW-1.1 SPDX header, andLICENSES/NVIDIA-OpenMDW-1.1.txtis added.feat(hip): shared dense vision operators. The ops image gains embedding, biased linear, layer norm, pointwise, layout, resize and content-normalized position kernels, with a vision handle that loads them. Twelve new kernel-contract cases cover gates 1-4, and a record step compiles the ops image for gfx1030, gfx1100, gfx1151 and gfx1201, so the launch records come from real objects. Gate 4 now names FP32 sums that pass within a derived bound, instead of calling them exact.feat(hip_serve): shared load options and FP32 score processing.hip_servewith an image field.docs/recipes/nemotron-parse.md, and the rest of the oracle (reference,positions,compare). It also sets.repeat_keyed = true(Prompt cache: an opt-in rule for families to count keyed image prompts as repeats #1929) andpub const reasoning = false;(Server: one engine flag for models whose replies carry no reasoning block #1933). Cancelling one request ends only that request, even inside a shared verification round.!reasoning(parse), and the oracle pins the SHA-256 of all 16 non-weight sources at4035ffb1dand the reference package versions, with asourcescommand that checks a checkpoint without a GPU.Known limits
Gates
SUMMARY gfx1201 64 kernels: 64 pass gates 1-4, 0 fail, 2 known findings; 5 controls: 5 red as planted, 0 missed; 124 limit problems. Theupstream/mainbase reports52 kernels ... 124 limit problemson the same GPU, with the same limit entries. An offline audit for four chips lists the same 503 entries asupstream/main.test-nemotron-parse-host,test-tool-parse-hostandtest-lane-api-hostpass.zig test zig/src/core/tokenizer/tokenizer.zig: 43/43.zig build test -Dhip=true: 497/497 steps.zig build test: every Parse step passes. One known-flaky CUDA disk test, which this PR does not touch, failed once and passed on rerun.native-hipfor gfx1201.