GLM-5.3-Flash on CUDA: prompt_logprobs on /v1/completions

[Bizuayeu]

What this changes

Part of GLM-5.3-Flash on the native CUDA engine (#593, where the three stacked PRs for GLM's ranks are summed up), and #298's request on that engine: scoring a fixed text, so quantizations, engines and kernel changes can be compared by NLL.

GLM-5.3-Flash's CUDA lanes fill Stream.prompt_rows, so nonstreamed /v1/completions serves vLLM's prompt_logprobs on GLM instead of HTTP 400. The rows come from each prompt chunk's final-norm rows: every rank runs them through its head columns, 128 rows a pass; Sampler.gather makes the rows whole on every rank (the grouped broadcasts a sampled draw already uses), and rank 0 runs sample.cu's tf_logprob_rows on them, as Nemotron's prompt rows do. Rank 0's prefill and chunk steps carry flag 1 so the other ranks join each pass in the same order. The passes' buffers (about 60 MB a rank at TP=2) are made by the first request that asks; if a rank cannot make them, an all-reduce sends every rank to the verify window's buffers, 8 rows a pass. A request with prompt rows prefills from position 0.

Receipt

• Environment: 28f3e930 on 9ccf4727, Zig 0.17.0, CUDA 13.3 (nvcc 13.3.73), NCCL 2.30.7, DGX Spark (GB10, sm_121), one GPU a host over RoCE, checkpoint nvidia/GLM-5.3-Flash-NVFP4 at 423acf37; TP=2 with --parallel 2 --context 131072 (two streams do not fit 245,760 at TP=2: 4.55 GiB against the 4.41 GiB rank 1 has left).
• Exactness: the same request twice and with "draft": false give the same rows. A 5,000-token prompt has a row for every token after the first, and its first chunk's 2,048 rows equal those of the same prompt cut at 3,001 tokens. A prompt-rows request beside a plain one (two streams) gives its solo rows, and the plain one keeps its token_sha. A prompt followed by its own 48-token greedy reply ranks 46 reply tokens first; one ties the first at the same logprob, and one trails by 0.125 (one bf16 step of a logit between 16 and 32), since decode windows and prompt chunks reach the final norm through different kernels.
• Teacher-forced NLL (score-nll over a 16-text set, each text twice; self-agreement: largest logprob move 0.0, argmax agreement 1.0): ja 2.5293, en 2.9109, code 1.3138, math 0.6272. The Python engine (a fork build of the frozen 0.6 line, GLM's CUDA family) on the same checkpoint at TP=2: 2.5474, 2.9257, 1.3184, 0.6250.
• Decode speed before and after: n/a, the decode path is unchanged (a plain request keeps its token_sha, 09a15f0582e7 before and after).
• Prompt speed before and after: a request without prompt_logprobs takes the path it took; with them a 5,000-token prompt's prefill_seconds was 3.97 against 3.68 without (+8%).
• Tests run, at 28f3e930 and at 9ccf4727 with fresh caches: zig build test-cuda-host 854/854 (852/852 at main; the two new tests), zig build test 1152/1159 with 7 skipped (859/866 at main; the count moves with the build cache, 0 failures in both), zig build and zig build native, zig fmt --check zig/src/families/glm, git diff --check, lean: 517 problems, 517 on the base, 0 new.
• Also at TP=3, the same commits on 1.0.12 with two unrelated local changes (rank 0's NCCL id handed to the others in a file, which three hosts on /32 addresses need until fabric: tensor parallelism across CUDA processes (bootstrap, NCCL, mailbox and RoCE collectives, plan link, fail-fast) (DeepSeek-V4.1 stack 3/14) #636, and a heat-wait ceiling), --context 1048576, one stream: the same row checks pass (the 5,000-token prompt's prefill_seconds 3.71 against 3.34 without rows); 46 of the 48 greedy reply tokens rank first (one ties the first at the top, one trails by 0.125); NLL ja 2.5246, en 2.9206, code 1.3147, math 0.6241, self-agreement 0.0 and 1.0.
• Not run, and why: Metal (the change is the CUDA lanes'), other CUDA GPUs.

Checklist

• ✓ Every token still goes through the lane rounds (the rows ride the prompt pass; no serial path; drafts stay on).
• ✓ Drafted output equals "draft": false (the same rows); a request with prompt rows prefills fresh by design; concurrent equals solo (two streams).
• ✓ No precision traded for speed: decode and prompt arithmetic are unchanged; the rows use the head's bf16 matmul and the existing logprob kernel.
• ✓ No new or changed GPU kernel (tf_logprob_rows and the bf16 matmul are existing ones).
• ✓ Prompt processing is no slower without prompt_logprobs (the path is unchanged; the plain token_sha holds).
• ✓ Platforms: CUDA GB10 at TP=2 and TP=3; Metal and other CUDA GPUs not run.
• ✓ New tests: the step flag is read on prefill and chunk steps only, and the head's columns at TP=2 and TP=3 take no K slices, so a pass of any row count needs no partial sums. They test new code, so there is no failing "before".
• ✓ Comments and docstrings are one line.
• ✓ No personal data, machine names, internal hosts or local paths; no AI attribution lines.