[Arminova]

What this changes

Qwen/Qwen3-Embedding-0.6B, -4B and -8B load on the native CUDA engine from the published BF16 weights and
serve POST /v1/embeddings. main refuses them on CUDA because qwen3 has no CUDA backend. Part 1 of the plan in #2261.
Part 2, the rerankers, stacks on this one.

Two commits:

1. qwen3: checkpoint names, the folder's head and the prompt lane backend move out of the HIP files.
   names.zig, head.zig and lane.zig take the parts that import no GPU runtime, so a second backend can share
   them. model.zig, native.zig and prompt.zig keep their names as aliases. The HIP engine and its tests do not
   change.
2. qwen3 cuda: Qwen3 embedding checkpoints on the native CUDA engine, in families/qwen3/cuda/:
   • BF16 weights as stored.
   • The batched prompt pass on CUDA's shared rows, with no new kernels: residual RMS norms, QK norm with RoPE into
     one cache, and Nemotron's GQA chunks one row block at a time.
   • Projections on cuBLASLt with one algorithm per (n, k), chosen once. Split-K and reduction schemes are refused,
     so each input's rows keep the bits they have alone, in any batch.
   • The engine is lane.zig's backend. cuda/native.zig opens a sentence-transformers folder as the HIP family
     does: the head comes from the folder, the window from the memory budget.
   • The engine takes Rerank on HIP: compute the shared trunk once a pass (25% faster, same bytes) #2262's trunk argument and refuses a nonzero trunk, which embeddings never send.
   • A reranker folder is refused on CUDA with a message, until part 2.

Shared files:

• cuda/cublaslt.zig gains three attribute ids from cublasLt.h and cublasLtMatmulAlgoConfigGetAttribute.

• native/cuda.zig gets a qwen3 entry. LFM2's refusal moves into native/lfm2.zig's open, so the file stays at
  600 lines.

• Issue: Qwen3 embedding and reranker checkpoints (BF16) on the native CUDA engine #2261 (claimed).

• Base: main 1d891001b. Merges clean with today's main c76c302e4.

• atc overlap --json: not run, because ATC is not installed here. I did the same test by hand: git merge-tree of
  this head against every open pull request whose files it shares (heads fetched today about 19:00Z).

  • New conflicts appear only where entries are added next to each other: native/cuda.zig with MoE route tap: optional TF_ROUTE_TAP trace of global top-k #1886, Living Weights on Qwen4 dense, step 2: the native CUDA and Metal learner (experimental, --slide refused) #2054, Living Weights on Gemma 4, step 2: the native GeGLU learner on CUDA and Metal (experimental, --slide refused) #2062
    and Unlimited-OCR-2 on CUDA: the family's steps on NVIDIA GPUs, the router's long-context speedup (stacked on #2074) #2077, and build/cuda_engines.zig with cluster: serve --cluster brings a GB10 and a Mac up as one cluster over ConnectX and agrees on a disaggregated model's plan, before any tokens run (#1594, step 2a) #2088.
  • Nothing new with the qwen3 files.

• Related: qwen3-asr: host slice, the shared tower and Qwen3 schedules over a backend binding (#2105) #2239 (@jschmied, Qwen3-ASR) carries a Qwen3 language model on CUDA for one sequence and the ASR
  checkpoint's names. This one serves the plain Qwen3 checkpoints, with many independent inputs in a pass.

Receipt

• Environment: one GB10 (sm_121), driver 580.173.02, nvcc 13.3.73, --tp 1, --context 32768.

  • Checkpoints: Qwen/Qwen3-Embedding-0.6B at 97b0c614, -4B at 5cf2132a, -8B at 1d8ad4ca.

• Builds:

  • All three sizes ran on 0daf91f3f. That is this change on main before Rerank on HIP: compute the shared trunk once a pass (25% faster, same bytes) #2262 landed. Its qwen3/cuda/ files
    equal this head's, except the trunk argument in engine.zig and its test's call.
  • The 8B ran again on 16cd92a8e, part 2's head, which contains this head. It gave the same distances to the last
    digit, on every input.

• Upstream's pinned oracle, tools/zig/oracle_qwen3_embedding.py (transformers 5.22.1, 1d8ad4ca; fp32 reference on
  the CPU, BF16 bar on the GPU): check distance 9.70e-05, norm 1.000000, the card's scores within 0.0013, PASS.

• Our own comparison: transformers 5.15 in fp32 on the GPU as the reference, with 8 inputs of 8, 23, 27, 31, 2,048,
  8,192, 16,384 and 30,000 tokens.

  • Every vector is closer to the fp32 vector than transformers' own BF16 run of the same input.
  • All norms are 1.000000.

  	worst cosine distance to fp32	transformers BF16's worst	short documents a second
  0.6B	1.55e-4	1.88e-4	628
  4B	1.34e-4	1.79e-4	249
  8B	1.08e-4	1.77e-4	185

  Speed is for batches of 32 from one client; the 8B gave 189 on 16cd92a8e. main has no number to compare,
  because it refuses these checkpoints on CUDA.

• Exactness, on every size:

  • The 8 inputs in one request give the same bytes as each one alone.
  • 4 clients sending 5 requests each at once: 20 of 20 equal their bytes alone.
  • Text input equals the same token ids sent as ids: 4 of 4.
  • dimensions (32, 1,024 and full): each vector equals the full one cropped and renormalised, within 1.8e-8.

• Not applicable: drafts and resumed prompts. This is a prompt-only engine with no draft and no prompt cache.

• Prompt speed: no number to compare, because main refuses these checkpoints on CUDA. The documents-a-second
  column above is this change's prompt pass.

• Leaks: no server reported one on exit, including the one that refused a reranker folder.

• Tests:

  • zig build test -j10: 549 / 549 steps; 7,131 of 7,190 tests passed, 59 skipped. That includes
    test-qwen3-checks-host, the GPU check's oracle pins, references and batching without a device.
  • zig build test-cuda-host: 313 / 313 steps; 4,941 of 4,964 tests passed, 23 skipped.

• Lean checker on the base and this head: 425 / 425, 0 new.

• zig fmt --check zig build.zig: clean.

• Not run:

  • qwen3-cuda-gpu-test: it is compiled in zig build test, and the served checks above cover the same pass.
  • HIP on a device: we have no AMD GPU. Its host tests pass. The split moves code into the shared files, and the HIP
    engine's own files (engine.zig, forward.zig) do not change.
  • Other CUDA GPUs, TP2, Windows and macOS: no runner here.

• Licence of new weights: n/a.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Concurrent equals solo, and a batch equals its inputs alone. Drafts and resumed prompts are not applicable,
  because the engine is prompt-only.
• No precision traded for speed: BF16 weights as published, f32 sums, and closer to fp32 than transformers' BF16.
• No new GPU kernel. It uses CUDA's shared row kernels and cuBLASLt.
• Prompt processing on main is unchanged for every other family.
• Platforms: CUDA on one GB10. Not run: other CUDA GPUs, Metal (no qwen3 Metal backend is touched), and HIP on
  a device.
• New tests pass. The GPU check skips cleanly without its checkpoint and references (TF_QWEN3_EMBED_*).
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[ashhart]

Thanks @Arminova, this is a careful first part: the family's backend-neutral pieces split out with HIP unchanged, BF16 weights as published, no new kernels, cuBLASLt pinned to one algorithm per shape so a batch keeps each input's bits, and every vector closer to fp32 than Transformers' own BF16, plus the pinned oracle passing on the 8B. Whether we take a Qwen3 backend on CUDA is still an open decision on our side, so we can't say yet whether this lands. Part of that is how it sits beside #2239, which carries a Qwen3 language model on CUDA for Qwen3-ASR: we'd want one Qwen3 CUDA forward, not two. We'll come back on #2261 once that's decided. Until then there's nothing you need to change here.