Skip to content

[NPU] keep Qwen3.6 hybrid SSM state stable across radix modes - #36096

Draft
McZyWu wants to merge 4 commits into
sgl-project:qwen_optimizefrom
McZyWu:codex/qwen36-0823-radix-correctness
Draft

[NPU] keep Qwen3.6 hybrid SSM state stable across radix modes#36096
McZyWu wants to merge 4 commits into
sgl-project:qwen_optimizefrom
McZyWu:codex/qwen36-0823-radix-correctness

Conversation

@McZyWu

@McZyWu McZyWu commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Summary: restore the stable torch_npu NPU MRoPE path, separate the kernel intermediate-state chunk size from the radix checkpoint grid, and split an unaligned Mamba prefill tail even when ChunkCache disables radix caching. This prevents padding rows and incorrect h-state indices from advancing the Qwen3.6 GDN recurrent state. Companion native AscendC causal_conv1d fix: sgl-project/sgl-kernel-npu#742. Validation: on machine 152 with NEXTN enabled, radix cold/hot and recold passed 5 of 5, and no-radix passed 2 of 2 with identical stopped output. Unit coverage includes state-grid indexing and no-radix tail splitting. Machine 3 validation is pending because its SSH endpoint is currently unreachable.


CI States

Latest PR Test (Base): ❌ Run #32724871949
Latest PR Test (Extra): ❌ Run #32724871530
Latest PR Test (AMD ROCm 7.2): ❌ Run #32724871929

@McZyWu

McZyWu commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

Machine-3 E2E validation completed on container sglwmc-821-m with Qwen3.6-35B-A3B, TP=2, Ascend attention, BF16 Mamba state, NEXTN enabled (3 steps), and the native AscendC conv1d fix from sgl-kernel-npu#742.

The production test was copied byte-for-byte except port 8998 -> 8898 so it could not hit the unrelated sglwmc-823 service.

Results:

  • Radix enabled: cold + 3 hot-prefix runs + cold-after-restart = 5/5 pass
  • Radix disabled: 3/3 pass
  • All 8 outputs: finish_reason=stop, 1319 SSE events, 11503 content chars, zero JSON parse errors, zero replacement characters
  • All 8 outputs were byte-identical after SSE reconstruction: SHA256 535bb041ed219fc53efde40bb1d83755437d48d521b6b0c3680c1f12a7529098
  • Radix hot runs reported 29056 cached tokens
  • Cold/no-radix physical chunks were 16384 + 12672 + 128, exercising the aligned Mamba tail path
  • Longest repeated suffix was 797 hyphens; this is explicitly accepted by the reporter and generation stopped normally

Runtime verification:

  • Tested functional SGLang HEAD: 3815a17
  • Current 488d387 only removes one isort-requested blank line
  • Loaded native library SHA256: 5fc99caf79b816955bf10026a65c59c403a34faedd5425524ecf6a125e08a8a8
  • /proc maps for the server and both TP workers all point to the fixed libsgl_kernel_npu.so

The radix service remains healthy on container sglwmc-821-m, port 8898, devices 4/5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant