[NPU] keep Qwen3.6 hybrid SSM state stable across radix modes - #36096
Draft
McZyWu wants to merge 4 commits into
Draft
[NPU] keep Qwen3.6 hybrid SSM state stable across radix modes#36096McZyWu wants to merge 4 commits into
McZyWu wants to merge 4 commits into
Conversation
Contributor
Author
|
Machine-3 E2E validation completed on container sglwmc-821-m with Qwen3.6-35B-A3B, TP=2, Ascend attention, BF16 Mamba state, NEXTN enabled (3 steps), and the native AscendC conv1d fix from sgl-kernel-npu#742. The production test was copied byte-for-byte except port 8998 -> 8898 so it could not hit the unrelated sglwmc-823 service. Results:
Runtime verification:
The radix service remains healthy on container sglwmc-821-m, port 8898, devices 4/5. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary: restore the stable torch_npu NPU MRoPE path, separate the kernel intermediate-state chunk size from the radix checkpoint grid, and split an unaligned Mamba prefill tail even when ChunkCache disables radix caching. This prevents padding rows and incorrect h-state indices from advancing the Qwen3.6 GDN recurrent state. Companion native AscendC causal_conv1d fix: sgl-project/sgl-kernel-npu#742. Validation: on machine 152 with NEXTN enabled, radix cold/hot and recold passed 5 of 5, and no-radix passed 2 of 2 with identical stopped output. Unit coverage includes state-grid indexing and no-radix tail splitting. Machine 3 validation is pending because its SSH endpoint is currently unreachable.
CI States
Latest PR Test (Base): ❌ Run #32724871949
Latest PR Test (Extra): ❌ Run #32724871530
Latest PR Test (AMD ROCm 7.2): ❌ Run #32724871929