[sglang-miles] Add SGLANG_DISABLE_MULTIMEM_AG to force the NCCL all-gather path - #36110
Merged
Merged
Conversation
yueming-yuan
requested review from
ch-wan,
merrymercy and
yizhang2077
as code owners
August 23, 2026 21:26
yueming-yuan
force-pushed
the
yueming/multimem-ag-env-gate
branch
from
August 24, 2026 06:48
d7906d4 to
013aa7f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
On GB300 NVL72 running 8 colocated tp8 engines (miles RL, DeepSeek-V4-Flash-0731), the multimem all-gather's symmetric-memory rendezvous is unreliable when many engines initialize concurrently: some ranks time out after 600s ("multimem all-gather disabled (wait timeout ...)") and others deadlock inside the rendezvous right after "Registering N cuda graph addresses", which blocks engine init forever (the ray init handle never resolves and the whole colocate bring-up hangs).
Modifications
Add an
SGLANG_DISABLE_MULTIMEM_AGenv (default off, no behavior change) that forcesMultimemAllGathererto the NCCL fallback path, skipping the symm-mem rendezvous entirely. Correctness is unaffected — the NCCL path is the existing fallback the gatherer already uses whenever multimem is unavailable.Validated on the 8-node GB300 bringup: with the gate set, all 8 engines initialize reliably and training runs (6+ consecutive rollout/train cycles).
CI States
Latest PR Test (Base): ❌ Run #32698829006
Latest PR Test (Extra): ❌ Run #32698828336
Latest PR Test (AMD ROCm 7.2): ➖ No AMD PR run found for this commit.