Skip to content

feat: Intel XPU (Arc Pro B70) training support for DSpark drafters - #769

Open
rmacy wants to merge 1 commit into
sgl-project:mainfrom
rmacy:xpu-training-support
Open

feat: Intel XPU (Arc Pro B70) training support for DSpark drafters#769
rmacy wants to merge 1 commit into
sgl-project:mainfrom
rmacy:xpu-training-support

Conversation

@rmacy

@rmacy rmacy commented Aug 16, 2026

Copy link
Copy Markdown

Summary

Adds Intel XPU device support to SpecForge offline DSpark training, enabling drafter training on Intel Arc Pro B70 (Battlemage) GPUs.

Changes

  • Device detection: XPU recognized alongside CUDA; xccl backend selected for distributed init
  • Attention backend: eager fallback (flex_attention is CUDA-only; flash_attn unavailable on XPU)
  • Memory management: CPU optimizer offload for memory-constrained XPU; pin_memory guard (not supported on XPU)
  • DDP/RNG fixes: single-GPU XPU runs work correctly
  • Qwen3.8-27B DSpark config: draft model config for the 1.36B DSpark architecture
  • Example offline training config: ready-to-use YAML for XPU training

Training constraints discovered (XPU-specific)

  • num_anchors <= 64 and objective_chunk_blocks <= 16 (larger silently OOMs -> NaN loss)
  • VLLM_XPU_ENABLE_XPU_GRAPH=0 required (graphs hang xe engines)
  • dataloader_num_workers=0 (workers cause hangs on XPU)

Verification

Trained a 1.36B DSpark drafter for Qwen3.8-27B FP8 on 2x Arc Pro B70 (TP=2):

  • 440 steps, lr 5e-5, warm-start from released RadixArk drafter
  • Result: 72.2 tok/s median on isolated C1 benchmark (vs 32.4 no-spec, 54.67 MTP2)
  • Pos-0 acceptance: 62-74%, mean acceptance length 2.5-3.5

Adds XPU device support to SpecForge offline DSpark training:
- Device detection and xccl backend selection for XPU
- Eager attention fallback (flex_attention is CUDA-only)
- CPU optimizer offload for memory-constrained XPU
- DDP/RNG fixes for single-GPU XPU runs
- pin_memory guard (not supported on XPU)

Also adds Qwen3.8-27B DSpark config and example offline training config.

Verified: trained a 1.36B DSpark drafter for Qwen3.8-27B FP8 on 2x Arc Pro
B70, achieving 72.2 tok/s median on isolated C1 (vs 32.4 no-spec, 54.67 MTP2).
@jiapingW

Copy link
Copy Markdown
Collaborator

Hi, thank you very much for your implementation on Intel XPU. Since we lack access to XPU hardware, if you wish to add support for it, please provide the training loss and model performance results to demonstrate its effectiveness. Ideally, please also open-source the draft model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants