Use model-native video preprocessing for Qwen3-VL

[avisingh599]

Motivation

SGLang currently pre-resizes Qwen2/2.5 and Qwen3 video frames through the same factor-28 path. The processor stacks differ: Hugging Face's Qwen2 video processor uses 14-pixel patches, while Qwen3 has a dedicated Qwen3VLVideoProcessor with 16-pixel patches and a temporal-spatial resize budget. Consequently, Qwen3 frames are resized using the legacy geometry before being resized again by their model-native processor. See also the Qwen3-VL technical report.

Modifications

• Split Qwen3-VL/Qwen3.5 registration into qwen3_vl.py.
• Preserve SGLang's existing equally-spaced frame selection and max_frames behavior.
• Pass decoded Qwen3 frames at source geometry so the model processor owns spatial resizing.
• Add CPU regression coverage for Qwen2 and Qwen3 behavior.
• Memory: frames are now pinned at source resolution. Worst case at the 768-frame cap with 1080p is ~4.8 GB pinned uint8 vs ~0.6 GB pre-resized before.

Accuracy Tests

On an 18-example temporal-localization evaluation with eight samples per example, macro F1@1s/3s/5s changed from 0.4298/0.6851/0.7458 to 0.4628/0.6970/0.7655.

---

CI States

Latest PR Test (Base): ❌ Run #32759734996
Latest PR Test (Extra): ❌ Run #32759734721
Latest PR Test (AMD ROCm 7.2): ❌ Run #32759735088

[JustinTong0323]

Pushed two commits on top (hope that is okay — happy to split them out if you would rather take them as suggestions):

• c248d389 fixes two call sites the processor split left behind. NATIVE_MM_FAMILIES in rust_server.py matches the mm-processor class by identity, so qwen3_vl/qwen3_5 resolving to the new Qwen3VLImageProcessor no longer matched the qwen_vl family and SGLANG_RUST_SERVER=1 raised RuntimeError at launch (even for image-only traffic, the gate is per-model). Added a family entry for the new class plus a gate test going through get_mm_processor_cls. Also routed qwen3* model types in the EPD encoder preprocessor (disaggregation/encoder/preprocessor.py) to qwen3_vl.preprocess_video — it called the legacy factor-28 path for every qwen model type, so encoder-disagg deployments would keep the double-resize.
• baf9b759 trims the new preprocess tests: the fps=1/2 cases recomputed expected indices with the same smart_nframes + linspace the implementation calls (asserting the code against itself) and duplicated the hardcoded [0, 3, 6, 9] oracle; the pinned flags asserted an implementation detail. The three remaining tests pin the actual contract: Qwen3 keeps source geometry, max_frames still spans the full video, Qwen2 keeps the legacy resize.

Two things that need your call:

1. Qwen3-Omni stays on the legacy pre-resize path, but HF Qwen3OmniMoeProcessor feeds videos to the same Qwen3-VL-style video processor (32-factor, temporal-spatial budget), so the same double-resize applies there. Intentional (audio-in-video interleaving) or a follow-up?
2. Memory: frames are now pinned at source resolution. Worst case at the 768-frame cap with 1080p is ~4.8 GB pinned uint8 vs ~0.6 GB pre-resized before. Probably fine in practice given fps-based sampling, but worth a sentence in the PR description if you agree.

Non-blocking: the accuracy evidence (18 examples x 8 samples) is on the thin side for the F1 deltas claimed — a VideoMME-class number would make this much easier to stamp.

CI note: the red gates are just the missing run-ci label, all real tests are skipped until a maintainer adds it.

[avisingh599]

Hi @JustinTong0323

1. Agree that Qwen3-Omni should use the source-geometry path too.

2. "Memory: frames are now pinned at source resolution. Worst case at the 768-frame cap with 1080p is ~4.8 GB pinned uint8 vs ~0.6 GB pre-resized before." --> Added to PR description.