[WIP] Add explicit LinearExperts2D -> 3D MoE repack for HF-native saves - #3017
[WIP] Add explicit LinearExperts2D -> 3D MoE repack for HF-native saves#3017GOavi101 wants to merge 2 commits into
Conversation
Restore fused expert modules (weights + weight_* qparams) before save_pretrained so architectures like qwen3_vl_moe can reload without relying on transformers one-to-many WeightConverter support (vllm-project#2699). Signed-off-by: Avishek Goswami <avishek.goswami@ibm.com>
|
Important Review skippedIgnore keyword(s) in the title. ⛔ Ignored keywords (3)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Merge Protections🔴 2 of 2 protections blocking · waiting on 👀 reviews
🔴 Require one maintainer reviewWaiting for any of
This rule is failing.All PRs must have at least one approving review from a maintainer before merging.
🔴 Require two reviewsWaiting for
This rule is failing.PRs labelled "two-reviews" must have at least two approving reviews before merging.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to repack linearized 2D experts back into native fused 3D expert modules, enabling compatibility with Hugging Face's native key layout during model saving. Key feedback includes ensuring the newly initialized fused module is moved to the correct device and dtype to prevent device mismatch errors, avoiding direct modifications to .data in favor of PyTorch's .copy_() for safer in-place copying, and validating quantization parameters across all experts rather than just the first to prevent silent failures.
Move fused experts to the linearized module device/dtype before packing, use in-place copy_() instead of .data assignment, and require all experts to have a qparam before packing it. Signed-off-by: Avishek Goswami <avishek.goswami@ibm.com>
|
👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review. Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed. |
1 participant
Restore fused expert modules (weights + weight_* qparams) before save_pretrained so architectures like qwen3_vl_moe can reload without relying on transformers one-to-many WeightConverter support (#2699).