Repository navigation

Integrate finegrained kernels: SOTA performance and per-module quantization - #49658

Draft

IlyasMoutawwakil wants to merge 101 commits into

mainfrom

finegrained-quantizer-redesign

Draft

Integrate finegrained kernels: SOTA performance and per-module quantization#49658
IlyasMoutawwakil wants to merge 101 commits into
mainfrom
finegrained-quantizer-redesign

Commits on Aug 13, 2026
- finegrained kernels
  IlyasMoutawwakil
  committed
  64708a2
Commits on Aug 15, 2026
- deepgemm and tp/ep fixes
  IlyasMoutawwakil
  committed
  7b21bdb
Commits on Aug 16, 2026
- finegrained: adopt the interleaved gate|up layout
  IlyasMoutawwakil
  committed
  05bd0f3
Commits on Aug 18, 2026
- finegrained: keep DeepGEMM Mega MoE on the stacked gate|up layout
  IlyasMoutawwakil
  committed
  686238e
- fix
  IlyasMoutawwakil
  committed
  641fcb2
- Merge remote-tracking branch 'origin/main' into finegrained-unified
  IlyasMoutawwakil
  committed
  e6b20c4
Commits on Aug 21, 2026
- finegrained: build the per-call kernel objects instead of caching them
  
  IlyasMoutawwakil
  and
  claude
  committed
  cb04cbe
- finegrained: fuse both MoE biases into the GEMMs
  
  IlyasMoutawwakil
  and
  claude
  committed
  e1c7a92
Commits on Sep 9, 2026
- Merge origin/main into finegrained-unified
  
  IlyasMoutawwakil
  and
  claude
  committed
  0a1b49b
- Merge remote-tracking branch 'origin/main' into finegrained-unified
  IlyasMoutawwakil
  committed
  9041320
Commits on Sep 11, 2026
Commits on Sep 13, 2026
- Revert "finegrained: pass input_recipe to the dispatchers, drop the Quantization/Epilogue bundle fields"
  
  IlyasMoutawwakil
  and
  claude
  committed
  0b6ca4c
Commits on Sep 14, 2026
- finegrained: pass activation_format straight through; let deep-gemm skip the kernels arch check
  
  IlyasMoutawwakil
  and
  claude
  committed
  513e231
- hub_kernels: finegrained-kernels ships as version 0
  
  IlyasMoutawwakil
  and
  claude
  committed
  240abc9
- hub_kernels: load finegrained-kernels from the kernels-staging build (pr-1018)
  
  IlyasMoutawwakil
  and
  claude
  committed
  30ab2c2
Commits on Sep 15, 2026
- finegrained: consume modelopt's calibrated globals, and fuse a model's per-expert output norm
  
  IlyasMoutawwakil
  and
  claude
  committed
  a562fb8
- finegrained: address the integration review
  
  IlyasMoutawwakil
  and
  claude
  committed
  89267bf
- finegrained: freeze the mxfp4 and nvfp4 integrations, and serve what they served
  
  IlyasMoutawwakil
  and
  claude
  committed
  09ea21c
- Merge branch 'main' into finegrained-unified
  IlyasMoutawwakil
  committed
  d59f903
- fix style
  IlyasMoutawwakil
  committed
  e139d2a
- FineGrainedFP8Config is its own class again, not an alias of the new one
  
  IlyasMoutawwakil
  and
  claude
  committed
  c665644
- FineGrainedFP8Config: drop the frozen-notice docstring line
  
  IlyasMoutawwakil
  and
  claude
  committed
  29cca04
Commits on Sep 16, 2026
- Tolerate an experts module without the norm, and a machine without CUDA
  
  IlyasMoutawwakil
  and
  claude
  committed
  f63ad97
- Pin that every modelopt converter target is a slot the module holds
  
  IlyasMoutawwakil
  and
  claude
  committed
  1265f0c
- is_sm100: answer False without CUDA, and fake both halves in the tests
  
  IlyasMoutawwakil
  and
  claude
  committed
  97dfa10
- fix
  IlyasMoutawwakil
  committed
  788d02d
- finegrained: drop the DTensor unwrap, and pass the kernel operands by name
  
  IlyasMoutawwakil
  and
  claude
  committed
  53a8970
- deepgemm: only Mega MoE cannot apply a post-expert norm
  
  IlyasMoutawwakil
  and
  claude
  committed
  e8e6b87
Commits on Sep 17, 2026
- Shard quantized expert companions under EP and TP
  
  IlyasMoutawwakil
  and
  claude
  committed
  43b9a74
- fixes for the new model
  IlyasMoutawwakil
  committed
  fe0b718
- fix multimodal and add some testing
  IlyasMoutawwakil
  committed
  22b1001
Commits on Sep 18, 2026
- static
  IlyasMoutawwakil
  committed
  f9302cc
- Load a calibrated checkpoint's input_scale, and fix what per-tensor weights reach
  
  IlyasMoutawwakil
  and
  claude
  committed
  daa2420
- fix
  IlyasMoutawwakil
  committed
  b4dcb7b
Commits on Sep 21, 2026
- Finish the finegrained integration: reload, TP sharding, dsv4 norms
  
  IlyasMoutawwakil
  and
  claude
  committed
  f7081ec
- Fix a destructive Qwen3 TP-plan override; split out the conversions
  
  IlyasMoutawwakil
  and
  claude
  committed
  9ddb214
- Collapse the conversion-op boilerplate; shorten the longest units
  
  IlyasMoutawwakil
  and
  claude
  committed
  e83eadf
Commits on Sep 22, 2026
- Key on capability not name; drop the last op boilerplate
  
  IlyasMoutawwakil
  and
  claude
  committed
  60f904e
- Quantize per module group; split the arms by producer, not by format
  
  IlyasMoutawwakil
  and
  claude
  committed
  4cb5d9c
- Carry the quantized-load fixes onto the per-scheme arms, and the same simplification
  
  IlyasMoutawwakil
  and
  claude
  committed
  a67e7ac
Commits on Sep 23, 2026
- Address the per-scheme review
  
  IlyasMoutawwakil
  and
  claude
  committed
  9fc683d
- trim
  IlyasMoutawwakil
  committed
  95b3e44
- fixes
  IlyasMoutawwakil
  committed
  2392e6a
- last
  IlyasMoutawwakil
  committed
  f7a8e54
- Cover the groups feature, and export what it asks callers to build
  
  IlyasMoutawwakil
  and
  claude
  committed
  4cbbe7f
- Point everything at the fine-grained integration; deprecate the frozen files
  
  IlyasMoutawwakil
  and
  claude
  committed
  f687860
- Bring two comment blocks under the noisy-comment limits
  
  IlyasMoutawwakil
  and
  claude
  committed
  fab7f74
- Bring two comment blocks under the noisy-comment limits
  
  IlyasMoutawwakil
  and
  claude
  committed
  21c0a07
- Do not let the tied-head plan raise in post_init
  
  IlyasMoutawwakil
  and
  claude
  committed
  12f5aa8
- Do not let the tied-head plan raise in post_init
  
  IlyasMoutawwakil
  and
  claude
  committed
  7abcedf
Commits on Sep 24, 2026
- Address the second review round
  
  IlyasMoutawwakil
  and
  claude
  committed
  db85a44
- Merge finegrained-replace-frozen-fp8 into finegrained-per-scheme-demo
  
  IlyasMoutawwakil
  and
  claude
  committed
  d5c361c
- Carry the review's comment and lint fixes to the sites it did not name
  
  IlyasMoutawwakil
  and
  claude
  committed
  9f7326c
- Make tests/kernels a package so the moved fp8 test can import its fixtures
  
  IlyasMoutawwakil
  and
  claude
  committed
  f1b593d
- Declare GLM-4v-MoE's head plan instead of deriving it in the mixin
  
  IlyasMoutawwakil
  and
  claude
  committed
  0067189
- cleanup guards and post expert norm in deepgemm
  IlyasMoutawwakil
  committed
  bb24085
- not now
  IlyasMoutawwakil
  committed
  39eb1d1
- minimal peft integration preparation
  IlyasMoutawwakil
  committed
  bf0e7ec
Commits on Sep 25, 2026
- addressing review comments
  IlyasMoutawwakil
  committed
  5b2b7f0
- Tiny checkpoints for every shipped layout, and the save/load fixes they found
  
  IlyasMoutawwakil
  and
  claude
  committed
  f3c7cc8
- Keep pre-quantized scale dtypes through the dtype plan
  
  IlyasMoutawwakil
  and
  claude
  committed
  6a7c92e
Commits on Sep 30, 2026
- Update src/transformers/distributed/sharding_utils.py
  
  IlyasMoutawwakil
  and
  ArthurZucker
  authored
  2d1f0bc
- Update src/transformers/distributed/sharding_utils.py
  
  IlyasMoutawwakil
  and
  ArthurZucker
  authored
  f37d1d4
- adapters
  IlyasMoutawwakil
  committed
  5ea2a3e
Commits on Oct 2, 2026
- Fine-grained layout ops: follow the experts module each tensor fills
  
  IlyasMoutawwakil
  and
  claude
  committed
  c766206
- Revert "adapters"
  
  IlyasMoutawwakil
  and
  claude
  committed
  7d44b81
- trimming
  IlyasMoutawwakil
  committed
  eb9c09f
Commits on Oct 5, 2026
- Fine-grained NVFP4: exact per-expert gate_up activation globals
  
  IlyasMoutawwakil
  and
  claude
  committed
  77f135c
Commits on Oct 7, 2026
Commits on Oct 15, 2026
- Merge origin/main into finegrained-quantizer-redesign
  
  IlyasMoutawwakil
  and
  claude
  committed
  504d2c3
Commits on Oct 16, 2026
Commits on Oct 17, 2026

Commits

Commits on Aug 13, 2026

Commits on Aug 15, 2026

Commits on Aug 16, 2026

Commits on Aug 18, 2026

Commits on Aug 21, 2026

Commits on Sep 9, 2026

Commits on Sep 11, 2026

Commits on Sep 13, 2026

Commits on Sep 14, 2026

Commits on Sep 15, 2026

Commits on Sep 16, 2026

Commits on Sep 17, 2026

Commits on Sep 18, 2026

Commits on Sep 21, 2026

Commits on Sep 22, 2026

Commits on Sep 23, 2026

Commits on Sep 24, 2026

Commits on Sep 25, 2026

Commits on Sep 30, 2026

Commits on Oct 2, 2026

Commits on Oct 5, 2026

Commits on Oct 7, 2026

Commits on Oct 15, 2026

Commits on Oct 16, 2026

Commits on Oct 17, 2026