IlyasMoutawwakil · Aug 13, 2026 · Aug 15, 2026 · Aug 16, 2026 · Aug 18, 2026 · Aug 18, 2026
diff --git a/docs/source/en/_toctree.yml b/docs/source/en/_toctree.yml
@@ -289,18 +289,24 @@
       title: torchao
     title: Quantize on the fly
   - sections:
+    - local: quantization/blockfp8
+      title: Block FP8
     - local: quantization/fbgemm_fp8
       title: FBGEMM
-    - local: quantization/finegrained_fp8
-      title: Fine-grained FP8
     - local: quantization/fouroversix
       title: Four Over Six
     - local: quantization/fp_quant
       title: FP-Quant
     - local: quantization/mxfp4
       title: MXFP4
+    - local: quantization/mxfp8
+      title: MXFP8
     - local: quantization/nvfp4
       title: NVFP4
+    - local: quantization/finegrained
+      title: Fine-grained
+    - local: quantization/finegrained_fp8
+      title: Fine-grained FP8 (deprecated)
     title: FP4 and FP8 formats
   - sections:
     - local: quantization/auto_round

diff --git a/docs/source/en/experts_interface.md b/docs/source/en/experts_interface.md
@@ -95,7 +95,7 @@ model = AutoModelForCausalLM.from_pretrained(
 
 ## DeepGEMM
 
-The `"deepgemm"` backend routes expert matmuls through the [DeepGEMM](https://github.com/deepseek-ai/DeepGEMM) kernels distributed by [kernels-community/deep-gemm](https://huggingface.co/kernels-community/deep-gemm). It works with unquantized `bfloat16` experts and with FP8/FP4-quantized experts loaded through [Fine-grained FP8](./quantization/finegrained_fp8).
+The `"deepgemm"` backend routes expert matmuls through the [DeepGEMM](https://github.com/deepseek-ai/DeepGEMM) kernels distributed by [kernels-community/deep-gemm](https://huggingface.co/kernels-community/deep-gemm). It works with unquantized `bfloat16` experts, with [Block FP8](./quantization/blockfp8) experts (128x128 blocks), and with [MXFP4](./quantization/mxfp4) experts run with FP8 activations (DeepSeek-V4's W4A8, Blackwell only), including a [Fine-grained](./quantization/finegrained) plan of them. On quantized experts it quantizes activations to FP8 at run time whatever their `activation_format`, and adds no expert biases; use `grouped_mm` or `batched_mm` for MXFP8 or NVFP4 experts, biased experts, or another activation format.
 
 The `"deepgemm"` backend requires:
 
@@ -117,7 +117,7 @@ The kernel is loaded lazily on the first forward.
 
 ### FP8 and FP4 quantized experts
 
-DeepSeek-style checkpoints are usually pre-quantized and carry their own quantization config, so you don't need to pass a [`FineGrainedFP8Config`]. The `"deepgemm"` backend automatically picks the FP8 (or FP4 on Blackwell) grouped-GEMM kernel. DeepGEMM requires dynamic per-row activation scales (`activation_scheme="dynamic"`) and rejects static (per-tensor) activation quantization.
+DeepSeek-style checkpoints are usually pre-quantized and carry their own quantization config, so you don't need to pass a [`BlockFp8Config`]. The `"deepgemm"` backend automatically picks the FP8 (or FP4 on Blackwell) grouped-GEMM kernel. DeepGEMM requires dynamic per-row activation scales (`activation_scheme="dynamic"`) and rejects static (per-tensor) activation quantization.
 
 ```py
 from transformers import AutoModelForCausalLM
@@ -133,14 +133,14 @@ For FP4-packed expert weights (DeepSeek V4-style), the GPU must be SM100+ (Black
 > [!NOTE]
 > On Blackwell (SM100+), the `"deepgemm"` and `"deepgemm_megamoe"` experts kernels require power-of-two UE8M0 expert scales. A checkpoint quantized with plain `float32` scales (`scale_fmt="float"`) raises a `ValueError` on the first forward instead of silently corrupting the output. Load a checkpoint quantized with `scale_fmt="ue8m0"`, or switch to `grouped_mm` or `batched_mm`, which consume `float32` block scales directly. Hopper (SM90+) consumes `float32` scales on the DeepGEMM path without conversion.
 
-The main reason to pass a [`FineGrainedFP8Config`] for a pre-quantized checkpoint is to dequantize it back to `bfloat16`, in which case the experts run in `bfloat16` rather than on the FP8/FP4 DeepGEMM path.
+The main reason to pass a [`BlockFp8Config`] for a pre-quantized checkpoint is to dequantize it back to `bfloat16`, in which case the experts run in `bfloat16` rather than on the FP8/FP4 DeepGEMM path.
 
 ```py
-from transformers import AutoModelForCausalLM, FineGrainedFP8Config
+from transformers import AutoModelForCausalLM, BlockFp8Config
 
 model = AutoModelForCausalLM.from_pretrained(
     "deepseek-ai/DeepSeek-V3",
-    quantization_config=FineGrainedFP8Config(dequantize=True),
+    quantization_config=BlockFp8Config(dequantize=True),
     experts_implementation="deepgemm",
 )
 ```

diff --git a/docs/source/en/main_classes/quantization.md b/docs/source/en/main_classes/quantization.md
@@ -70,14 +70,6 @@ Learn how to quantize models in the [Quantization](../quantization/overview) gui
 
 [[autodoc]] MetalConfig
 
-## Mxfp4Config
-
-[[autodoc]] Mxfp4Config
-
-## NVFP4Config
-
-[[autodoc]] NVFP4Config
-
 ## FbgemmFp8Config
 
 [[autodoc]] FbgemmFp8Config
@@ -98,8 +90,37 @@ Learn how to quantize models in the [Quantization](../quantization/overview) gui
 
 [[autodoc]] SpQRConfig
 
+## BlockScaledConfig
+
+[[autodoc]] BlockScaledConfig
+
+## BlockFp8Config
+
+[[autodoc]] BlockFp8Config
+
+## Mxfp8Config
+
+[[autodoc]] Mxfp8Config
+
+## Mxfp4Config
+
+[[autodoc]] Mxfp4Config
+
+## NVFP4Config
+
+[[autodoc]] NVFP4Config
+
+## FineGrainedConfig
+
+[[autodoc]] FineGrainedConfig
+
 ## FineGrainedFP8Config
 
+> [!WARNING]
+> `FineGrainedFP8Config` is deprecated and will be removed in a future release. Passed to `from_pretrained`, it loads
+> through its own frozen integration; use [`BlockFp8Config`], or [`Mxfp8Config`] for a `[1, 32]` block with UE8M0
+> scales (see [Fine-grained FP8](../quantization/finegrained_fp8)).
+
 [[autodoc]] FineGrainedFP8Config
 
 ## QuarkConfig

diff --git a/docs/source/en/model_doc/deepseek_v32.md b/docs/source/en/model_doc/deepseek_v32.md
@@ -55,23 +55,13 @@ In DeepSeek-V3.2 **every layer runs its own indexer** — there is no cross-laye
 
 ## Usage examples
 
-DeepSeek-V3.2-Exp is distributed as an FP8 checkpoint. The indexer projections are kept out of FP8 quantization, since the checkpoint stores them in bf16/fp32:
+DeepSeek-V3.2-Exp is distributed as an FP8 checkpoint, loaded from its own quantization config. The router and the indexer's `weights_proj` stay in full precision:
 
 ```python
-from transformers import FineGrainedFP8Config, AutoModelForCausalLM, AutoTokenizer
-import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
 model_name = "deepseek-ai/DeepSeek-V3.2-Exp"
-quantization_config = FineGrainedFP8Config(
-    modules_to_not_convert=["model.layers.*.mlp.gate.*", "*.self_attn.indexer.weights_proj.*"],
-    weight_block_size=(128, 128),
-)
-model = AutoModelForCausalLM.from_pretrained(
-    model_name,
-    torch_dtype="auto",
-    device_map="auto",
-    quantization_config=quantization_config,
-)
+model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
 tokenizer = AutoTokenizer.from_pretrained(model_name)
 
 inputs = tokenizer("What are we having for dinner?", return_tensors="pt").to(model.device)

diff --git a/docs/source/en/model_doc/deepseek_v4.md b/docs/source/en/model_doc/deepseek_v4.md
@@ -29,6 +29,41 @@ share the same architecture; they differ only in width / depth / expert count an
 > [!TIP]
 > Set `use_kernels=True` in [`~PreTrainedModel.from_pretrained`] to replace supported layers with optimized kernels from the Hub. Refer to [Loading kernels](../kernel_doc/loading_kernels) to learn more.
 
+## Usage examples
+
+DeepSeek-V4 checkpoints store block-FP8 weights with MXFP4 routed experts (`expert_dtype="fp4"` in the model config).
+They load as-is, with no quantization config:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "deepseek-ai/DeepSeek-V4-Flash"
+model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+inputs = tokenizer("What are we having for dinner?", return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=20)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+The checkpoint mixes two formats, so it loads through [`FineGrainedConfig`]. The config below is the one built
+automatically; pass your own to change a format, for example `Mxfp4Config(activation_format="bf16")` to run the experts
+weight-only, or `dequantize=True` to load everything in bf16:
+
+```python
+from transformers import AutoModelForCausalLM, BlockFp8Config, FineGrainedConfig, Mxfp4Config
+
+quantization_config = FineGrainedConfig(
+    dtype_plan={
+        r"\.experts($|\.)": Mxfp4Config(activation_format="mxfp4"),
+        ".*": BlockFp8Config(weight_block_size=(128, 128), scale_fmt="ue8m0"),
+    }
+)
+model = AutoModelForCausalLM.from_pretrained(
+    "deepseek-ai/DeepSeek-V4-Flash", dtype="auto", device_map="auto", quantization_config=quantization_config
+)
+```
+
 ## Architecture (paper §2)
 
 ### Hybrid attention (§2.3)

diff --git a/docs/source/en/model_doc/minimax_m3_vl.md b/docs/source/en/model_doc/minimax_m3_vl.md
@@ -153,7 +153,7 @@ from transformers import (
     AutoModelForImageTextToText,
     AutoTokenizer,
     CompileConfig,
-    FineGrainedFP8Config,
+    Mxfp8Config,
 )
 from transformers.distributed import DistributedConfig
 
@@ -164,7 +164,7 @@ model = AutoModelForImageTextToText.from_pretrained(
     "MiniMaxAI/MiniMax-M3-preview",
     dtype=torch.bfloat16,
     # Dequantize the native MXFP8 weights to bf16 at load (the speed win); needs even TP/EP sharding.
-    quantization_config=FineGrainedFP8Config(dequantize=True),
+    quantization_config=Mxfp8Config(dequantize=True),
     distributed_config=DistributedConfig(
         tp_size=int(os.environ["WORLD_SIZE"]),
         ep_size=int(os.environ["WORLD_SIZE"]),

diff --git a/docs/source/en/quantization/blockfp8.md b/docs/source/en/quantization/blockfp8.md
@@ -0,0 +1,130 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contains specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Block FP8
+
+Block FP8 stores weights in 8-bit floating point (`float8_e4m3fn`) with one scale per 2D block of the weight matrix
+(`weight_block_size=(128, 128)` by default), one scale per tensor (`weight_block_size=None`), or one scale per
+output channel (`weight_block_size="row"`). Rowwise FP8 uses dynamic per-token activation scales. Activations are
+quantized to FP8 too, per token and block at run time (`activation_scheme="dynamic"`), or against a calibrated scale
+shipped with the checkpoint (`activation_scheme="static"`).
+
+It is the format of [DeepSeek-V3](https://hf.co/papers/2412.19437) and DeepSeek-R1, Qwen3-FP8, and Mistral's static
+FP8 checkpoints. [`BlockFp8Config`] loads these checkpoints as they ship and quantizes full-precision checkpoints on the
+fly. Checkpoints declare it as `quant_method="fp8"`.
+
+<div class="flex justify-center">
+    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/b7b3b34bf826a6423ea82ffc57ecac80c46c3c76/transformers/quantization/quantization_deepseek.png">
+</div>
+
+> [!TIP]
+> Quantized compute needs a CUDA GPU with compute capability 8.9 or newer (Ada, Hopper, Blackwell) or an Intel XPU,
+> and the [kernels](https://github.com/huggingface/kernels) package, which loads the
+> [finegrained kernels](https://huggingface.co/kernels-community/finegrained-kernels) from the Hub. Without one, a
+> pre-quantized checkpoint is dequantized to full precision as it loads.
+
+```bash
+pip install --upgrade accelerate kernels
+```
+
+## Load a pre-quantized checkpoint
+
+A pre-quantized checkpoint carries its quantization config, so no config is needed.
+
+```py
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Qwen/Qwen3-8B-FP8"
+model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+inputs = tokenizer("What are we having for dinner?", return_tensors="pt").to(model.device)
+output = model.generate(**inputs, max_new_tokens=20)
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+
+Checkpoints exported by [compressed-tensors](./compressed_tensors) in block FP8 (`format="float-quantized"`) load as
+[`BlockFp8Config`] as well.
+
+Rowwise FP8 from FBGEMM, ModelOpt (`FP8_PER_CHANNEL_PER_TOKEN`), and compressed-tensors (channel weights with
+token activations) uses the same finegrained runtime. Producer-specific key and scale layouts are converted on load.
+
+## Quantize on the fly
+
+Pass a [`BlockFp8Config`] to [`~PreTrainedModel.from_pretrained`] to quantize a full-precision checkpoint as it loads.
+Set `dtype="auto"` so the unquantized modules keep the checkpoint's dtype.
+
+```py
+from transformers import AutoModelForCausalLM, BlockFp8Config
+
+quantization_config = BlockFp8Config(modules_to_not_convert=["lm_head"])
+model = AutoModelForCausalLM.from_pretrained(
+    "meta-llama/Llama-3.1-8B", dtype="auto", device_map="auto", quantization_config=quantization_config
+)
+```
+
+[`~PreTrainedModel.save_pretrained`] writes the quantized model back in the same layout, and
+[`~PreTrainedModel.from_pretrained`] reloads it.
+
+```py
+model.save_pretrained("llama-3.1-8b-fp8")
+model = AutoModelForCausalLM.from_pretrained("llama-3.1-8b-fp8", device_map="auto")
+```
+
+## Activation formats
+
+`activation_format` picks what the activations are quantized to before each matmul.
+
+| `activation_format` | Scheme | Activations |
+|---|---|---|
+| `None` (default) or `"fp8"` | W8A8 | FP8, per token and block at run time, or against a calibrated per-tensor scale with `activation_scheme="static"` |
+| `"bf16"` | W8A16 | Full precision; block-FP8 weights are decoded inside the matmul |
+
+## Options
+
+- `weight_block_size` is the `(rows, columns)` block each weight scale covers; `None` gives one scale per tensor,
+  and `"row"` gives one FP32 scale per output channel.
+- `activation_scale_ub` caps activation amax for dynamic rowwise W8A8; values beyond it saturate. FBGEMM checkpoints
+  retain their configured bound (1200 by default). Other rowwise formats leave it unset.
+- `activation_scheme="static"` quantizes activations against the calibrated scale a checkpoint ships (its
+  `input_scale`), instead of deriving one per token at run time.
+- `scale_fmt="ue8m0"` holds the block scales as powers of two (`float8_e8m0fnu`, with a `uint8` exponent fallback when the dtype is unavailable), as DeepSeek-V4 ships them.
+  Float32 scale containers, such as DeepSeek-V4-Base's, load and save as native UE8M0.
+- `modules_to_not_convert` keeps the modules it names in full precision; `modules_to_convert` names embedding tables to
+  hold in FP8 with one per-tensor scale (a table too large to keep in full precision).
+- `dequantize=True` loads a pre-quantized checkpoint dequantized to full precision.
+
+## DeepGEMM fast path
+
+On Hopper GPUs, a dense FP8 linear with `weight_block_size=(128, 128)` and dynamic activations dispatches to the
+[DeepGEMM](https://github.com/deepseek-ai/DeepGEMM) kernels from
+[kernels-community/deep-gemm](https://huggingface.co/kernels-community/deep-gemm). On Blackwell the Triton kernels are
+faster and are always used.
+
+DeepGEMM JIT-compiles its kernels with the system `nvcc`, so a full CUDA toolkit must be installed (12.3 or later on
+Hopper). Transformers finds the toolkit by checking `CUDA_HOME`, then `CUDA_PATH`, then `nvcc` on your `PATH`, then
+`/usr/local/cuda`. If DeepGEMM cannot load, Transformers logs a warning once and uses the Triton kernels. Set
+`TRANSFORMERS_DISABLE_DEEPGEMM_LINEAR=1` to always use them.
+
+For MoE experts, DeepGEMM is opt-in: pass `experts_implementation="deepgemm"` (or `"deepgemm_megamoe"` on Blackwell)
+to [`~PreTrainedModel.from_pretrained`]. The DeepGEMM experts kernels on Blackwell read only UE8M0 scales; with
+`float32` scales, use the default `grouped_mm` and `batched_mm` experts implementations. See the
+[Experts backends](../experts_interface) guide for every option.
+
+## Related formats
+
+- [MXFP8](./mxfp8) holds FP8 weights with one power-of-two scale per 32 values of a row.
+- [Fine-grained](./finegrained) mixes block FP8 with the 4-bit formats in one model, as DeepSeek-V4 ships it.
diff --git a/docs/source/en/quantization/compressed_tensors.md b/docs/source/en/quantization/compressed_tensors.md
@@ -49,6 +49,14 @@ pip install -e .
 
 Search using the compressed-tensors [tag](https://huggingface.co/models?other=compressed-tensors) to find a compatible model on the Hugging Face Hub.
 
+> [!NOTE]
+> A checkpoint in a format the [fine-grained](./finegrained) kernels serve loads through them, with
+> `checkpoint_format="compressed-tensors"`: `float-quantized` FP8 with per-tensor or block weight scales as
+> [`BlockFp8Config`], `nvfp4-pack-quantized` as [`NVFP4Config`], and one whose config groups mix formats as a
+> [`FineGrainedConfig`]. Integer, per-channel FP8, sparse and transformed checkpoints stay with compressed-tensors, as
+> do `naive-quantized` ones such as the examples on this page. Pass a [`CompressedTensorsConfig`] to
+> [`~PreTrainedModel.from_pretrained`] to load any checkpoint with compressed-tensors.
+
 Pre-quantized models can be loaded directly. To quantize a model into the compressed-tensors format, see [llm-compressor](https://github.com/vllm-project/llm-compressor). Alternatively, models can be created independently and serialized with a compressed-tensors config.
 
 ```python

diff --git a/docs/source/en/quantization/concept_guide.md b/docs/source/en/quantization/concept_guide.md
@@ -110,7 +110,7 @@ FP8 is used in the *A8W8* quantization scheme, which quantizes both activations
 
 While int8 has broad support, efficient FP8 computation requires specific hardware capabilities found in newer GPUs like NVIDIA H100/H200/B100, AMD Instinct MI300 series, and Intel XPU. Without native hardware acceleration, the benefits of FP8 might not be fully realized.
 
-Transformers supports FP8 through specific backends like [FBGEMM](./fbgemm_fp8), [FineGrainedFP8](./finegrained_fp8), and [compressed-tensors](./compressed_tensors). These backends handle the underlying FP8 conversion and computation when the appropriate hardware and configurations are used.
+Transformers supports FP8 through specific backends like [FBGEMM](./fbgemm_fp8), [Block FP8](./blockfp8), [MXFP8](./mxfp8), and [compressed-tensors](./compressed_tensors). These backends handle the underlying FP8 conversion and computation when the appropriate hardware and configurations are used.
 
 ## Granularity
 

diff --git a/docs/source/en/quantization/contribute.md b/docs/source/en/quantization/contribute.md
@@ -46,7 +46,7 @@ Some quantization methods may require "pre-quantizing" the model through data ca
 
 ## Create new HFQuantizer class
 
-0. The best starting point would be to have a look at another quantization method such as Finegrained Fp8. You will have to update or create three files in total: the [config file](https://github.com/huggingface/transformers/blob/main/src/transformers/utils/quantization_config.py), the [integration file](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/finegrained_fp8.py) and the [quantizer file](https://github.com/huggingface/transformers/blob/main/src/transformers/quantizers/quantizer_finegrained_fp8.py).
+0. The best starting point would be to have a look at another quantization method such as Block FP8. You will have to update or create three files in total: the [config file](https://github.com/huggingface/transformers/blob/main/src/transformers/utils/quantization_config.py), the [integration file](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/finegrained/layers.py) and the [quantizer file](https://github.com/huggingface/transformers/blob/main/src/transformers/quantizers/quantizer_blockfp8.py).
 
 1. Create a new quantization config class inside [src/transformers/utils/quantization_config.py](https://github.com/huggingface/transformers/blob/abbffc4525566a48a9733639797c812301218b83/src/transformers/utils/quantization_config.py). Add the new quantization config to the [\_import_structure](https://github.com/huggingface/transformers/blob/abbffc4525566a48a9733639797c812301218b83/src/transformers/__init__.py#L1088) inside Transformers' [src/transformers/\_\_init\_\_.py](https://github.com/huggingface/transformers/blob/abbffc4525566a48a9733639797c812301218b83/src/transformers/__init__.py) file.