From 4412626681fa655f16f8846034ff3a0810b3f67b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=BCrgen=20Schmied?= <600316+jschmied@users.noreply.github.com> Date: Tue, 20 Oct 2026 23:23:33 +0200 Subject: [PATCH] docs: the 27B recipe names its native CUDA backend and which formats it reads The page said the 27B has no CUDA backend; 1.1.0's native engine serves the MLX affine checkpoints on CUDA. Its CUDA section, which promises NVFP4 and EXL3 checkpoints, describes the Python engine (0.6.6); it now says so, and that neither reader is ported to the native engine yet. --- docs/recipes/qwen3.8-27b.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/recipes/qwen3.8-27b.md b/docs/recipes/qwen3.8-27b.md index a4b93279af..acd1afa3d3 100644 --- a/docs/recipes/qwen3.8-27b.md +++ b/docs/recipes/qwen3.8-27b.md @@ -2,7 +2,9 @@ The native engine serves `TensorFold/Qwen3.8-27B-MLX-4bit` on Metal, qualified on M5 Max and M3 Ultra, from `zig/src/families/qwen27/`. It drafts with DFlash2 (`zig/src/families/qwen27/dflash/`) only when `--drafter` names -the drafter; without it the model decodes plain. It has no CUDA backend yet. +the drafter; without it the model decodes plain. On CUDA the native engine serves the MLX affine checkpoints (2, 3, +4, 5, 6 and 8 bits in groups of 64) from `zig/src/families/qwen3_5/cuda_*.zig`, with DFlash2 drafts and +`--kv-cache fp8`; it reads no NVFP4 or EXL3 checkpoint yet. ```bash tensorfold pull TensorFold/Qwen3.8-27B-MLX-4bit @@ -83,6 +85,10 @@ quality comparisons are TBD [release-0.3.5]. ## CUDA +This section is the Python engine's (0.6.6). The native CUDA engine reads only MLX affine checkpoints so far: the +NVFP4 and EXL3 readers below have not been ported to it (1.1.0 carries the shared EXL3 module of #832, which the +27B does not read through yet). + On CUDA the 27B serves NVFP4 and EXL3 checkpoints, and the MLX 4-bit checkpoint as the portable option: the same files a Mac serves, and the only format two ranks read. `tensorfold serve` loads the checkpoint you name; it picks none by itself. Use the [CUDA container setup](https://github.com/ashhart/TensorFold/blob/v0.6.6/RUNBOOK.md#nvidia-gpus)