Repository navigation
New model family: MiniMax-M3.1-Flash VL (minimax_m3_vl) EXL3, TP2-served #1280
Copy link
Copy link
Closed as not planned
Closed as not planned
Copy link
Description
What
New model family minimax_m3_vl: MiniMax-M3.1-Flash (Mia-AiLab/MiniMax-M3.1-Flash-EXL3-EAQ-v1-3.06-bpw) running natively in TensorFold's Zig engine on CUDA, served through the existing lane surface (TP1) and over tensor-parallel on 2 ranks (2× DGX GB10 over NCCL/RoCE).
- New family directory
zig/src/families/minimax_m3_vl/(config, EXL3 weight load/unpack, sparse attention, layer graph, serve, CUDA-native adapter including the TP2 rank boot, mirrored on the existing glm env-rank pattern — no new scheduler, no parallel lane machinery). - New CUDA kernel file
zig/kernels/cuda/minimax_m3.cuplus additive changes tozig/kernels/cuda/exl3_experts.cufor the family's routed experts. - Minimal core glue only: build registration, kernel vtable wiring, CLI hub/info entries. Shared lane verification and caches stay in the core.
- Tokenizer free-text path refuses explicitly until a tokenizer is provided (
ServeTextNeedsTokenizer); encode parity vs Hugging Facetokenizers0.22.2 is bit-exact on the 19-case probe (ASCII, CJK, emoji, tabs/newlines, JSON punctuation).
Proposed gate
- Device kernel parity vs the engine's CPU references at the fp16 store (scores/marks/decode+combine/swigluoai; NB/nsplit bound guards).
- Real-pack f64 grade: greedy tokens EXACT vs an independent f64 reference, per-row logits within quant tolerance.
- TP1 vs TP2 token identity on the real pack.
- Drafted/plain, resumed/fresh, concurrent/solo equalities on the served path.
- Bit-exact HF encode parity for the free-text tokenizer path.
Evidence already collected (verification records in hand)
- Device kernel parity: 10/10 PASS cases on GB10 CUDA (
max|delta|0e0 for scores/marks/swigluoai; decode+combine 1.5e-5, one fp16 ulp at the fp16 store; 3 kernel bugs found and fixed on the way). - Real-pack f64 grade (first 2/60 layers, 35 rows, 4 greedy steps): greedy tokens EXACT vs f64, logits
max|delta|0.003237 (quant tolerance), no NaN/inf. - TP2 real-pack load split: rank 0 75.79 GiB / rank 1 79.68 GiB, both ranks admit the split independently.
- TP2 real-pack boot end-to-end: real greedy tokens
88638 88638 52236 40265with clean logits; decode reproduces via the pack's own tokenizer. - TP2 lane surface answers a 32-row prompt over the split pack, token-for-token EXACT and logits-identical to the TP1 verification (max 6.418439, mean −0.942022).
- Host suite on the canonical tree:
zig build test-cuda-hostexit 0, 177/177 steps, 158/159 tests passed (1 pre-existing recorded skip). - Server attach: the family attaches to the REAL server path on :8000 (
/v1/modelslive at fixture scale); the full-depth real-pack TP2 server proof is in flight and will be linked when it lands.
Honest boundaries
- The f64 grade covers the first 2 of 60 real layers; deeper layers are graded through the TP1/TP2 boot self-consistency, no f64 bar exists yet.
- The HTTP server proof is verified at fixture scale; the full-depth real-pack TP2 server run is still in progress — I will cite it only as done when its verification exists.
- CUDA only so far (GB10); Metal and other GPUs not run.
- This PR will be one change: one new family + minimal core glue, separate from the sibling Metal effort. Note on overlap: minimax_m3: serve MiniMax-M3.1-Flash (428B-A23B MoE, images) natively on Metal #1235 (Metal
minimax_m3) shares two small glue files (zig/src/cli/hub.zig,zig/src/native/metal.zig); my additions there are additive and I'm happy to coordinate or rebase after theirs lands. - The estate that exercised this is a private 2× DGX GB10 setup over RoCE; no public benchmark claims are implied.
PR to follow on a feature branch; this issue stands as the new-family gate per CONTRIBUTING.md.
Metadata
Metadata
Assignees
Labels
No labels
Activity
styles01 commentedon Oct 25, 2026
The family is up for review: the minimax_m3_vl EXL3/CUDA/TP2 PR linking this gate is #1281 (temperature-agnostic requests serve fixed greedy; other sampling knobs and grammar still refuse by exact name).
ashhart commentedon Oct 27, 2026
Thank you, @styles01, for the MiniMax-M3.1-Flash bring-up and the parity work behind it. As we said on #1281, we'd like this model to be one family, minimax_m3, with Metal (#1235) and CUDA (#1388) backends serving through the lane core, so we're closing this proposal for a second family. Your EXL3 expert path would be very welcome as a PR on that family, since the published pack fits two GB10s.