Skip to content

New model family: MiniMax-M3.1-Flash VL (minimax_m3_vl) EXL3, TP2-served #1280

Closed as not planned

Description

@styles01

What

New model family minimax_m3_vl: MiniMax-M3.1-Flash (Mia-AiLab/MiniMax-M3.1-Flash-EXL3-EAQ-v1-3.06-bpw) running natively in TensorFold's Zig engine on CUDA, served through the existing lane surface (TP1) and over tensor-parallel on 2 ranks (2× DGX GB10 over NCCL/RoCE).

  • New family directory zig/src/families/minimax_m3_vl/ (config, EXL3 weight load/unpack, sparse attention, layer graph, serve, CUDA-native adapter including the TP2 rank boot, mirrored on the existing glm env-rank pattern — no new scheduler, no parallel lane machinery).
  • New CUDA kernel file zig/kernels/cuda/minimax_m3.cu plus additive changes to zig/kernels/cuda/exl3_experts.cu for the family's routed experts.
  • Minimal core glue only: build registration, kernel vtable wiring, CLI hub/info entries. Shared lane verification and caches stay in the core.
  • Tokenizer free-text path refuses explicitly until a tokenizer is provided (ServeTextNeedsTokenizer); encode parity vs Hugging Face tokenizers 0.22.2 is bit-exact on the 19-case probe (ASCII, CJK, emoji, tabs/newlines, JSON punctuation).

Proposed gate

  • Device kernel parity vs the engine's CPU references at the fp16 store (scores/marks/decode+combine/swigluoai; NB/nsplit bound guards).
  • Real-pack f64 grade: greedy tokens EXACT vs an independent f64 reference, per-row logits within quant tolerance.
  • TP1 vs TP2 token identity on the real pack.
  • Drafted/plain, resumed/fresh, concurrent/solo equalities on the served path.
  • Bit-exact HF encode parity for the free-text tokenizer path.

Evidence already collected (verification records in hand)

  • Device kernel parity: 10/10 PASS cases on GB10 CUDA (max|delta| 0e0 for scores/marks/swigluoai; decode+combine 1.5e-5, one fp16 ulp at the fp16 store; 3 kernel bugs found and fixed on the way).
  • Real-pack f64 grade (first 2/60 layers, 35 rows, 4 greedy steps): greedy tokens EXACT vs f64, logits max|delta| 0.003237 (quant tolerance), no NaN/inf.
  • TP2 real-pack load split: rank 0 75.79 GiB / rank 1 79.68 GiB, both ranks admit the split independently.
  • TP2 real-pack boot end-to-end: real greedy tokens 88638 88638 52236 40265 with clean logits; decode reproduces via the pack's own tokenizer.
  • TP2 lane surface answers a 32-row prompt over the split pack, token-for-token EXACT and logits-identical to the TP1 verification (max 6.418439, mean −0.942022).
  • Host suite on the canonical tree: zig build test-cuda-host exit 0, 177/177 steps, 158/159 tests passed (1 pre-existing recorded skip).
  • Server attach: the family attaches to the REAL server path on :8000 (/v1/models live at fixture scale); the full-depth real-pack TP2 server proof is in flight and will be linked when it lands.

Honest boundaries

  • The f64 grade covers the first 2 of 60 real layers; deeper layers are graded through the TP1/TP2 boot self-consistency, no f64 bar exists yet.
  • The HTTP server proof is verified at fixture scale; the full-depth real-pack TP2 server run is still in progress — I will cite it only as done when its verification exists.
  • CUDA only so far (GB10); Metal and other GPUs not run.
  • This PR will be one change: one new family + minimal core glue, separate from the sibling Metal effort. Note on overlap: minimax_m3: serve MiniMax-M3.1-Flash (428B-A23B MoE, images) natively on Metal #1235 (Metal minimax_m3) shares two small glue files (zig/src/cli/hub.zig, zig/src/native/metal.zig); my additions there are additive and I'm happy to coordinate or rebase after theirs lands.
  • The estate that exercised this is a private 2× DGX GB10 setup over RoCE; no public benchmark claims are implied.

PR to follow on a feature branch; this issue stands as the new-family gate per CONTRIBUTING.md.

Activity

  1. styles01 commented on Oct 25, 2026

    @styles01
    Author

    The family is up for review: the minimax_m3_vl EXL3/CUDA/TP2 PR linking this gate is #1281 (temperature-agnostic requests serve fixed greedy; other sampling knobs and grammar still refuse by exact name).

  2. ashhart commented on Oct 27, 2026

    @ashhart
    Owner

    Thank you, @styles01, for the MiniMax-M3.1-Flash bring-up and the parity work behind it. As we said on #1281, we'd like this model to be one family, minimax_m3, with Metal (#1235) and CUDA (#1388) backends serving through the lane core, so we're closing this proposal for a second family. Your EXL3 expert path would be very welcome as a PR on that family, since the published pack fits two GB10s.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions