You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Drama 3: native Metal support for the pinned NVFP4 pruned checkpoint #1266
The Drama recipe now recommends plotarmordev/drama-3-NVFP4-Pruned@494db08481db952f834b2241bce63fb3dfa72554. Its pinned CUDA engine (plotarmordev/TensorFold@b1f4b9b4830c9bd43bf7c432cffe7dea21313537) supports the checkpoint, but has no Drama Metal backend. The existing upstream Drama Metal loader expects the full FP8-expert checkpoint, all 256 experts per layer, and BF16 dense/tower projections. A launcher-only Mac change would therefore be misleading.
Proposed scope
Add support to the existing native Drama Metal family, preserving the existing full-model path and keeping the recipe's CUDA pin unchanged:
Read the checkpoint's actual tensor formats and compact expert metadata; validate dtype, shape, scales, kept expert IDs and routing before use. The legacy bsgemm-moe config string alone is not sufficient to identify these mixed-format tensors.
Execute packed NVFP4 experts on Metal without converting/requantizing the checkpoint or expanding the routed experts to a full BF16 model.
Handle the checkpoint's dense/tower projection formats explicitly, retaining documented BF16 activations and FP32 accumulation. Unsupported combinations must fail clearly, never silently substitute another checkpoint/backend.
Support the reference-conditioned guidance needed by the recipe, with explicit platform capability and voice-format handling rather than accepting ignored CUDA-only flags.
Keep the change opt-in through selecting this checkpoint. No installed service changes, speed claim, quality-equivalence claim, or hardware-wide qualification claim.
Verification plan
Before claiming support: deterministic malformed-format/compaction/refusal tests, synthetic Metal projection checks against an independent FP64 decoder/reference (including signs, scale boundaries and mixed/pruned layers), native builds, and isolated real-checkpoint speech checks. Preserve the full-model behavior and exercise repeated requests, reference/default transitions, streaming/cancellation and owned-process cleanup where affected. Publish exact source/checkpoint/tool pins, attempted checks, failures and untested platforms. CPU tests are not GPU qualification; a successful startup is not successful synthesis.
The recipe PR will point to the engine contribution and its retained verification. Implementation is being prepared; no Mac NVFP4 compatibility result is claimed yet.
For the CUDA side of this checkpoint: the pinned engine (plotarmordev/TensorFold drama-nvfp4) now has a voice reference-prefix cache proposed in plotarmordev#4, the CUDA counterpart of main's Metal reference-prefill cache (a06a4df). On a DGX Spark it takes a catalog voice's first audio from about 415 ms to about 121 ms with byte-identical audio across 14 voices. Our Metal cross-check for #1268 is in the comment there.
Done on main by #1268 (@jerryfane's native Metal NVFP4 for the pruned Drama 3 checkpoint, landed 3abeeff..f3c1ab0, and in 1.1.5). On an M3 Ultra the pruned checkpoint gives 6 of 6 equal repeats and concurrent requests equal their solo runs. Default Drama 3 and the catalog voices stay byte-identical. Thank you!
Activity
jerryfane commentedon Oct 25, 2026
/atc claim
jerryfane commentedon Oct 25, 2026
/atc presence 938f7514940145df5446d049f66593122b008be57c998f2ec2db1de2657b4c78
drowzeys commentedon Oct 25, 2026
For the CUDA side of this checkpoint: the pinned engine (plotarmordev/TensorFold
drama-nvfp4) now has a voice reference-prefix cache proposed in plotarmordev#4, the CUDA counterpart of main's Metal reference-prefill cache (a06a4df). On a DGX Spark it takes a catalog voice's first audio from about 415 ms to about 121 ms with byte-identical audio across 14 voices. Our Metal cross-check for #1268 is in the comment there.jerryfane commentedon Oct 25, 2026
/atc release
ashhart commentedon Oct 26, 2026
Done on main by #1268 (@jerryfane's native Metal NVFP4 for the pruned Drama 3 checkpoint, landed 3abeeff..f3c1ab0, and in 1.1.5). On an M3 Ultra the pruned checkpoint gives 6 of 6 equal repeats and concurrent requests equal their solo runs. Default Drama 3 and the catalog voices stay byte-identical. Thank you!
- Synthema, Ash's AI Assistant.