Skip to content

Drama 3: native Metal support for the pinned NVFP4 pruned checkpoint #1266

Closed

Description

@jerryfane

Problem

The Drama recipe now recommends plotarmordev/drama-3-NVFP4-Pruned@494db08481db952f834b2241bce63fb3dfa72554. Its pinned CUDA engine (plotarmordev/TensorFold@b1f4b9b4830c9bd43bf7c432cffe7dea21313537) supports the checkpoint, but has no Drama Metal backend. The existing upstream Drama Metal loader expects the full FP8-expert checkpoint, all 256 experts per layer, and BF16 dense/tower projections. A launcher-only Mac change would therefore be misleading.

Proposed scope

Add support to the existing native Drama Metal family, preserving the existing full-model path and keeping the recipe's CUDA pin unchanged:

  • Read the checkpoint's actual tensor formats and compact expert metadata; validate dtype, shape, scales, kept expert IDs and routing before use. The legacy bsgemm-moe config string alone is not sufficient to identify these mixed-format tensors.
  • Execute packed NVFP4 experts on Metal without converting/requantizing the checkpoint or expanding the routed experts to a full BF16 model.
  • Handle the checkpoint's dense/tower projection formats explicitly, retaining documented BF16 activations and FP32 accumulation. Unsupported combinations must fail clearly, never silently substitute another checkpoint/backend.
  • Support the reference-conditioned guidance needed by the recipe, with explicit platform capability and voice-format handling rather than accepting ignored CUDA-only flags.
  • Keep the change opt-in through selecting this checkpoint. No installed service changes, speed claim, quality-equivalence claim, or hardware-wide qualification claim.

Verification plan

Before claiming support: deterministic malformed-format/compaction/refusal tests, synthetic Metal projection checks against an independent FP64 decoder/reference (including signs, scale boundaries and mixed/pruned layers), native builds, and isolated real-checkpoint speech checks. Preserve the full-model behavior and exercise repeated requests, reference/default transitions, streaming/cancellation and owned-process cleanup where affected. Publish exact source/checkpoint/tool pins, attempted checks, failures and untested platforms. CPU tests are not GPU qualification; a successful startup is not successful synthesis.

The recipe PR will point to the engine contribution and its retained verification. Implementation is being prepared; no Mac NVFP4 compatibility result is claimed yet.

Activity

  1. jerryfane commented on Oct 25, 2026

    @jerryfane
    ContributorAuthor

    /atc claim

  2. jerryfane commented on Oct 25, 2026

    @jerryfane
    ContributorAuthor

    /atc presence 938f7514940145df5446d049f66593122b008be57c998f2ec2db1de2657b4c78

  3. drowzeys commented on Oct 25, 2026

    @drowzeys
    Contributor

    For the CUDA side of this checkpoint: the pinned engine (plotarmordev/TensorFold drama-nvfp4) now has a voice reference-prefix cache proposed in plotarmordev#4, the CUDA counterpart of main's Metal reference-prefill cache (a06a4df). On a DGX Spark it takes a catalog voice's first audio from about 415 ms to about 121 ms with byte-identical audio across 14 voices. Our Metal cross-check for #1268 is in the comment there.

  4. jerryfane commented on Oct 25, 2026

    @jerryfane
    ContributorAuthor

    /atc release

  5. ashhart commented on Oct 26, 2026

    @ashhart
    Owner

    Done on main by #1268 (@jerryfane's native Metal NVFP4 for the pruned Drama 3 checkpoint, landed 3abeeff..f3c1ab0, and in 1.1.5). On an M3 Ultra the pruned checkpoint gives 6 of 6 equal repeats and concurrent requests equal their solo runs. Default Drama 3 and the catalog voices stay byte-identical. Thank you!

    - Synthema, Ash's AI Assistant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions