Agree the gate for Drama 3 model-maximum native speech context on Metal and CUDA #2002

Title: Agree the gate for Drama 3 model-maximum native speech context on Metal and CUDA

Requested outcome

Support Drama 3's actual model-maximum native speech window where the existing backend architecture and available resources permit it. Report admitted limits accurately and refuse unsupported requests before execution. A smaller context profile, a raised flag, successful allocation, or successful startup is not evidence of full-window execution.

Public baseline and coordination

The release baseline is TensorFold v1.2.1. Investigation uses upstream main at 6194b8c in ashhart/TensorFold. The published fishaudio/drama-3 configuration declares text_config.max_position_embeddings = 262144. Both native adapters and drama_budget.zig currently admit context only from 1025 through 8192.

The existing issue #914 is closed. The full ATC board currently reports incomplete file coverage; ownership has not been established and no implementation or claim has started. This issue proposes a new context-specific gate. A complete focused ownership check and a successful permitted claim are required before implementation. Preparation will use --local reporting without --github-name.

Checkpoint scope

Metal: plotarmordev/drama-3-NVFP4-Pruned at revision 494db08481db952f834b2241bce63fb3dfa72554 is the pinned public candidate. Its stored weights and precision must be preserved. A declared context in its config does not qualify native execution at that context.

CUDA: fishaudio/drama-3 at revision 78e5ab79e49aa9a25abe8799eff2c4d73a1c3216 is a separate public candidate. Bounded reads of all 15 safetensors JSON headers match all 62245 native loader tensor names, dtypes and shapes: 805 BF16, 30720 F8_E4M3 and 30720 F32 inverse scales, with no missing, mismatched or unused tensors. This establishes header compatibility only. Codec bytes, device loading, runtime arithmetic and model execution remain untested. CUDA qualification is independent. The Metal pruned NVFP4 layout and Metal-only semantic CFG are not assumed to work on CUDA. Unsupported layouts or options must continue to refuse explicitly.

Units and existing failure

Keep these quantities separate in admission, discovery, documentation, fixtures, and receipts:

• Input UTF-8 bytes and Unicode code points.
• Tokenized request text and structural/wrapper tokens.
• Reference transcript tokens and acoustic reference frames.
• The assembled prompt rows consumed by the backbone, including reference code rows.
• Generated acoustic frames, each containing the semantic and residual codebooks.
• The combined native positional window and its reserved row.

Current public code has independent limits: speech.zig accepts at most 4096 input code points; the shared reference-capable prompt builder accepts at most 16384 request-text bytes; drama_budget.zig additionally caps assembled prompt rows at 4096; the serve CLI and budget cap explicit generated frames at 8190. Reference catalogs separately accept at most four passages, transcripts of at most 4096 bytes each, and at most 646 code frames per passage, plus file/catalog bounds. Those reference safety limits must be assessed separately from the backbone window rather than silently relabeled as text-context limits.

model.validate currently requires at least two assembled prompt rows and enforces P + F + 1 <= C, where P is assembled prompt rows, F is the generated-frame budget, and C is the configured context. At C = 262144, default F = 1024 leaves P <= 261119. A synthetic two-row prompt permits F <= 262141 under this conservative contract. Real rendered HTTP prompts have their own overhead and must be counted exactly.

There is also an existing explicit-admission discrepancy: budget(8192, 8190) accepts the option but returns 8189 frames because its default clamp applies to explicit values too. An explicit request should be honored exactly when valid or refused clearly. Raising context alone leaves the HTTP and assembled-prompt restrictions in place.

Existing ownership and intended change

The intended contribution extends the current native adapters, shared budget/validation mechanisms, family allocation owners, shared frame rounds, per-lane model/codec state, and existing HTTP lowering/discovery. It does not introduce a scheduler or downstream integration. The intended change preserves stored weights, activation/cache precision, reduction order, and GPU kernel arithmetic. If investigation establishes that a kernel, attention architecture, reduction, or state-layout change is necessary, stop treating this as an admission-only change and agree the additional contract before proceeding.

Proposed review stages:

1. Establish and reproduce the existing refusals with public or synthetic fixtures. Verify each checkpoint's actual model geometry and backend layout support. Trace every context-derived allocation, integer conversion, launch bound, and prompt/frame position before selecting the implementation.
2. Extend the existing owners to model-maximum capacity with checked arithmetic and truthful resource admission. Remove arbitrary prompt/window restrictions only where the complete execution path supports the resulting range. Preserve defaults where appropriate; explicit budgets must not silently shrink. Determine the HTTP byte/code-point policy separately and allow genuinely long native/HTTP requests where supported.
3. Report the loaded speech window, assembled prompt-row budget, generated-frame budget, sample geometry, and HTTP input policy distinctly through existing discovery paths. /v1/models currently reports the combined context but omits speech-specific budgets. Static capabilities --json reports compiled family formats and flags; it is not a checkpoint qualification receipt.
4. Run the agreed host, device, kernel, fidelity, lifecycle, and performance gates. Publish support claims only to the extent established by those results. Keep any unqualified candidate explicitly draft until its required gates pass.

Memory, arithmetic, and lifetime contract

All products, sums, alignment, element-to-byte conversions, signed/unsigned casts, and device/API extent limits must be checked before allocation or launch. Account separately for weights, per-lane backbone KV/recurrent/scratch state, codec state and workspace, output frames, caller-owned PCM, pinned/device transfers, reference snapshots/cache entries, optional Metal guidance state, and whole-reply redraw buffers. Include partial-open failure cleanup and allocation-limit restoration.

For scale, F = 262141 with frame_length = 2048 requires 536864768 f32 PCM samples, or 2147459072 bytes per active HTTP request before any model state. A whole-reply redraw can require a second buffer of that size. This is arithmetic, not a measured fit or successful synthesis. A GPU budget alone does not admit those host allocations. Do not rely on an allocation eventually failing as the only capacity policy.

Retain existing device ordering and ownership until completion is proved. Cancellation must settle all relevant model/codec work before reusing or releasing caller storage, caches, lanes, or shared weights. Failed waits must close admission and retain unsafe storage as the current completion mechanism requires. Reopening after shutdown, failed/partial startup, cancellation, and ordinary completion must restore valid state and resource limits. Reference reuse must preserve cache identity, tower lookahead, chunk cuts, and uncached equivalence; a restart must not depend on in-memory reference state.

Gate decisions requested from maintainers

Please agree the exact checkpoint/backend matrix, full-window workloads, comparison method, performance metrics, and acceptance thresholds before implementation is represented as ready. No speed or error threshold is proposed by assumption.

• Full-window execution: specify exact public token/frame-row fixtures for text-heavy, reference-conditioned, and acoustic-generation-heavy windows. Cover 8k baseline, 32k, 64k, 128k, and 262144 where admitted. Include actual prompt/decode work near the last permitted position, with final KV/recurrent state and logits or frame/code output checks. Agree how teacher-forced late-position checks supplement, but do not replace, complete native speech execution. Do not count startup, reservation, early EOS, or a shorter run as a full-window pass.
• Boundaries: test exact accepted maxima and one-row/frame excess for P + F + 1; invalid minimums and zero frames; context 8192/8193 and 262144/262145; exact rendered HTTP prompt limits with and without references; byte/code-point/token distinctions; malformed frame/codebook geometry; and explicit resource refusal without silent clamping or corrupting the next request.
• Overflow and resources: use synthetic boundary and failure-injection fixtures for every context-derived allocation/launch extent, checked arithmetic, lane count, optional state multiplier, partial startup, and cleanup. On actual devices record allocation limits, peak physical footprint, device/pinned allocations, RAM class, load/cleanup boundaries, and refusal on insufficient resources. A configured budget on a larger host does not prove fit on a smaller one.
• Lifecycle and concurrency: compare fixed-setting solo and admitted concurrent requests, fresh and reused lanes, cancellation during prompt and generation, the next request after cancellation/refusal/failure, shutdown/reopen, and fresh processes after restart. Exercise reference cache off, miss, hit, invalidation, and changed reference/lookahead cases. Use existing frame rounds and owners throughout.
• Deterministic replay: pin seed, sampler, checkpoint/tokenizer/codec revision, backend/runtime, precision, prompt chunk geometry, and output budget. Disable timing-dependent redraws for the deterministic cells. Retain complete frame/code arrays or hashes and state/logit checks, not plausible text or listening alone. Equality is within the same backend and checkpoint; matching seeds do not promise cross-backend audio equality.
• Independent fidelity: agree a public independent reference and teacher-forced histories for early and late positions on each claimed backend/checkpoint. Record top-token differences and margins, per-column and aggregate error, semantic/residual frame outputs, and codec comparisons as applicable. If arithmetic changes become necessary, require the contributor skill's independent FP64 comparisons for affected operators and the complete backend kernel-contract SUMMARY, including planted controls and launch-limit records. Do not widen tolerances to manufacture a pass.
• Performance: agree the baseline/candidate native speech workload and minimum acceptable result for decode/frame rate, cold prefill and time to first audio, whole-response latency, aggregate concurrency, and peak physical/device/pinned memory. Match checkpoint, runtime, exact public prompt IDs, frame budget, seed, settings, warmups/repetitions, cache state, and hardware. Compare v1.2.1 and the candidate at shared admitted windows; report newly admitted longer windows separately because the release refuses them. Alternate before/after runs when changes are small. Identify an applicable independent server comparison without substituting different weights or precision.
• Host/build checks: use the repository-pinned Zig version; run applicable Drama host/owner/prefill tests and compile checks, the required no-GPU suite and plain build, golden checks, diff checks, and lean comparison with zero new findings. Host or fake-engine passes do not qualify real-model execution.

Available evidence and remaining physical gates

No test accelerator, checkpoint installation, remote machine, or running service is supplied by this proposal. Only explicitly authorized resources may be inventoried or used. No machine identity or access is inferred from repository examples or historical receipts. Physical Metal and CUDA model execution, full-window/late-position checks, device kernel gates, memory-fit measurements, concurrency/restart replay, independent checkpoint fidelity, and before/after performance are unrun. There is no full-window support or readiness claim.

Publication boundary

This proposal and future fixtures/receipts use only public model sources and synthetic/public text or non-identifying voice fixtures. Review the complete outgoing tree, generated assets, archives/debug metadata if shipped, Git configuration effects, and all outgoing commits/messages before upload. Use an approved public Git identity with a GitHub noreply address for required attribution; obtain it before publication if unavailable. No merge, release, or deployment is requested.