server: accept dialogue speaker tags (depends on #1277)

CHANGELOG.md

@@ -4,6 +4,9 @@ Each release's page on GitHub has its notes and measurements. See [the 1.0.0 rel
 
 ## Unreleased
 
+- Drama speech accepts dialogue tags `<|speaker:0|>` through `<|speaker:15|>` in whole and streamed HTTP requests and catalog reference transcripts. Untagged text still uses speaker 0, and catalogs still allow at most four reference passages. See [dialogue speakers](docs/drama-http-voices.md#dialogue-speakers).
+- Drama speech requests accept `temperature`, `top_p`, `top_k` (1 through 64) and exact unsigned 64-bit `seed` values. Whole replies, streams, catalog voices and redraws retain the requested settings; omitted or null fields keep the existing defaults. Explicit seeds advance by the attempt number for redraws. See [speech sampling](docs/speech-sampling.md).
+
 - Flash Next on CUDA with NVFP4 checkpoints prefills 24-29% faster with the same tokens: NVFP4 experts on prompt tiles with the decode form's sums (#1143), no zero fills the one-rank ModelOpt pass reads back (#1147), the hyper-connection normalize over 256-column slices and the dense mix a warp a 32-column group (#1164), and half tiles for decode calls whose 64-column grid leaves SMs idle (#1171). On a GB10 with Qwen3.8 Flash Next FP4-Local, prefill at 16k goes from 1,357 to 1,748 tokens/s and decode gains 1-2%. Attention caches are readable through a page table and the streams beside the first take KV pages from the core pool (#1169, #1170).
 
 - GLM-5.3-Flash on Metal decodes faster with the same tokens by reading the weights once per decode window (#1130): on an M3 Ultra, +2.6% at one stream and +10.2%, +6.4% and +6.3% at two, four and eight.

README.md

@@ -178,7 +178,7 @@ Conversions are explicitly marked: their upstream weights need the linked conver
 | Step-5-Preview, converted affine | `hf download stepfun-ai/Step-5-Preview --local-dir ~/models/step5-preview-fp8` | One 256 GB Mac; text 166.9 GiB, MTP adds 2.5 GiB and vision 3.7 GiB. | [Convert and serve](docs/recipes/step-5-preview.md). The documented 256 GB setup serves converted affine weights; converting the upstream FP8 download needs Python and mlx. |
 | Kolibri-1, block FP8 (research) | `tensorfold pull Aleph-Alpha/Kolibri-1@e52eb4627d11516b0c01de49210ab5a4e4061444` | One NVIDIA GPU; GB10 measured; 74 GB checkpoint. | [Kolibri](docs/recipes/kolibri-1.md). Text only; copy drafts, no EAGLE-3/neural drafter. |
 | Liquid AI d1-3B, decisions | Not documented yet. | Native CUDA; RAM minimum not published. | [API section below](#serve-flags). `/v1/systemone` and `/v1/decisions`. |
-| Fish Audio Drama 3, speech | Not documented yet. | Native Metal or CUDA; GB10 measured. | [Release audio instructions](CHANGELOG.md#110). `/v1/audio/speech`; checkpoint and `codec.pth` required. |
+| Fish Audio Drama 3, speech | Not documented yet. | Native Metal or CUDA; GB10 measured. | [Release audio instructions](CHANGELOG.md#110), [reference voices and dialogue speakers](docs/drama-http-voices.md). `/v1/audio/speech` with [sampling controls](docs/speech-sampling.md); checkpoint and `codec.pth` required. |
 
 ### GLM-5.3: one machine or two?
 
@@ -293,7 +293,7 @@ The binary's `capabilities --json` response lists its supported flags and platfo
 | `--silence-onset-voiced-s SECONDS` | Drama speech only: the voiced seconds below which `--silence-onset-s` applies; required with it, with no default. Calibrate on public or built-in speech. |
 | `--silence-mid-s SECONDS` | Drama speech only: with the silence guard on, finish as `silence` when end of speech falls in a run this long below the floor, after the explicit voiced minimum. Whole replies carry the decimal `X-Silence-Mid-Trips` count, including zero. Must be below `--silence-seconds` and paired with `--silence-mid-min-voiced-s`. Unset by default: off. |
 | `--silence-mid-min-voiced-s SECONDS` | Required with `--silence-mid-s`: positive finite cumulative voiced duration before the mid rule applies, including equality. No default; calibrate with the other silence thresholds. |
-| `--silence-redraws N` | Drama speech only: draw a whole reply the silence guard ended again, up to N times, as seed salt + k; N of 1 or more needs the guard. Unset by default: off, as is 0; streams are never redrawn. See [speech redraws](docs/speech-redraws.md). |
+| `--silence-redraws N` | Drama speech only: draw a whole reply the silence guard ended again, up to N times, as seed salt + k or explicit request seed + k; N of 1 or more needs the guard. Unset by default: off, as is 0; streams are never redrawn. See [speech redraws](docs/speech-redraws.md). |
 | `--silence-redraw-budget-s SECONDS` | Start a redraw only while elapsed time plus the last take's duration fits; a request's `X-Deadline-S` can lower it. No default budget. |
 | `--silence-redraw-controls LIST` | Redraws speak the input filtered to the listed controls, as `--allowed-controls` filters; an empty list removes every control. |
 | `--short-reply-ratio R` | Also redraw a whole reply that finished `stop` with less audio than R (above 0, below 1) of its text's expected length; needs `--silence-redraws` and `--short-reply-chars-per-s`. No default. |

docs/drama-http-voices.md

@@ -22,7 +22,27 @@ The model's licence, the [Fish Audio Community License Agreement](https://huggin
 
 Reference `text` is the exact transcript. Codes are frame-major signed integers, with the semantic code first and residual codebooks following, in the same order as upstream `drama3-generate --encode` emits. Do not substitute codes from an unrelated codec.
 
-The loader enforces the family's reference count, transcript length, frame count and codebook ranges before serving. It also rejects empty references, structural prompt tokens, unsupported speaker tags, oversized files and oversized catalogs. The assembled prompt must still fit the engine's advertised prompt budget. Reference rows go through the same family `prompt.build` used by the generation CLI; only the tokenizer interface is adapted.
+The loader enforces the family's reference count, transcript length, frame count and codebook ranges before serving. It also rejects empty references, structural prompt tokens, malformed speaker tags, oversized files and oversized catalogs. The assembled prompt must still fit the engine's advertised prompt budget. Reference rows go through the same family `prompt.build` used by the generation CLI; only the tokenizer interface is adapted.
+
+## Dialogue speakers
+
+Speech input and reference transcripts accept `<|speaker:N|>` with decimal IDs from `0` through `15`. Tags can change speakers within one request. Untagged text defaults to speaker 0; a leading tag after Unicode whitespace is preserved. Leading zeros, negative IDs, non-ASCII digits, incomplete tags and IDs above 15 are rejected.
+
+For example, send this body to `POST /v1/audio/speech` with your served model name:
+
+```json
+{
+  "model": "MODEL",
+  "input": "<|speaker:0|> Is the gate open? <|speaker:15|> Yes, come through. <|speaker:0|> Thank you.",
+  "response_format": "wav"
+}
+```
+
+For streamed audio, set `"stream": true` and `"response_format": "pcm"`. To condition speakers on reference audio, create a catalog entry whose reference transcripts have matching tags, such as `<|speaker:0|> First reference transcript.` and `<|speaker:15|> Second reference transcript.`, then select its name with `"voice": "conversation"`. The HTTP `speaker` field remains an alias for that catalog name; dialogue speaker IDs belong in the text tags.
+
+One catalog entry still accepts at most **four reference passages**, even though there are sixteen valid speaker IDs. Each passage keeps its own transcript and code rows. Reusing a catalog entry preserves its exact reference prefix; changing a reference speaker tag changes that prefix. Speaker tags specify the prompt's roles; this does not guarantee distinguishable voices without suitable references or establish voice similarity.
+
+The public [Drama 3 MLX prompt implementation](https://github.com/Sogni-AI/drama3-mlx/blob/ecc235039ccd5f982f66a6982184f57432354ab4/drama3_mlx/prompt.py) provides a behavior reference for these speaker markers. TensorFold uses its existing native prompt builder.
 
 ## Writing a voice file
 

docs/speech-redraws.md

@@ -12,12 +12,18 @@ enabled silence guard; `--silence-redraw-controls` and the
 
 ## Seeds
 
-Redraw k (1 to N) seeds its prompt with `TENSORFOLD_SEED_SALT` plus k; the first
+Without an explicit request `seed`, redraw k (1 to N) seeds its prompt with `TENSORFOLD_SEED_SALT` plus k; the first
 take keeps the stock seed, salt plus 0. Redraws are therefore deterministic. With
 the same prompt, redraw k under salt s is the same draw as the first take under
 salt s + k, so a sweep over salts should space compared salts more than N apart,
 or N + 1 with the register guard.
 
+An explicit request `seed` instead uses that exact integer for the first take
+and seed + k modulo 2^64 for each redraw, counting both silence and register
+attempts. Requested `temperature`, `top_p` and `top_k` remain in effect through
+all redraws and control filtering; see [speech sampling](speech-sampling.md).
+The seed rule does not make timing-based redraw budgets deterministic.
+
 ## Budget
 
 A redraw starts only when t + d fits the budget, where t is the time since the
@@ -119,7 +125,7 @@ below 8 kHz, all-zero audio, and takes with fewer than two retained phrases are
 not measured. A measured take with no qualifying jump counts as zero.
 The detector is not Praat's analysis: its values can differ from Praat's.
 
-A take that trips is drawn once more, as the next seed, salt + k after any
+A take that trips is drawn once more, using the next seed under the [seed rule](#seeds) after any
 silence redraws, if t + d fits the budget. The redraw is sent only if it does not
 end as `silence` and has a strictly smaller jump; ties retain the first take.
 A `length` redraw can replace a `stop` take. An unmeasured redraw can win only

docs/speech-sampling.md

@@ -0,0 +1,54 @@
+# Speech sampling
+
+`POST /v1/audio/speech` accepts these optional fields for Drama 3. Omission or
+`null` keeps the existing default for that field. They apply to WAV replies,
+whole PCM replies, streamed PCM and catalog voices.
+
+| Field | Accepted JSON value | Default |
+| --- | --- | --- |
+| `temperature` | A finite number at least 0, within the `f32` range. A positive value must remain positive when converted to `f32`. | `1` |
+| `top_p` | A finite number greater than 0 and at most 1, still greater than 0 after conversion to `f32`. | `0.9` |
+| `top_k` | An integer from 1 through 64. | `30` |
+| `seed` | An integer from 0 through 18446744073709551615. | The existing hash of the prompt and `TENSORFOLD_SEED_SALT`. |
+
+Numbers encoded as strings and booleans are refused. `top_k` and `seed` must
+use integer JSON syntax: `30.0` and `1e3` are refused. Seeds are parsed as exact
+64-bit unsigned integers, including values above 9007199254740991; clients
+whose JSON number type loses precision must preserve the integer when encoding
+the request. Invalid fields return HTTP 400 before submission to the engine.
+
+For `temperature` and `top_p`, the JSON parser first produces an `f64` value;
+the bounds above apply to that value and its conversion to `f32`. Extremely
+small floating literals such as `1e-400` and `-1e-400` already become zero in
+that parser, so both request temperature 0; they are refused for `top_p`.
+A positive parsed value such as `1e-100` is refused if it becomes zero only
+when narrowed to `f32`.
+
+```sh
+curl http://127.0.0.1:8080/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"local-model","input":"Hello there.","temperature":0.8,"top_p":0.9,"top_k":30,"seed":42}' \
+  --output speech.wav
+```
+
+The route passes these settings to the native sampler. On Metal and CUDA, `temperature`
+of 0 or `top_k` of 1 selects its existing greedy path, which bypasses repetition
+aware sampling (RAS). Other runtimes may handle repetition differently; matching
+fields and seeds do not promise identical audio across backends or checkpoints.
+The HTTP limit of 64 candidates fits both native Metal and CUDA admission.
+
+An explicit `seed` is used exactly for the first take. If configured
+[speech redraws](speech-redraws.md) admit another take, its seed is the original
+seed plus the take's attempt number (1, 2, ...), wrapping modulo 2^64. This
+number includes both silence and register redraws. All takes keep the requested
+sampling values, including when controls are stripped or a reference prompt is
+rebuilt. With no seed, the existing prompt hash and server salt plus attempt
+number are unchanged. Streams keep their existing no-redraw behavior.
+
+Seed selection is deterministic. The final take can still change if timing or
+request deadlines admit a different number of redraws.
+
+The API design references the public
+[Sogni-AI/drama3-mlx generation interface](https://github.com/Sogni-AI/drama3-mlx/blob/ecc235039ccd5f982f66a6982184f57432354ab4/drama3_mlx/model.py#L79).
+TensorFold's parsing and request propagation are an independent implementation;
+this does not change the native sampling algorithms.

zig/src/families/drama3/prompt.zig

@@ -226,3 +226,32 @@ test "reference cache: last reference code and equal-length changed tower lookah
     first.frames[saved_rows][0] += 1;
     try t.expect(!try prefix.begin(t.allocator, first.frames, saved_rows));
 }
+
+test "reference cache: a reference speaker change misses while request speaker changes reuse the prefix" {
+    const t = std.testing;
+    const cache = @import("reference_cache.zig");
+    var prefix: cache.Prefix = .{};
+    defer prefix.deinit(t.allocator);
+    const codes = [_][cfg.codebooks]i32{@splat(1)};
+    var refs = [_]Reference{
+        .{ .transcript = "<|speaker:1|> First.", .codes = &codes },
+        .{ .transcript = "<|speaker:15|> Second.", .codes = &codes },
+    };
+    const first = try build(t.allocator, ByteTokenizer{}, "<|speaker:1|> One.", &refs);
+    defer t.allocator.free(first.frames);
+    const saved_rows = cache.prefillRows(first.frames.len, first.reference_rows, 1);
+    try t.expect(!try prefix.begin(t.allocator, first.frames, saved_rows));
+    prefix.saved();
+
+    const request = try build(t.allocator, ByteTokenizer{}, "<|speaker:15|> Two. <|speaker:1|> Three.", &refs);
+    defer t.allocator.free(request.frames);
+    try t.expectEqual(first.reference_rows, request.reference_rows);
+    try t.expect(try prefix.begin(t.allocator, request.frames, saved_rows));
+
+    refs[0].transcript = "<|speaker:2|> First.";
+    const changed = try build(t.allocator, ByteTokenizer{}, "<|speaker:1|> One.", &refs);
+    defer t.allocator.free(changed.frames);
+    try t.expectEqual(first.frames.len, changed.frames.len);
+    try t.expectEqual(first.reference_rows, changed.reference_rows);
+    try t.expect(!try prefix.begin(t.allocator, changed.frames, saved_rows));
+}

zig/src/server/drama_prompt.zig

@@ -63,36 +63,13 @@ fn speakerId(digits: []const u8) ?u8 {
     return if (n <= max_speaker_id) n else null;
 }
 
-/// Why input cannot be spoken, in the reference's words and order, or null.
+/// Why input cannot be spoken; canonical speaker tags 0..15 are accepted in requests and references.
 pub fn problem(a: Allocator, input: []const u8) Allocator.Error!?[]const u8 {
-    // speech_request.py: well-formed tags naming a speaker other than 0 (malformed ones are the builder's)
-    var others: u16 = 0;
-    var at: usize = 0;
-    while (std.mem.indexOfPos(u8, input, at, "<|speaker:")) |start| {
-        if (tagAt(input, start)) |tag| {
-            if (speakerId(tag.digits)) |n| if (n != 0) {
-                others |= @as(u16, 1) << @intCast(n);
-            };
-            at = tag.end;
-        } else at = start + 1;
-    }
-    if (others != 0) {
-        var list: std.ArrayList(u8) = .empty;
-        try list.appendSlice(a, error_prefix ++ " only speaker 0 is supported; got speaker tags [");
-        var first = true;
-        for (1..max_speaker_id + 1) |n| if (others & (@as(u16, 1) << @intCast(n)) != 0) {
-            if (!first) try list.appendSlice(a, ", ");
-            try list.print(a, "{d}", .{n});
-            first = false;
-        };
-        try list.append(a, ']');
-        return list.items;
-    }
     // prompt.py's validate_caller_text(text, "input")
     if (trimStart(input).len == 0) return error_prefix ++ " input: text must not be empty";
     for (structural_tokens) |token| if (std.mem.indexOf(u8, input, token) != null)
         return try std.fmt.allocPrint(a, error_prefix ++ " input: {s} is written by the prompt builder and must not appear in caller text", .{token});
-    at = 0;
+    var at: usize = 0;
     while (std.mem.indexOfPos(u8, input, at, "<|speaker")) |start| {
         const tag = tagAt(input, start);
         if (tag == null or speakerId(tag.?.digits) == null)
@@ -137,27 +114,30 @@ test "speaker 0 tags untagged text; a leading tag after white space is kept" {
         .{ "\n<|speaker:0|> hi", "\n<|speaker:0|> hi" },
         .{ "\u{3000}<|speaker:0|> hi", "\u{3000}<|speaker:0|> hi" },
         .{ "a <|speaker:0|> b", "<|speaker:0|> a <|speaker:0|> b" },
+        .{ "a <|speaker:15|> b", "<|speaker:0|> a <|speaker:15|> b" },
         .{ "\x1c<|speaker:0|> hi", "<|speaker:0|> \x1c<|speaker:0|> hi" },
         .{ "\x1f", "<|speaker:0|> \x1f" },
     };
     for (cases) |c| {
         try std.testing.expect(try problem(a, c[0]) == null);
         try std.testing.expectEqualStrings(c[1], try tagged(a, c[0]));
     }
+    for (0..16) |id| {
+        const dialogue = try std.fmt.allocPrint(a, "\u{3000}<|speaker:{d}|> First. <|speaker:0|> Reply.", .{id});
+        try std.testing.expect(try problem(a, dialogue) == null);
+        try std.testing.expectEqualStrings(dialogue, try tagged(a, dialogue));
+    }
 }
 
-test "input the reference refuses" {
+test "empty input, malformed speaker tags and structural tokens are refused" {
     var arena: std.heap.ArenaAllocator = .init(std.testing.allocator);
     defer arena.deinit();
     const a = arena.allocator();
-    const malformed = [_][]const u8{ "<|speaker:99|> x", "<|speaker:16|> x", "a <|speaker:99|> b", "<|speaker:01|> x", "<|speaker:00|> x", "<|speaker:|> x", "<|speaker0|> x", "hi <|speaker", "<|speaker:abc|> x", "<|speaker:0> x", "  <|speaker:abc|> hi" };
+    const malformed = [_][]const u8{ "<|speaker:99|> x", "<|speaker:16|> x", "a <|speaker:99|> b", "<|speaker:01|> x", "<|speaker:00|> x", "<|speaker:|> x", "<|speaker0|> x", "hi <|speaker", "<|speaker:abc|> x", "<|speaker:0> x", "  <|speaker:abc|> hi", "<|speaker:2|> x <|speaker:01|>" };
     for (malformed) |text| {
         const message = (try problem(a, text)) orelse return error.TestUnexpectedResult;
         try std.testing.expect(std.mem.startsWith(u8, message, "Drama 3 request: input: speaker tags must be written <|speaker:N|> with N in 0..15 and no leading zero; got "));
     }
-    try std.testing.expectEqualStrings("Drama 3 request: only speaker 0 is supported; got speaker tags [1, 15]", (try problem(a, "<|speaker:15|> a <|speaker:1|> b <|speaker:1|> c <|speaker:0|> d")).?);
-    // other speakers are refused before malformed tags, as speech_request.py checks them first
-    try std.testing.expectEqualStrings("Drama 3 request: only speaker 0 is supported; got speaker tags [2]", (try problem(a, "<|speaker:2|> x <|speaker:01|>")).?);
     for ([_][]const u8{ "", " ", "\u{3000}\u{2028}\t" }) |blank|
         try std.testing.expectEqualStrings("Drama 3 request: input: text must not be empty", (try problem(a, blank)).?);
     for (structural_tokens) |token| {

zig/src/server/drama_voice_tool.zig

@@ -263,7 +263,7 @@ test "voice tool: a synthetic tone's encoder codes become an attested file the c
     const out = try c.path("tone.json");
     try c.tmp.dir.writeFile(std.testing.io, .{ .sub_path = "unrelated.txt", .data = "unrelated" });
     try c.tmp.dir.symLink(std.testing.io, "unrelated.txt", "tone.json", .{});
-    const texts = [_][]const u8{ "A small cloud passes.", "Rain taps on a tin roof." };
+    const texts = [_][]const u8{ "<|speaker:0|> A small cloud passes.", "\u{3000}<|speaker:15|> Rain taps on a tin roof." };
     try std.testing.expectEqual(@as(u8, 0), try c.call(&.{&.{
         "--codec",       "synthetic.pth", "--generate",    encoder,
         "--ref-audio",   wav,             "--ref-text",    texts[0],