[1/3] Preserve speech controls and CJK boundaries in long-form chunks

CHANGELOG.md

@@ -4,6 +4,9 @@
 
 ### Fixed
 
+- CUDA NVFP4 long-form splitting preserves paired delivery tags, bracketed cues and special tokens,
+  including topic-shift handling, and recognizes CJK sentence/clause punctuation. An indivisible
+  protected span may exceed the character target; existing prompt and frame limits still apply.
 - Metal reference-prefill reuse now invalidates before incompatible/default prefill, compares the
   complete invariant prefix plus tower lookahead, preserves original chunk boundaries, and reuses
   one GPU-owned snapshot buffer pair per lane instead of allocating on every voice switch.
@@ -19,6 +22,10 @@
 
 ### Added
 
+- A checksummed CUDA NVFP4 engine patch series without changing its pinned upstream or weights.
+  Model-free build tests exercise real patch application, selection on Linux/macOS, corruption
+  refusal, repeat builds and preservation of modified checkouts. Manifests must cover each
+  selected patch exactly once; extra, missing and duplicate entries fail before checkout.
 - Native Apple-silicon support for the pinned NVFP4-pruned checkpoint: packed experts, compact
   original-ID routing and reference-conditioned semantic CFG, with independent backbone states.
   The Mac engine uses `engine/nvfp4/macos/UPSTREAM` and its checksum-verified patches; the existing

NOTICE

@@ -4,17 +4,18 @@ Copyright 2026 plotarmordev
 License
 -------
 All code in this repository is licensed under the Apache License, Version 2.0 (LICENSE). That covers the recipe,
-scripts, benchmark and qualification tools, results, documentation and the engine patches in engine/patches/
-and engine/nvfp4/macos/patches/.
+scripts, benchmark and qualification tools, results, documentation and the engine patches in engine/patches/,
+engine/nvfp4/patches/ and engine/nvfp4/macos/patches/.
 
 TensorFold
 ----------
 The engine patches apply to TensorFold (https://github.com/ashhart/TensorFold), which is licensed under the
 Apache License, Version 2.0. scripts/build-engine.sh clones TensorFold at the commit pinned in engine/UPSTREAM
 and applies the patches with `git am`. On Linux, --weights nvfp4 instead clones the TensorFold fork
 https://github.com/plotarmordev/TensorFold at the commit pinned in engine/nvfp4/UPSTREAM, also Apache-2.0,
-and applies no patches. On Darwin, --weights nvfp4 uses upstream TensorFold at engine/nvfp4/macos/UPSTREAM
-and applies the ordered patches in engine/nvfp4/macos/patches/.
+and applies the ordered patches in engine/nvfp4/patches/ after checking their SHA256SUMS manifest. On Darwin,
+--weights nvfp4 uses upstream TensorFold at engine/nvfp4/macos/UPSTREAM and applies the ordered patches in
+engine/nvfp4/macos/patches/.
 
 The patches modify TensorFold files. They add opt-in serving features and Drama 3 fixes: new files under
 zig/src/server/, zig/src/core/, zig/src/native/, zig/kernels/metal/, zig/tests/, docs/ and tools/zig/,

README.md

@@ -26,7 +26,7 @@
 
 [TensorFold](https://github.com/ashhart/TensorFold) runs Drama 3 natively since v1.1.0. This repository pins
 upstream TensorFold and adds a small series of opt-in patches for the full model. NVFP4 retains its
-measured CUDA engine pin and uses a separate upstream pin plus Metal patches on Mac. It builds the engine
+measured CUDA engine pin with a checksummed patch series and uses a separate upstream pin plus Metal patches on Mac. It builds the engine
 and starts a local speech server with an OpenAI-style `/v1/audio/speech` route, whole or streamed.
 
 **Built with Fish Audio.** The model weights are not in this repository; the launcher downloads them from
@@ -64,7 +64,8 @@ For the full model, write `--weights full` in both commands.
 The first start downloads the weights at the revision pinned in [`recipe/sources.json`](recipe/sources.json),
 into `drama-3-NVFP4-Pruned/` or `drama-3/`. Every start checks them against [`recipe/checksums`](recipe/checksums)
 and refuses missing or changed files; for the 21 GiB NVFP4 weights that takes about 11 seconds on a GB10.
-On Linux, `--weights nvfp4` builds [`engine/nvfp4/UPSTREAM`](engine/nvfp4/UPSTREAM) and serves with
+On Linux, `--weights nvfp4` builds [`engine/nvfp4/UPSTREAM`](engine/nvfp4/UPSTREAM) plus
+[`engine/nvfp4/patches`](engine/nvfp4/patches) and serves with
 guidance γ 1.5 and batched guidance, as measured; a `--cfg-gamma` of your own after `--` replaces both.
 On Mac it builds [`engine/nvfp4/macos/UPSTREAM`](engine/nvfp4/macos/UPSTREAM) plus its patch series.
 Metal's `--cfg-batched` pairs independent states in one command buffer, not a vectorized GEMM.
@@ -139,6 +140,23 @@ Guidance-enabled replies carry the configured γ in `X-Cfg-Gamma`; builtin voice
 
 ## The patches
 
+### CUDA NVFP4 long speech
+
+The CUDA NVFP4 patch series preserves delivery markup when `--long-form` splits a request.
+Paired tags such as `<whisper>…</whisper>`, bracketed cues and special tokens stay whole;
+Chinese/Japanese sentence and clause punctuation are recognized. A topic-shift mark inside a
+protected span does not create a cut. Plain English without these spans retains its existing cuts.
+
+`--long-form-chars` remains a code-point target, with one exception: an indivisible protected span
+may exceed it. The engine's prompt, segment-count and generated-frame limits still apply.
+The full-model and Metal NVFP4 engines do not expose this legacy long-form route.
+
+The behavioral reference is [Sogni-AI/drama3-mlx's chunking](https://github.com/Sogni-AI/drama3-mlx/blob/ecc235039ccd5f982f66a6982184f57432354ab4/drama3_mlx/chunking.py).
+The native patch extends TensorFold's own implementation; it does not copy that Fish-licensed source.
+See the [CUDA patch index](engine/nvfp4/patches/README.md) for scope and tests.
+
+### Full-model engine
+
 The patches build the full-model engine (`--weights full`). Upstream TensorFold already serves Drama 3. Each patch
 below adds one feature that stays off until you pass its flag, so a server started without them answers as
 upstream does. The exceptions are the first two: batched prefill is on by default (`TF_DRAMA_PREFILL_ROWS=1`

engine/nvfp4/UPSTREAM

@@ -1,5 +1,5 @@
-# The engine `scripts/build-engine.sh --weights nvfp4` builds, unpatched: the only engine that loads the NVFP4
-# pruned weights (plotarmordev/drama-3-NVFP4-Pruned). Commit b1f4b9b4 on branch drama-nvfp4 of the TensorFold fork;
+# The CUDA engine base for `scripts/build-engine.sh --weights nvfp4`, followed by patches/.
+# Commit b1f4b9b4 on branch drama-nvfp4 of the TensorFold fork loads the NVFP4 pruned weights;
 # results/full-vs-quant-01 measured the same source (v1.0.11-193-g70dec469; the two differ in comments and test
 # wording only).
 TENSORFOLD_REPOSITORY=https://github.com/plotarmordev/TensorFold.git