Repository navigation
Conversation
…m nothing A pull request reuses main's newest cache, but main built from nothing, so during a landing burst its saves fell several commits behind and one shared-source commit in between undid most of the reuse. Main now restores its newest cache like a pull request and saves it again, and zig_cache_restat --prune deletes the entries a checkout can't hit (a changed or missing file, or another dead entry's output), so the saved cache stays about one build. A nightly run skips every restore, so a pass reuse could hide, such as a test reading a file the build doesn't know of, fails within a day, and its cache starts the next day clean.
They set the server's defaults for the request fields of the same names (server.zig:264-292 resolveSampling merges Config.default_sampling under the request's fields; the request wins). The defaults are resolved from three layers (family_sampling.zig:21-35): a family's built-in sampling — only DeepSeek-V4/V4.1 have one, temperature 1 and top_p 1, the model cards' recommended sampling (family_sampling.zig:13-18) — then the checkpoint's generation_config.json temperature/top_k/top_p/min_p with do_sample false meaning temperature 0 and do_sample true without a temperature meaning 1 (family_sampling.zig:24-29), then the serve flags, each set one replacing the layers below (family_sampling.zig:31-34). Nothing set means no default (family_sampling.zig:35); the serving line prints the resolved defaults or greedy (serve.zig:338-348 shownSampling). The flags parse without range checks (cli.zig:457); the merged values are validated per request like request fields (server.zig:267-270 via fields.zig:60-77): min_p must be 0..1 (fields.zig:71), a negative top_k clamps to 0 (fields.zig:73), temperature and top_p must be finite (fields.zig:70). A greedy-only engine refuses while the server samples by default and suggests --temperature 0 (server.zig:273-276).
…flags Server-side conversation compaction for chat completions (compact.zig, reached from chat.zig:279-282 and openai.zig:494-563; the Responses API runs through the same handler, responses.zig:193). Off without --compact-at (compact.zig:105-108): an over-long conversation keeps the usual context_length refusal. --compact-at parses auto or a fraction in (0, 1] (cli.zig:458-461). auto reserves max(15% of the window, 16384) tokens for the reply and compacts when prompt plus wanted reply tokens cross what is left (compact.zig:47-52 reserveTokens, :78-84 over, wanted :86-89); a fraction f thresholds at floor(f * window) (compact.zig:70-76). A plain prepare refused with context_length also compacts (compact.zig:114-119). rebuild (compact.zig:136-206) keeps the leading messages and a recent tail (--compact-keep tokens, an integer from 0, default min(20000, window/4): cli.zig:463-466, compact.zig:65-68), summarizes the middle into a note generated by the model itself, greedy with thinking off and max_tokens 1024 a chunk (compact.zig:259-261), under the note task's Markdown sections and 800 words (compact.zig:15-21), inserted as one message beginning with the marker line (compact.zig:13), retrying over smaller tails until the rebuilt conversation fits (compact.zig:190-202). --compact-memory DIR (cli.zig:467, no parse validation) stores <key>.md plus a <key>.json with the covered message count and prefix hash (compact_mem.zig:42-51); a later request whose prefix hashes the same reuses the note (compact_mem.zig:21-25, compact.zig:139-147). The reply's tensorfold object gains context_window, context_used and, when compacted, a compaction object with messages_compacted, kept_messages, note_tokens and note (compact.zig:122-134). Messages must be an array; other inputs are never compacted (compact.zig:110-113).
--loops N overrides the checkpoint's own n_recurrent_loops with N passes
over Dory's recurrent block (cli.zig:143, .native = cuda_build;
families/dory/cuda_weights.zig:110-114 load applies withLoops when the
flag is set). Parsed in the CUDA builds only (cli_gpu_flags.zig:19): a
negative count is refused there ("a count from 1"), 0 passes parse and
is refused by the Dory open with anything above max_loops
(families/dory/cuda_native.zig:106), whose refusal names the range
(cuda_native.zig:190). max_loops is 4, llama.cpp's limit and the card's
largest benchmarked count (families/dory/config.zig:6). The schedule at
L loops runs I + R*L + O steps — 16 + 40*L + 16 — with 8 + 20*L + 8
attention caches (families/dory/schedule.zig:44 build, test :96-107),
and a token's KV and stream state is 36, 56 or 96 KiB at 1, 2 or 4 loops
(families/dory/memory.zig test :75-80). A saved state carries its loop
count in its key and never crosses to another count
(families/dory/cuda_snap.zig:8-20). Non-readers refuse the flag by name:
other CUDA families (native/cuda_refusal.zig:66), HIP (native/hip.zig:89),
Kimi on Metal (native/kimi_metal.zig:81), and tensorfold run on any
non-Dory model (cli/cuda_main.zig:195-198).
--max-frames parses to an integer 1 through 8190, refused otherwise
(cli.zig:425-428), and reaches the engine as Open.max_frames
(cli.zig:527). Drama 3 validates it at open on both opens
(native/drama_open.zig:26,90 via frameProblem :111-118): the frame
budget is min(flag or 1024, context - 3) frames with the prompt capped
at min(4096, context - frames - 1) (native/drama_budget.zig:3-12); an
explicit value outside 1..8190 is refused ("Drama 3 supports
--max-frames 1 through 8190"), one at or above context - 1 leaves no
prompt room ("use --max-frames < --context - 1"), and the context
itself must be 1025 through 8192. The engine option defaults to 1024
frames when the flag is absent (native/drama_host.zig:385) and the
engine advertises the cap as SpeechInfo.max_frames with frame_length
2048 at 44100 Hz (native/drama_host.zig:405), which the speech route
passes to each request (server/speech.zig:197); the engines generate up
to the smallest of the request cap, the engine option and the context
room (families/drama3/metal/engine.zig:389,
families/drama3/cuda/engine.zig:543). Other engines read no
Open.max_frames and no refusal names the flag; video input is limited by
the vision families' own caps (VisionOpen.max_frames, default 256,
core/engine_vision.zig:82, has no serve flag).
Both parse into the silence guard's config (cli.zig:86-87, :432) and are validated at startup (cli.zig:319-320, "invalid silence guard configuration"): the two onset flags are required together (core/speech_guard.zig:23-25), onset_seconds must be above 0 and below --silence-seconds (speech_guard.zig:26, which requires the seconds flag) and onset_voiced_seconds must be a finite number above 0 (speech_guard.zig:27). The guard itself needs --silence-floor-dbfs and a positive --silence-seconds to be enabled (speech_guard.zig:30-32). In Guard.hear a reply trips when its silent run reaches the onset seconds while the total voiced audio stays under the voiced seconds (speech_guard.zig:95) — an early reply that never really began speaking; the startup sentence names the rule (speech_guard.zig:44). The flags are Drama-speech-only: any other model type is refused at startup by name (speech_guard.zig:34-38). An onset trip latches like the main silence limit (speech_guard.zig:96-97), so the finish is a silence trip and is redrawn like any other.
…uto compaction's 16,384 floor holds only where the window does (compact.zig:58)
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #2402 #2345 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.