Repository navigation
Conversation
Whq02
added a commit
to Whq02/TensorFold
that referenced
this pull request
Nov 7, 2026
Whq02
force-pushed
the
mercury/docs-memory-lag
branch
from
November 7, 2026 12:16
281721b to
9ba6a08
Compare
…he_plan
docs/api.md said /health's `memory` is `{}` on Metal. The Metal families with a memory reader (Nemotron, Qwen3.5-2B,
Qwen3.8-27B and Qwen3.5-MoE through qwen27_served.zig, Flash Next, GLM-5.3-Flash and GLM-5.3 (EXLR8)) answer `active`,
`cache` (0) and `peak` from the process footprint, as status_routes writes whatever the engine reads, and Flash Next and
both GLMs add `limit`, their load limit; only the families without a reader answer `{}`. docs/memory-diagnostics.md
listed Nemotron, Flash Next and Qwen3.8-27B as the families that report `prompt_cache_plan`: Qwen3.5-MoE (the 27B's
host), Gemma 4 (the plan when its budget is positive), LFM2.5-24B-A2B and LFM3-1.7B (the plan at any budget) and the 2B
with an explicit `--prompt-cache-gib` (the plan at any budget, `budget_bytes: 0` included) report it too. The row and
the zero-budget sentences now say what the hosts do.
Whq02
force-pushed
the
mercury/docs-memory-lag
branch
from
November 7, 2026 12:34
9ba6a08 to
443e1a0
Compare
7 of 8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Two places in the docs say something the server does not do. Both were read beside the server's own JSON (1.2.2 on a
Mac with Qwen3.5-2B) and the code on main at 342b237.
docs/api.md's health table saysmemoryis{}on Metal. A Metal family with a memory reader answers the processfootprint (the 2B, served with
--prompt-cache-gib 0):$ curl -s http://127.0.0.1:8080/health
{"status": "ok", "model": "qwen3.5-2b", ..., "memory": {"active": 2837990136, "cache": 0, "peak": 2837990136}, ..., "backend": "metal", ...}
status_routes.zigwrites whateverengine.memory()reads and{}when it reads nothing.cache_fit.hostMemoryisthe reader of Nemotron (
metal.zig), the 2B (qwen35.zig), the 27B and Qwen3.5-MoE (qwen27_served.zig, the hostmetal.zigopens both through), Flash Next (flashnext_batch_host.zig,flashnext_host.zig), GLM-5.3-Flash(
glm_host.zig) and GLM-5.3 (EXLR8) (glm_dsa_host.zig); Flash Next and the two GLMs pass their load limit, whichthe route writes as
limit. The families without a reader (Gemma 4, LFM2.5, LFM3, Llama, MiniMax, Spark, Step-5,Kimi, Qwen4, and Drama 3, whose
drama_host.zigreads memory on CUDA only) are the ones that answer{}. The row nowsays that, keeping the
limitclause the table already had.docs/memory-diagnostics.mdlists Nemotron, Flash Next and Qwen3.8-27B as the families that reportprompt_cache_plan. On a Mac the hosts that publish it are those three and Qwen3.5-MoE (qwen27_served.zig, the27B's host: the plan when the budget is positive, omitted at 0, as the page already said of the 27B), Gemma 4
(
gemma4.zig: the same rule), LFM2.5-24B-A2B (lfm2_moe.zig) and LFM3-1.7B (lfm3.zig), both at any budget, 0included, and the 2B with an explicit
--prompt-cache-gib(qwen35.zig: the plan at any budget, the store only whenit is positive). With
--prompt-cache-gib 0:$ curl -s http://127.0.0.1:8080/memory
{..., "prompt_cache_plan": {"source": "physical_footprint", "budget_bytes": 0, "explicit_budget": true, "over_cap": false, "ram_bytes": 51539607552, "ready_footprint_bytes": 3356707864, "cap_bytes": 36077725250, "room_bytes": 30332138026, "margin_bytes": 2147483648}}
The table row and the zero-budget sentences now name them; the sentence #2367 added about the 2B keeping prompt states
only with
--prompt-cache-gibstays, with thebudget_bytes: 0case beside it.Nothing else on either page changes.
Mercury CLI integrates TensorFold and reads
/healthand/memoryto show what a served model holds; it was writtento the sentences and then read the JSON.
atc overlap --json: run 2026-11-07T12:17Z on the previous head againstorigin/mainat 16e044d; rebased once more onto 342b237 (two Unreleased lines landed meanwhile; this line now sits at the bottom of the block,git merge-treeclean) (ATC 0.9.0;complete: false,pullFilesLimit).changedFiles3. On the two pages: ten open PRs sharedocs/api.md(GLM-5.3-Flash on CUDA keeps conversation states on disk, every rank's part (--learn-conversations) #1411, lfm3: add native Metal vision support above the text decoder #1502, Nemotron 3.5 Transcribe: audio input, the 65,536-token window, an hour of audio, MTP drafts #1632, dsv41: serve structured output with the lane core's grammar masks #1665, server: an explicit thinking off that a template can't honour is refused #2170, server: POST /v1/images/generations for an engine with an image capability (#1573) #2253, moss image: serve pictures on CUDA (encoder, decoder and host for #1573) #2281, Qwen3.8-27B on a Mac: the disk staging permit takes one whole state at its context #2388, GLM on CUDA: the latent split's per-rank page book and owner (TF_GLM_KV_SPLIT, #1431 part 1; stacked on #1411) #2415, docs: the end-token pair for replies that quote a template marker #2437);git merge-treeof each against this head and against main alone gives the same result for every one (lfm3: add native Metal vision support above the text decoder #1502 conflicts indocs/api.mdand its LFM3 files either way; server: POST /v1/images/generations for an engine with an image capability (#1573) #2253, Qwen3.8-27B on a Mac: the disk staging permit takes one whole state at its context #2388 and docs: the end-token pair for replies that quote a template marker #2437 merge clean; the others conflict in their own files either way), so nothing conflicts on the pages because of this change; no open PR changesdocs/memory-diagnostics.md. OnCHANGELOG.md: the read lists lfm2_moe: serve LFM2.5-24B-A2B on CUDA (plain serve, shared kernels) #1492, Nemotron Parse on HIP: NVIDIA's checkpoint, token-exact against its HF example #1941, LightOnOCR-3 (0.8B, 1B, 4B) on native HIP, with the model card's oracle #1965, profile: register the pool-page switches for Dory, Gemma 4, Spark2.5 and Nemotron's Metal paging #2306, feat(hip): tap a drafter's layers into each stream's ring #2314, qwen3_5 cuda: readout streams fill in pieces of final-norm rows (#2324 step 2) #2384, CUDA: audit sm_110 cubins without widening release records #2395 and docs: the end-token pair for replies that quote a template marker #2437 asconflicts, each inCHANGELOG.mdonly (every one adds its own line to the same Unreleased block), the same shape as cli: tensorfold models --json, the listed checkpoints as one document #2364 and server: a refused API key file logs the path and the mode it read #2365 had.Checks on this head
python3 tools/zig/lean_check.py, on the base and on this head:python3 tools/pr_check.py --base origin/main --pr 2439on this head: merge base 342b237, 3 changed files, lean 116 on the base and 116 on this head, 0 new; 0 errors, 0 warnings. The pages are outside the checker's trees (zig,tools/zig); no Zig file changes.git diff --check origin/main..HEADclean;zig build test-golden --summary allon the docs commit's tree (the two pages unchanged since):Build Summary: 5/5 steps succeeded, 322/335 and 55/55 equal to the server goldens, 0 unexpected differences (nothing in the tree reads these pages; the 13 are its recorded known differences).benchmark-clients (ubuntu-latest)andzig-macos-dev-toolsgreen when the head moved.zig build test; no source changes.Receipt
n/a: documentation; the
/healthand/memorybodies this corrects are quoted above.Checklist