Repository navigation
Conversation
The recipe's measurements covered only the MLX affine checkpoint. Add the ModelOpt one, measured with beec1aa built unpatched for sm_121: decode and concurrency from bench_openai and bench_concurrent, cold first tokens from bench_cache with exact token arrays, the served exactness checks, the seams at fixed histories, and what the 128 GB machine admits.
|
The
|
|
Thanks @akol1 for carrying @balancedscorpion's measurements with their authorship. The supplied #952 write-up supports the code rates, cold prompts, memory figures and equality checks; I couldn't match the concurrent-chat rates to a receipt there, so please attach that receipt or omit that row. We'll add: “These are @balancedscorpion's measurements on Posted by Ash's AI Assistant Synthena |
…ncy row The ashhart#952 write-up records the concurrent aggregate for code at 1 / 4 / 8 streams, not for chat, so the chat row is dropped rather than carried without a receipt. The section now says whose measurements these are, which build they ran, and that 1.1.2 has not remeasured them.
|
Row dropped and the provenance sentence added at The #952 write-up's concurrency receipts are The section now reads: "These are @balancedscorpion's measurements on Documentation only, so no test receipt; |
…dits The 1.1.2 section had no entries for the contributed pull requests carried into this release, so neither their changes nor their authors reached the release notes. Add an entry for each, written from its commits: Flash Next's CUDA prompt tiles and structured output (#1046, #1062), --thinking-reserve, the long-loop guard and the --vision template diagnostic (#1064, #1075, #1076), GLM's Metal eh_proj tile and CUDA waiting ranks (#1066, #1053), M1's simdgroup cap (#1055), Nemotron's CUDA image and shared-window fixes (#1057, #1058), Qwen's live-row expert routing (#1047), the shared CUDA operators and contract fixture (#1056, #1068, #1070, #1071, #1073), the ROCm receipt, drafter and draft-count work (#1043, #1044, #1045), the ModelOpt recipe numbers (#1042), the Windows host build (#1034) and the session store's allocation-free rollback (#637). Thank #643's and #1060's reporters as the other entries thank theirs.
|
The ModelOpt documentation is included in 1.1.2 (https://github.com/ashhart/TensorFold/releases/tag/v1.1.2), with the original authorship of both commits retained: the measurements and your follow-up that dates them. It labels the single-GB10 measurements as results from the earlier build, not a new 1.1.2 run. Closing this as carried, and thank you. It landed as 923aeae and 177f35b. - Synthema, Ash's AI Assistant. |
What this changes
Adds the ModelOpt
nvidia/Qwen3.8-Flash-Next-FP4-Localcheckpoint todocs/recipes/flash-next-cuda.md. The recipe'smeasurements covered only the MLX affine checkpoint; this is one new "### The ModelOpt checkpoint on one GB10"
section with the decode and concurrency table, the cold first-token timings, the served exactness checks and the
memory seams.
The commit is unchanged from #952 (
b6fd0d92, authored by @balancedscorpion). The 1.1.0 merge53abe455took theearlier head
d71a2361, so this docs commit and the experts-rows test fix were left out. The test fix is going in asits own test-only PR so this stays one change. Documentation only: no source, tests, fixtures or generated files
change.
Review follow-up
The concurrent-chat row is dropped: the #952 write-up records the 1 / 4 / 8-stream aggregate for code only, so there is no receipt for the chat numbers. The section now dates the measurements as @balancedscorpion's, on
beec1aad6, the #952 branch build before 1.1.0, not remeasured on 1.1.2. Diff:c44892e5f..1613ff9d7, documentation only.Receipt
b6fd0d92on top of76aafcf27; documentation only. The measurements are the commit author's:GB10 (128 GB, driver 580.178, CUDA 13.3),
beec1aad6built unpatched for sm_121, checkpointnvidia/Qwen3.8-Flash-Next-FP4-Localatb9ce18a6,--prompt-cache-gib 0, with the recipe'sbench_openai,bench_concurrentandbench_cache. They are quoted as measured; nothing was generated here.author's receipts (each concurrent reply equals its solo run, drafted equals
"draft": false, cancelled/restartedequal fresh, cold
ttft_s, peak host and device memory).receipt").
git diff --checkis clean. The lean checker does not readdocs/.a new claim. Host and CUDA CI on the PR covers the tree.
Checklist
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (docs only)SUMMARYline is in the receipt. (no kernel or backend change)