bakana39 · Oct 23, 2026
diff --git a/docs/receipts/glm-fp4local-dual-gb10-reference-20261023/PUBLICATION_CHECKS.md b/docs/receipts/glm-fp4local-dual-gb10-reference-20261023/PUBLICATION_CHECKS.md
@@ -0,0 +1,29 @@
+# Publication-only checks, 23 October 2026
+
+These are archive integrity checks, not newly run inference or GPU qualification.
+
+- Source base and loader dependency are separated from local experimental increments.
+- The six patches applied sequentially to the pinned main base without rejects.
+- After applying all patches, 3383 selected native source/build files compared
+  byte-for-byte equal to the corresponding frozen v9 source snapshot, with zero
+  missing files and zero differences. Selected types: Zig, CUDA, headers, Metal,
+  build manifests and the Zig-version pin.
+- `git diff --check -- zig build.zig build.zig.zon .zig-version` on the replay
+  tree exited 0. This is not a full native build or a lean-check qualification.
+- The served-comparison script's CPU self-test passed, including deliberately
+  mismatched settings, tokens and missing evidence.
+- Recomputing the v5/v9-off/v9-on/reverse-v5 comparison from the sanitized
+  records reported all comparisons qualified, with zero error runs. Here
+  "qualified" means its listed recorded-evidence comparison checks only,
+  not the upstream kernel or model gates.
+- The public tree excludes weights, model shards, profiler binaries, credentials,
+  deployment controllers and real user prompt/reply bodies. Supplied prompts
+  are the public synthetic benchmark fixtures.
+- Recorded runtime hashes are preserved rather than presented as independently
+  recomputed hashes of unbundled generated output token arrays.
+- Code and receipts remain experimental. No new GPU run, service stop, restart,
+  model load or device configuration change was performed for publication.
+
+The final `SHA256SUMS` covers every bundled file other than itself. Private source
+archive/image identifiers in provenance identify historical measured builds;
+those archives/images are not included. Sanitized labels are not registry URLs.
diff --git a/docs/receipts/glm-fp4local-dual-gb10-reference-20261023/README.md b/docs/receipts/glm-fp4local-dual-gb10-reference-20261023/README.md
@@ -0,0 +1,194 @@
+# Reference only: GLM FP4-Local on two GB10s
+
+**Not a merge proposal. No review or merge is requested.** This archive preserves
+local experiments from 22–23 October 2026, including unsuccessful optimizations.
+The accompanying PR is intended to be closed by its author immediately after
+submission. It does not change TensorFold's runtime, defaults or support table.
+Maintainers have no obligation to review, reproduce, maintain or adopt it.
+
+## Attribution and review status
+
+The local implementation, test orchestration, analysis and this report were
+performed by OpenAI ChatGPT/Codex at the hardware owner's direction. The human
+operator tested the service but did **not** perform a line-by-line code review.
+This disclosure is not a claim of independent human validation, formal proof,
+upstream qualification, or endorsement by OpenAI or the TensorFold maintainers.
+
+TensorFold and its existing kernels remain the work of their original authors.
+In particular, the FP4-Local loader is borrowed from
+[Bizuayeu's PR #1050](https://github.com/ashhart/TensorFold/pull/1050), pinned at
+`b960da830a2722e39513f354211efe2f4531cd02` for these experiments; it is **not**
+claimed as new work by this archive. That PR has continued evolving independently.
+This archive is not a competing implementation or a request to close that PR.
+
+The dependency patch is separated from the local increments. It records the
+loader integration on the tested main base, not the entire original contributor's
+commit history. Do not reattribute it when reusing the experiment. Consult the
+original PR for its complete receipts and authorship. TensorFold's Apache-2.0
+license and required notices apply; model weights retain their own licenses.
+No weights, model packs, native binaries or third-party shared libraries are
+redistributed here. DFlash2's CC BY-NC-ND 4.0 terms must be evaluated separately.
+
+## Scope and tested configuration
+
+| Item | Recorded configuration |
+|---|---|
+| Source base | `f8d254c5e0f9ab91eb325a2e7c28d2e5cb07c94d` |
+| Engine | Native Zig/CUDA; not the frozen Python inference engine |
+| Hardware | Two NVIDIA GB10 hosts, 128 GB unified memory each, TP=2, RoCE |
+| Toolchain | Zig 0.17.0, CUDA compiler 13.0.88, architecture-specific SM121a |
+| Main checkpoint | `nvidia/GLM-5.3-Flash-FP4-Local` revision `76348ddb2be2e35e39ecfc00e75dc07703f4f3d5` |
+| Drafter | `incoai/GLM-5.3-Flash-DFlash2` revision `bf582e4eacc1810f76656d1811693ff6c6737d2a` |
+| Main precision | Stored mixed FP4/MXFP8/BF16/FP32; routed A4 prompt path, A16 verification/decode |
+| Other precision | Existing native FP8 latent KV; no extra Dense FP8 conversion |
+| Serving budget | 524288 context; four admitted streams; 2 GiB requested prefix cache; 6 GiB OS reserve |
+| Draft policy | Native DFlash2 Q4, adaptive up to seven drafts, cumulative confidence 0.7 |
+| Default sampling | Temperature 1, top-p 0.95, thinking enabled/high; test overrides explicitly recorded |
+| Storage | One read-only checkpoint copy, second host reads it over NFS; no weight-byte changes |
+
+Detailed OS/driver inventory is not supplied in this public bundle; compiler,
+binary, image and checkpoint identities are retained. Clocks were variable, not
+locked for benchmark qualification. These omissions limit reproducibility.
+
+The selected local baseline is **v5**, binary SHA-256
+`4c75b2b6182560bdbee9eef8267218bfaa975541464761a7ad0476c8fec2e0ad`.
+V6/V8/V9 were not promoted. Packaging this archive did not restart or change the
+running service. This is not a comparison claiming superiority to vLLM.
+
+## What was retained, and what was rejected
+
+1. **DFlash-only loading:** omit the unused MTP transformer while retaining the
+   same draft vocabulary projection. Recorded model allocations fell from 94.83
+   to 92.78 GiB per head-node receipt. This is not total process or system RSS.
+2. **Native A4 prompt path:** reuse the existing compact A16 weight pack, apply
+   each projection's own scales and GLM clipping, without a second weight copy.
+   A generic SM121 build initially trapped; architecture-specific SM121a and an
+   early native-MMA marker check corrected this. The failed launch is documented,
+   not counted as successful inference.
+3. **A4 N64 to N128 tiles:** retained after same-output comparisons. On the fixed
+   2K–66K prompt fixtures, prefill took about 14% less time (16–17% more tokens/s).
+   Decode did not improve. This comparison is within the existing A4 path; adding
+   A4 in the first place is not bit-equivalent to an A16-only prompt implementation.
+4. **Decode warp count and DFlash confidence tuning:** cache-resident microbench
+   gains or more accepted drafts did not translate into better served results.
+   Four warps and confidence 0.7 were retained; the cost override remains unset.
+5. **V6 copy/tail/gather changes:** KDA direct output, shared-expert tail fusion
+   and a window-wide sampled-logit gather passed listed local checks but did not
+   produce a repeatable served gain. They remain independent, default-off options.
+6. **V7 profiling:** target experts and KDA projections dominate the sampled
+   windows. GPU activity was about 95%; that is not proof of DRAM saturation.
+7. **V8 low-eight-row experts:** fewer registers did not yield a robust isolated
+   chain improvement; no complete model was loaded with this candidate enabled.
+8. **V9 KDA projections:** six pipeline/warp variants were screened. An existing
+   ordered non-cluster O reduction, keeping SK=4, was integrated experimentally.
+   Output hashes matched but served performance did not reliably beat frozen v5.
+
+## Representative results
+
+The public synthetic prose/HTML fixtures are not the operator's private tasks.
+The reported token rate includes reasoning tokens. Prefill rows below are
+cache misses after separate kernel warmup, not a fresh-process, cache-disabled
+full qualification sweep. All compared prompt rows report zero reused tokens.
+
+| Prompt tokens | V4 prefill seconds | V5 prefill seconds |
+|---:|---:|---:|
+| 2027 | 2.022 | 1.735 |
+| 8108 | 8.503 | 7.334 |
+| 32988 | 33.742 | 28.941 |
+| 66429 | 67.562 | 57.888 |
+
+Decode medians for the same short controlled fixtures were essentially flat:
+40.09→39.63, 40.51→40.18, 35.66→35.59 and 47.05→46.82 tokens/s.
+Do not describe v5 as a demonstrated decode acceleration.
+
+V9 fixed-token comparisons: thinking enabled/low, temperature 1, top-p .95,
+seed 119, 256 output tokens, two repetitions per cell. Identical prompt/output
+hashes and speculative counters were required before interpreting timings.
+
+| Workload | Frozen v5 | V9 off | V9 on | Reverse v5 |
+|---|---:|---:|---:|---:|
+| Prose 8K | 33.948 | 33.002 | 33.622 | 33.938 |
+| Code 8K | 44.372 | 42.516 | 44.144 | 44.388 |
+| Prose 32K | 30.182 | 29.581 | 29.718 | 30.315 |
+| Code 32K | 47.660 | 47.650 | 47.238 | 47.779 |
+| Prose 128K | 30.306 | 29.442 | 30.193 | not rerun |
+| Code 128K | 29.280 | 28.426 | 28.645 | not rerun |
+
+Reverse v5 has three repetitions per 8K/32K cell. All 36 new requests matched
+the prior v5 correctness/speculation evidence. Sequential run variation and
+recompilation effects limit causal attribution. The 128K code fixture produced
+only reasoning within the cap, not a finished HTML page. Cached-prefix lengths
+differ, so these rows cannot establish cold-prefill or total-wall-time gains.
+
+## Evidence and known gaps
+
+- V5 CUDA-host build: 1057 passed, 4 skipped. This subset is not the complete
+  upstream host/golden/fuzz/Metal/device qualification.
+- Listed local tests include draft/plain, fixed seed, four short solo/concurrent
+  replies, >32K prefix reuse, an interrupted SSE request followed by retry,
+  tools, structured output, direct images and tool-returned images.
+- One 521113-token checksum/capacity request succeeded on v3. Four concurrent
+  approximately 41K requests were separately exercised. Neither proves four
+  simultaneous full 512K requests, nor broad long-context reasoning quality.
+- Focused operator tests checked immutable-baseline bytes, five repeats, guard
+  bytes, unchanged inputs and sampled independent FP64 references. They do not
+  replace the complete unified kernel runner and its planted failure controls.
+- No official-backend teacher-forced fidelity qualification, full all-platform
+  kernel `SUMMARY`, or complete new-finding-free lean baseline is claimed.
+  The historical whole-tree lean check reported 439 findings; its full pinned
+  baseline comparison was not completed.
+- Small intermittent paging was recorded; idle zero swap-in/out is not a claim
+  of zero swap residency or an entirely paging-free run.
+- V9's wider vision/concurrency/cancel/512K matrix was not rerun after it failed
+  the performance promotion gate. V5 validation does not transfer automatically.
+
+The detailed reports are historical experiment notes. Some refer to additional
+private local artifacts not included here; their file names are not promises of
+public downloads. Only files listed in `SHA256SUMS` belong to this bundle.
+
+## Using the reference artifacts
+
+`patches/00-...` is the separately attributed loader dependency integration.
+`01-...` reaches frozen v5. `02`–`05` advance through the non-promoted v6–v9
+experiments. Apply only in a **new disposable checkout** at the pinned base,
+not directly to current main or to a running deployment:
+
+```sh
+git checkout --detach f8d254c5e0f9ab91eb325a2e7c28d2e5cb07c94d
+git apply /path/to/reference/patches/00-upstream-pr1050-dependency.patch
+git apply /path/to/reference/patches/01-v5-increment.patch
+# Stop here for the selected v5 source; later patches are experiments.
+zig build native test-cuda-host -Dnvcc=/usr/local/cuda/bin/nvcc -Dsm=121 -Dstrip=true -j4 --summary all
+```
+
+Use Zig 0.17.0 and qualify the resulting binary independently. No deployment
+controller, automatic image replacement or credential-bearing configuration is
+included. `local-reference/...` names in provenance are anonymized labels for
+local images, not published registries. Original image/archive hashes identify
+the measured private artifacts; they are not hashes of this sanitized bundle.
+
+Python scripts are development-only clients/analysis, not inference dependencies.
+For clients importing `upstream/tools`, arrange a sibling `upstream` checkout of
+the pinned engine, or add that directory to the import path. GPU harnesses need
+the indicated built fatbins and their original runtime dependencies; these are
+not redistributed. To recompute the included served comparison without a GPU:
+
+```sh
+python3 tests/compare_kda_served_v9.py --self-test
+python3 tests/compare_kda_served_v9.py --baseline v5 \
+  v5=evidence/decode-v7-baseline-observed.jsonl \
+  v9-off=evidence/decode-v9-off.jsonl v9-on=evidence/decode-v9-on.jsonl \
+  v5-return=evidence/decode-v5-after-v9.jsonl --output comparison.json
+```
+
+Public synthetic request bodies and exact prompt token IDs are retained. Generated
+reply/reasoning bodies and transport chunks are omitted; runtime token hashes,
+usage, speculative counters and timings remain. Hash equality is checked against
+the recorded hashes, not independently recomputed from unbundled output IDs.
+Private addresses, host names, user paths and image names are replaced with
+documentation examples. Credentials, model files and captured profiler binaries
+are excluded. The publication scan supplements, rather than replaces, inspection.
+
+This archive intentionally records unfinished and rejected work. Anyone adopting
+a patch must establish the current upstream gate, preserve attribution, review
+the implementation and requalify it on their own checkpoint and hardware.
diff --git a/docs/receipts/glm-fp4local-dual-gb10-reference-20261023/SHA256SUMS b/docs/receipts/glm-fp4local-dual-gb10-reference-20261023/SHA256SUMS
@@ -0,0 +1,68 @@
+76507460d134e0f3d289eeadecbe9fe5a20f7b0a415ffb6014bcc288885974f5  PUBLICATION_CHECKS.md
+56c8320db0f692a713b693dd8636f286e73a74d2269777a482fb1c228555120a  README.md
+39e09c00f4b6e304d3014260ec033d49824bddfc6b2bd12d733225da3750d7a3  candidates/qmmf-pipeline-variants-v9.cu
+e3299c87cce1b20f0121f90fefda8e4c0eb5f5082412585cbe841a97181c6f61  evidence/comparison.json
+68947987cc250e3cadc9ac58b034c3285dea1c1b268e67e1fc4bf61c20704d62  evidence/decode-v5-after-v9.jsonl
+ab4a2a427301b194778d24ba5d7ce0f13fe66193fdfeddb192676ee21c473bc1  evidence/decode-v7-baseline-observed.jsonl
+d80a43ba2aea78336ab4397d40c70a732e205781db94e17f248b3d1f3e16c7de  evidence/decode-v7-baseline-summary.json
+4f15678095ce7a61d80f1e5853bc4d7e39d039fd3ecabc84c3c85e18b415b7ff  evidence/decode-v7-fixtures.json
+ea05312f31edeb7d74a6b21bd92c85f23fe2b48f07687ef0bf460c669ac2a4e3  evidence/decode-v9-off.jsonl
+6595033c56536316489ae9e2bce4ec086add7b8d898b53f2ecb495595248cea8  evidence/decode-v9-on.jsonl
+3fe04c54f96cc071feace23a77b901e82924d1a4f6e72eed6683b99b2b9e8e64  evidence/events-v7-summary.json
+1b4d91e098d27bb51f3272bb09463150d3b5a4703c05392df4a815833d25d558  evidence/kda-v9-final-build-correctness.jsonl
+eacd5e3e0b7920aefc309d74d3bda715dccff181dce582d4cd2e6375a6e726e8  evidence/kda-v9-native-projection-f32.log
+72cb25542fcd11d60ed3af0f7dff442e1059ee8888b553ecd331d0b2aa07e9d1  evidence/nsight-window-audit.md
+03dadd42a9df3b2fbe38952c3b217478fde1af0fe775f3c4a4f350594c620847  evidence/perf-v4-baseline.jsonl
+6efedbec997549cff63b10e335034f8bf1a649fd841fe6e73b21daf02862c0fa  evidence/perf-v5-before-v6.jsonl
+98c0fc1d34c82c0937b2b54ab8d0ce58638ed1d667fcb72f932c3d33a72790db  evidence/perf-v5-final.jsonl
+b7a5f7c92eaa516acf111227f13b902f123a0394c7282df61fd0acc9c2da4b95  evidence/perf-v5-kernels.jsonl
+0a08498a11b313da04eaf3ab3d3aff7689599e6f5980a0b0dc6d760a787f6a8c  evidence/perf-v5-tuned.jsonl
+7a2cb55195d9b8ada1e2b91e25f04a91c0a8d164f0392684ee5ea4495350851e  evidence/perf-v6-all-repeat.jsonl
+6e4170d7bb7f4d86b86a48a777c672ae4bf4511ffdf65840d7fa22b339aab8a5  evidence/perf-v6-all.jsonl
+82812373c897a486d700d596ce944c9e23a546e51fd97c13ebb9cf1eec27da18  evidence/perf-v6-off-repeat.jsonl
+036a391064d74f07bb323758c09ac3f351994655603ea77b692441e27253df4c  evidence/perf-v6-off.jsonl
+5683841502a6dc01dad4e5b8affd778f2a02e1dcb0e71cbe67c53051314ce816  evidence/prefill-v6-all.jsonl
+cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  licenses/LICENSE
+cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  licenses/LICENSES/Apache-2.0.txt
+be6a9ee4462784a762b21790954b8e76c4dcacc9fb6da2a40d03a8a9bddec93e  licenses/LICENSES/MIT.txt
+ed01c9ef861bcb1c20f028739d8c7102c6f04364ffe9903e5088c6d6449a4aff  licenses/LICENSES/MiaAI-Lab-MIT.txt
+15181e7363dca9aed78b79bebebc7fde7f1814b8bd311ea3b87ae8ccadfc185b  licenses/LICENSES/Pillow-MIT-CMU.txt
+bd018feef8825e88181c84eb7e3aa4eafb8f08a20d9fd6ef948569610c4a3e43  licenses/LICENSES/PyTorch-BSD-3.txt
+74b59811a5ebc3c28e7fc89f8f189854b6ca6d44392a98394950b24ba3c1b8e7  licenses/NOTICE
+5913cada1f8bb978d95a8b76795fb127c5966713dc4db9203132d2a7b3f718b8  licenses/THIRD_PARTY_NOTICES.md
+01faf6749666fb49ab56c38c283d95dd40773c4afcd553d1290eb503b39e1d31  patches/00-upstream-pr1050-dependency.patch
+95bccb258b3862bcc1d9d4b7c7b3301d5233a30c6ffb0f173aa4f309d6803bc0  patches/01-v5-increment.patch
+03bec910d35d4d95eb1bda3e2866fece454f82ca206b9c6dcd88460dbe797c31  patches/02-v6-increment.patch
+40cf9ab0cb5786e9b2a25b7d67df896715beb55dad72683dca6089c320375ab4  patches/03-v7-increment.patch
+eacdfbdf24843becfee52399230cc4c390a26bff16df1b535c2e71a429670850  patches/04-v8-increment.patch
+5ab72518760b9dfcebcb0d96912ad1ca222263b2860bd223e78e838a804602c2  patches/05-v9-increment.patch
+814f9b73578a0a5964e2b2fc4c75d1ad6e3e42796a646e31bc8113bf73e09f7e  provenance/provenance-v3.json
+49d1a10c8b45115ce5d0f4f5cc0aee75c6dd6f2d520f12a1bbce0216e7545260  provenance/provenance-v4.json
+bc153030ea4944021a2271c9f927be9bece38387514ee2beb62459e45d2fcbf8  provenance/provenance-v5.json
+0856fa99b450dfe422b04b8c745d2aaa7c35856605f47d84da40d9e98a529926  provenance/provenance-v6.json
+73c001732b533f3f37917e87e1f3391fcc3530417bd788479b38321f98fae01b  provenance/provenance-v7.json
+2c390f2e08c5102aa925b8ed50bc841dc781b0bb4e6a47de3db33fde29cf37e1  provenance/provenance-v8.json
+7638c103e2a1a5eea92cbf532049a25c1946ff195c4577112c483f2f7b2a5a40  provenance/provenance-v9.json
+12a5c8f757c6a7ce123ee9bc4e06396439d4baa9797ee5935759a49d377dd16e  provenance/source-snapshots.json
+9b52677a1f526e0b544e167c190c1096a51e30f68de4bd89a09acf248a29cc38  reports/DECODE_OPTIMIZATION.md
+2048ef28c1be85ffe9efe498688ded1b3aad8a6c89b2d5119435d0107ef737aa  reports/DECODE_TIMELINE_V7.md
+121526d633f53ae0bd9270f103304c442bf2afa613aa4f7961e78163ab36fa89  reports/KDA_PROJECTION_V9.md
+c2e863ba65b1277041434eef96486c23a385442765c2cd56f823ce846feab25d  reports/OPTIMIZATION.md
+a590d3c65f983ed3fc70c4d977579d51f5d07e69dc9e87009cea4de760da4b42  tests/a4_bridge.cu
+275da62e4116aea60dc7de77937a08861ec3e399752612380a00ecb19c8eff95  tests/analyze_decode_v7.py
+de870886c8b9c6beda535216d5afb398d7f421619bcee9154ba72ad9b350e688  tests/analyze_nsys_families_v7.py
+b33019b0a5fa9321f1f86c29642e832ffc2e6526cce18efb4788bbc4c4eb8fd7  tests/analyze_nsys_v7.py
+64348876f468372eb45628ffba2928750b2fc652e33c69534e2d4cbbb3ce00a2  tests/bench_decode_v7.py
+0cbf053c67346ea9b57815c6fdc2fcc690379d5a1a826d1e4e22caaa7f689e4c  tests/bench_kda_direct.py
+b4297756985e7ed6e3d4712f5114284046f4eafeb09575d97aff95794e97126d  tests/bench_kda_projection_v9.py
+8db82ab2da59057a435dea1b8d6a5e6b986f794225d69e3fadfb844eea5fe57c  tests/bench_nvfp4_candidate_v8.py
+2eff4103b03f6137ac4e5fd152752327ebcc4d42d8229390ef8abd6c9ac210a0  tests/compare_kda_served_v9.py
+429a431487438af005bfca4256dd1e62892f4e377318e8f3ab407ca60e0916e3  tests/compare_perf.py
+11e487db60e4a2010467614bb229d3147906766731daa3fd01cdbf3092f0d6fc  tests/perf_parallel.py
+6679d8b32a28301303291d50d96054dbc7c52d412bfef8bf379283e9a5a31c6b  tests/perf_round.py
+037348d7fb52507a9aba08264e97f9f20fbe5d5f0c1ca329038b99232167a6f5  tests/served.py
+5ac05a56f73447482de79404e4bd884a1cbb485b218159f1c66012f46a8d9158  tests/summarize_decode_v7_baseline.py
+8aa3866e59e178dcba58fad2372cddce7b6930accb9834cda83ca43321e041df  tests/summarize_v6.py
+3c790d27aae74b89f77f82ed183ad9cba4fa5297edf7094bf9f1240878caf4c7  tests/test_a4.py
+5082f28cd22ae92872ddaec41e4406341701831978d2a72c7d138ceaaab2f776  tests/test_analyze_decode_v7.py
+a5e7333887555c5b526b49a73afeacbb2be31ddbf1d0a09d23ebb92a356b892f  tests/test_moe_tail.py