Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Each release's page on GitHub has its notes and measurements. See [the 1.0.0 rel
- The EXL3 routed experts' SwiGLU clamps keep a NaN, as torch.clamp does, where they turned it into the limit; on the decode path a limit of 0 now means none, as it already did for prompts.
- GLM-5.3-Flash on two CUDA ranks reads Mia-AiLab's EXL3 checkpoint, its 4-bit mcg routed experts on the common EXL3 module, and with `--vision` takes images and video: rank 0's tower encodes them (video needs libtfvideo) and sends their rows and spans to the other rank. The ranks watch each other, so a rank that fails stops the other instead of leaving it waiting, and they agree on the checkpoint's bytes and every media limit before either loads; verify windows reach the decode buffers' 8 rows, and the bf16 linears run on glm_mm's tiles (#855, #976). Thanks to @BobClawblaw.
- Qwen3.6-35B-A3B serves on the Qwen CUDA engine through the native server: its routed experts in 4 or 3 bits behind a softmax router (the 3-bit planes run beside a 4-bit shared expert), the MTP layer beside the checkpoint drafting chains and trees, and the dense family's prompt cache and image tower. On a GB10 drafted replies equal plain ones, 17 of 17, and concurrent replies their runs alone (#856, #972). Ours: its router, top-k and combine launches and its MTP layer's are in the CUDA kernel contract, each against float64, with records for nvcc 13.0 and 13.2. Thanks to @BobClawblaw and @CanadaApollo6.
- A backend can ask the lane core to send a stream's first token before it prepares the first drafts, on a whole prompt pass or its last chunk; the Qwen CUDA backend does. The receipts are in `docs/receipts/qwen-cuda-first-token/` (#932). Thanks to @jregan-beasley-smc.
- A backend can ask the lane core to send a stream's first token before it prepares the first drafts, on a whole prompt pass or its last chunk; the Qwen CUDA backend does. The receipts are in `docs/receipts/qwen-cuda-first-token/` (#932). Thanks to @jrb-ai.
- Nemotron 3.5 Bolt (74B-A9B) serves on CUDA from NVIDIA's 24 GB checkpoint: puzzle layers, LatentMoE with its experts in GGML's IQ2_XXS and Q8_0 blocks on kernels of ours, image input, prompt states in pinned host memory on a card that cannot hold the prompt cache (the startup line says where they went), shared rounds of streams, and structured output and forced tool calls (#897, #938). On an RTX PRO 6000 (sm_120, nvcc 13.2) its drafted and plain replies equal those of #897's own build, concurrent replies equal their runs alone, and decode runs at 153 to 155 tok/s against that build's 144 to 146, with the same tokens. Ours: a failed load keeps its destinations and GPU parents until recovery proves them unused, the released router top-k kernel stays as it was beside a wide one for Bolt's 496 experts, and a negative `moe_latent_size` is refused before it becomes a width. Thanks to @BobClawblaw and @MiaAI-Lab.
- Nemotron Omni takes video through the RADIO tower: frames sampled into groups the tower reads as images, each with its time, and video tokens budgeted apart from image tokens; the reserved video token in user text stays text (#940). Ours: the video config, frame rate, indices, dimensions and grid are refused before any decode or cast. Thanks to @MiaAI-Lab and @cwschroeder.
- Gemma 4 serves natively on Metal: the 31B with its own dense and fused kernels on the tensor units, prompts in 512-row chunks, DFlash drafts, images and structured output (#879), and the 12B (`gemma4_unified`) with its unified text and image embedder and its MTP assistant as a native drafter (#975). Gemma's arithmetic runs in its own kernel copies, so no other family's bits move, and its kernels are in the Metal kernel contract (on M5, 12 of 12 at two seeds). Ours: `config.json`'s context and EOS ids are bounded before their casts, and the packed buffers and private pipelines keep one owner through every failure. Thanks to @BobClawblaw.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,7 @@ Thank you to everyone who has sent TensorFold a pull request, a measurement or a
[@johnymoo](https://github.com/johnymoo),
[@JordiPosthumus](https://github.com/JordiPosthumus),
[@JRaxworthy](https://github.com/JRaxworthy),
[@jregan-beasley-smc](https://github.com/jregan-beasley-smc),
[@jrb-ai](https://github.com/jrb-ai),
[@jschmied](https://github.com/jschmied),
[@juliankang4](https://github.com/juliankang4),
[@kelchm](https://github.com/kelchm),
Expand Down
2 changes: 1 addition & 1 deletion RELEASE-NOTES-1.0.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ Thank you to all 145 public contributors who have sent TensorFold a pull request
[@johnymoo](https://github.com/johnymoo),
[@JordiPosthumus](https://github.com/JordiPosthumus),
[@JRaxworthy](https://github.com/JRaxworthy),
[@jregan-beasley-smc](https://github.com/jregan-beasley-smc),
[@jrb-ai](https://github.com/jrb-ai),
[@jschmied](https://github.com/jschmied),
[@juliankang4](https://github.com/juliankang4),
[@kingjamez](https://github.com/kingjamez),
Expand Down