Repository navigation
dsv41 Zig: dense bf16 linears of a mixed EXL3 pack (EAQ v1.1 indexer path) - #44
Conversation
A mixed EXL3 pack stores some whole linears as a plain bf16 <prefix>.weight [n, k] instead of trellis parts (Mia-AiLab EAQ v1.1: wq_a, the indexer wq_b and wk, and the compressor wkv and wgate of the indexer layers). - plan: wantsDense plans a dense linear as one bf16 .weight entry; a split (TP column or row part) dense linear is refused. The rank cache builder takes the dense set from the shards, the checker from the cache. - weights: Linear.dense; linear() loads a dense entry when the source has no trellis for the prefix. - prompt / round: dense linears run through cuBLAS GemmEx (bf16 operands, COMPUTE_32F, bf16 or fp32 out) in grouped(), mm and the decode q_proj (the indexer wq_b from q's normed rows); no counters, rotated rows or partials.
…very pull request) zig fmt 0.17.0 changes only layout and @intFromEnum to @backingInt. The Zig format check fails on these five base-branch files in every open dsv41-zig pull request, also where the pull request does not touch them.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1104203a46
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1104203a46
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…run path for it yet) Round.init copies each wo_a slice's suh, and the wo_a decode runs groupedRotated, which refuses dense layers. A mixed pack with a dense wo_a slice thus planned and loaded, then failed at init. plan.denseRunnable now refuses it in the planner (error.DenseLinearUnsupported) and the loader checks it too. New test case in the dense plan test.
|
@codex review |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
What this changes
Two commits that let the Zig dsv41 engine load a mixed EXL3 pack whose some whole linears are plain bf16
<prefix>.weight[n, k] tensors, not trellis parts (Mia-AiLab EAQ v1.1:wq_a, the indexerwq_bandwk, and the compressorwkvandwgateof the indexer layers).29f91f69:plan.wantsDenseplans a dense linear as one bf16.weightentry (a TP-split dense linear is refused); the rank cache builder takes the dense set from the shards, the checker from the cache.weights.Linear.dense:linear()loads a dense entry when the source has no trellis for the prefix. Dense linears run through cuBLAS GemmEx (bf16 operands, COMPUTE_32F, bf16 or fp32 out) ingrouped(),mmand the decodeq_proj.64bb1e48: the dense plan test's context is a byte, not a type (test fix).Serving target E serves from this branch (image
tensorfold-dsv41-zig:dense-64bb1e48, top commit64bb1e48). These two commits are not rebased or changed here: the running image is built from them.Base is
dsv41-zig-devso that #42 (based on this branch) can land after it.Receipt
dsv41-zig-dev@cc9be34eplus these 2 commits. Zig 0.17.0. Two NVIDIA GB10 nodes at TP=2 (serving target E).Checklist
"draft": false: not checked in this pull request.This change was written with help from an AI coding assistant.