Repository navigation
Conversation
… Qwen3.5-0.8B projections, m in {1,2,4,8,16} (Refs #5303, #5312)
Q4_K, Q5_K, Q6_K and Q8_0 all measured bitwise equal on an RTX 4090 (lambda);
no qtype stays on the per-row loop for this file. The test-only
CudaExecutor::invalidate_q8_activation lets a host rewrite of an input clear the
Q8_1 activation cache.
Agent: aprender-w5404
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
§13.11 rung 1 — quorum shadow verdict Shadow mode: this records a verdict and merges nothing. A refusal |
|
seat 1, w5382 @be5293fd23 — test-only (+ a #[cfg(test)] cache-invalidate helper). Reviewed: bitwise compare of batched vs single-row GEMV across every distinct (qtype,n,k) and m∈{1,2,4,8,16}; q8 activation cache invalidated before each call, so the comparison cannot pass by reusing a stale quantization; SKIPs when the GGUF or GPU is absent. No commits by me in this PR. |
noahgift
left a comment
There was a problem hiding this comment.
seat 1, triage-worker @be5293fd23bcc8eab8089467832cfc6e51b5a5bc (non-author; head re-read; 0 triage-worker trailers)
Read the full diff (3 files, +171/-0), all #[cfg(test)]: a invalidate_q8_activation test helper and one new test comparing batched_gemv_dispatch rows bitwise against gemv_dispatch for m in {1,2,4,8,16} over every distinct (qtype,n,k) in Qwen3.5-0.8B. Finding: no defect in the diff.
- The test names each unequal qtype with its measured difference (no mean, no tolerance), and it also resets the Q8 activation cache between calls, which is the right keying hazard to guard.
- Caveat, not a defect: the test returns early (SKIP) when the GGUF or a GPU is absent, so a green CI run on a host without them proves nothing about the kernels. The evidence is a GPU-host run; the PR should cite one.
- The red: x86-main and gate failed because section
guard-treefailed (run 39116016996; it ran 1067 s and a cargo-free guard,guard-tree#m0, was still running past 600 s). None of the 3 changed files is in guard-tree's scope, so this is infra/guard timing, not this PR. Which guard went red I did not measure (the log shows no FAIL row for a named guard); a re-run of the failed jobs is the next step.
A push voids this seat.
noahgift
left a comment
There was a problem hiding this comment.
seat 2, pace @be5293fd23bcc8eab8089467832cfc6e51b5a5bc APPROVE (the diff), with one caveat on what 'green' means here. Non-author (0 aprender-pace trailers). Read all 3 files: a #[cfg(test)]-only invalidate_q8_activation() on the executor, a test-module registration, and one GPU test comparing batched_gemv_dispatch rows bitwise (to_bits) with gemv_dispatch row by row, m in 1/2/4/8/16, over each distinct (qtype,n,k) of the Qwen3.5-0.8B projections. Good: bitwise compare, not a tolerance; the Q8 activation cache is invalidated before the batched call and before each solo call, so a stale cache cannot make two paths agree by sharing a quantization; the assert fails with the qtype/shape/row/col/diff named. CAVEAT: the test returns early (pass) if /home/noah/models/Qwen3.5-0.8B-Q4_K_M.gguf is absent or no CUDA device (cuda_executor_or_skip). That path is hard-coded to one host, so on a CI runner without the file or a GPU the check is green while testing nothing; green CI here is not evidence the kernels agree. It only counts when it runs on the GPU host, so the claim in #5303 needs a run log from there. Nit: the doc on distinct_weights says 'every FFN gate/up/down and lm_head' but it also pushes the attention and DeltaNet projections. No bank receipt for this PR is in its comments, so I post no C314 note. I do not arm.
|
Superseded by #5732 (fold). Closed at operator instruction 2026-10-19. |
Adds
batched_gemv_rows_equal_the_single_row_kernel: for every distinct (qtype, n, k) among the Qwen3.5-0.8B-Q4_K_M projections (attn q/k/v/output, deltanet qkv/alpha/beta/gate/out, FFN gate/up/down, lm_head) and m in {1,2,4,8,16}, each row ofbatched_gemv_dispatchis compared bit for bit withgemv_dispatchon that row alone.Witness (lambda-vector, RTX 4090, idle before the run,
setsid timeout -s KILL):cargo test -p aprender-serve --features cuda --lib batched_gemv_rows_equal-> 1 passed. Qtypes in the file: Q4_K, Q5_K, Q6_K, Q8_0; every row bitwise equal at every m (no qtype needs to stay on the per-row loop). Log: /mnt/nvme-raid0/aprender-w5404-target/gemv.log.The only non-test change is a
#[cfg(test)]CudaExecutor::invalidate_q8_activation. Skips with a message when the GGUF or CUDA is absent.Refs #5303, #5312 (not Closes; keep-open).
🤖 Generated with Claude Code
keep-open: #5312 stays open after this merges. It is not closed by this PR; the operator's batch call decides it.