Skip to content

test(serve): batched GEMV rows bitwise equal the single-row kernel, Qwen3.5-0.8B, m 1/2/4/8/16 (Refs #5303) - #5497

Closed
noahgift wants to merge 1 commit into
mainfrom
fix/5303-batched-gemv-row-equal
Closed

noahgift wants to merge 1 commit into
mainfrom
fix/5303-batched-gemv-row-equal

Conversation

@noahgift

@noahgift noahgift commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Adds batched_gemv_rows_equal_the_single_row_kernel: for every distinct (qtype, n, k) among the Qwen3.5-0.8B-Q4_K_M projections (attn q/k/v/output, deltanet qkv/alpha/beta/gate/out, FFN gate/up/down, lm_head) and m in {1,2,4,8,16}, each row of batched_gemv_dispatch is compared bit for bit with gemv_dispatch on that row alone.

Witness (lambda-vector, RTX 4090, idle before the run, setsid timeout -s KILL): cargo test -p aprender-serve --features cuda --lib batched_gemv_rows_equal -> 1 passed. Qtypes in the file: Q4_K, Q5_K, Q6_K, Q8_0; every row bitwise equal at every m (no qtype needs to stay on the per-row loop). Log: /mnt/nvme-raid0/aprender-w5404-target/gemv.log.

The only non-test change is a #[cfg(test)] CudaExecutor::invalidate_q8_activation. Skips with a message when the GGUF or CUDA is absent.

Refs #5303, #5312 (not Closes; keep-open).

🤖 Generated with Claude Code

keep-open: #5312 stays open after this merges. It is not closed by this PR; the operator's batch call decides it.

… Qwen3.5-0.8B projections, m in {1,2,4,8,16} (Refs #5303, #5312)

Q4_K, Q5_K, Q6_K and Q8_0 all measured bitwise equal on an RTX 4090 (lambda);
no qtype stays on the per-row loop for this file. The test-only
CudaExecutor::invalidate_q8_activation lets a host rewrite of an input clear the
Q8_1 activation cache.

Agent: aprender-w5404
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=5497 head=be5293fd23bcc8eab8089467832cfc6e51b5a5bc verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

@noahgift

Copy link
Copy Markdown
Contributor Author

seat 1, w5382 @be5293fd23 — test-only (+ a #[cfg(test)] cache-invalidate helper). Reviewed: bitwise compare of batched vs single-row GEMV across every distinct (qtype,n,k) and m∈{1,2,4,8,16}; q8 activation cache invalidated before each call, so the comparison cannot pass by reusing a stale quantization; SKIPs when the GGUF or GPU is absent. No commits by me in this PR.

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat 1, triage-worker @be5293fd23bcc8eab8089467832cfc6e51b5a5bc (non-author; head re-read; 0 triage-worker trailers)

Read the full diff (3 files, +171/-0), all #[cfg(test)]: a invalidate_q8_activation test helper and one new test comparing batched_gemv_dispatch rows bitwise against gemv_dispatch for m in {1,2,4,8,16} over every distinct (qtype,n,k) in Qwen3.5-0.8B. Finding: no defect in the diff.

  • The test names each unequal qtype with its measured difference (no mean, no tolerance), and it also resets the Q8 activation cache between calls, which is the right keying hazard to guard.
  • Caveat, not a defect: the test returns early (SKIP) when the GGUF or a GPU is absent, so a green CI run on a host without them proves nothing about the kernels. The evidence is a GPU-host run; the PR should cite one.
  • The red: x86-main and gate failed because section guard-tree failed (run 39116016996; it ran 1067 s and a cargo-free guard, guard-tree#m0, was still running past 600 s). None of the 3 changed files is in guard-tree's scope, so this is infra/guard timing, not this PR. Which guard went red I did not measure (the log shows no FAIL row for a named guard); a re-run of the failed jobs is the next step.
    A push voids this seat.

@noahgift noahgift left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat 2, pace @be5293fd23bcc8eab8089467832cfc6e51b5a5bc APPROVE (the diff), with one caveat on what 'green' means here. Non-author (0 aprender-pace trailers). Read all 3 files: a #[cfg(test)]-only invalidate_q8_activation() on the executor, a test-module registration, and one GPU test comparing batched_gemv_dispatch rows bitwise (to_bits) with gemv_dispatch row by row, m in 1/2/4/8/16, over each distinct (qtype,n,k) of the Qwen3.5-0.8B projections. Good: bitwise compare, not a tolerance; the Q8 activation cache is invalidated before the batched call and before each solo call, so a stale cache cannot make two paths agree by sharing a quantization; the assert fails with the qtype/shape/row/col/diff named. CAVEAT: the test returns early (pass) if /home/noah/models/Qwen3.5-0.8B-Q4_K_M.gguf is absent or no CUDA device (cuda_executor_or_skip). That path is hard-coded to one host, so on a CI runner without the file or a GPU the check is green while testing nothing; green CI here is not evidence the kernels agree. It only counts when it runs on the GPU host, so the claim in #5303 needs a run log from there. Nit: the doc on distinct_weights says 'every FFN gate/up/down and lm_head' but it also pushes the attention and DeltaNet projections. No bank receipt for this PR is in its comments, so I post no C314 note. I do not arm.

@noahgift
noahgift added this pull request to the merge queue Oct 18, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 18, 2026
@noahgift

Copy link
Copy Markdown
Contributor Author

Superseded by #5732 (fold). Closed at operator instruction 2026-10-19.

@noahgift noahgift closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant