Repository navigation
wgpu forward hardcodes rope_theta=1e6 (wgsl_forward.rs:970): Llama/Mistral wrong on wgpu (R12-1) #4832
Closed as not planned
Description
Parent: #3999 (E6 0.73 epic). Ref: la-0.73 R12-1.
Defect
crates/aprender-compute/src/backends/gpu/device/linalg/wgsl_forward.rs:970 (main @ 67e7518):
let rope_theta = 1_000_000.0f64; // Qwen2 rope_thetaThe wgpu forward path hardcodes RoPE base 1e6 (Qwen2) for both the Q (:977) and K (:993) rotations. Models with another base, e.g. Llama (10 000 / 500 000) and Mistral (10 000 / 1e6 per version), get the wrong rotary angles on wgpu, and the output is wrong. No error is raised.
Fix
Read rope_theta from the model config, as the CPU/CUDA paths do, and pass it into the wgsl forward. If it is missing, fail loudly instead of falling back to 1e6.
Acceptance
- No literal rope base remains in the wgsl forward path.
- A unit test drives the wgpu forward with two configs whose rope_theta differ and asserts the rotations differ, so the test fails if the constant comes back.
- A parity row (Llama or Mistral, wgpu vs CPU) reports argmax/logit agreement.
Owner: aprender-cd.
Activity
docs(lookahead-0.73): row 10 leg A wgpu arm draft
docs(lookahead-0.73): row 6 Qwen3.5 on wgpu draft
noahgift commentedon Oct 13, 2026
park proposal: parked, defect confirmed still present on main @7f9a15a52b (
wgsl_forward.rs:970still readslet rope_theta = 1_000_000.0f64); it is scheduled under the 0.73 epic #3999 with an owner, so it waits there.noahgift commentedon Oct 14, 2026
merged into #5313 (master ticket for epic #3999; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.
noahgift commentedon Oct 14, 2026
Moved: this sub-ticket is now a row in #5338 (Backend portability: correct kernels and config on every CPU, wgpu and CUDA target), split out of #5313 by theme.