Skip to content

PMAT-3477: Qwen3.5 (Gated DeltaNet hybrid) runs on CUDA — nine device kernels, Qwen35CudaModel, refusal sites lifted, apr run/chat/qa route to the GPU (#3090) - #3517

Merged
noahgift merged 12 commits into
mainfrom
PMAT-3477-gdn-gpu
Sep 19, 2026

Conversation

@noahgift

@noahgift noahgift commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

Closes #3090.

Operator decision 2026-09-19 (in-session, verbatim selection): "GPU required — build #3090 now". This PR is the GPU half of the 0.68.2 interrupt (EPIC #3477); the CPU half landed in #3484.

What lands

commit what falsifier
5ebc79d six Gated-DeltaNet PTX kernels in crates/aprender-gpu/src/kernels/gdn/ (causal conv1d+SiLU, per-head L2 norm, dt/β gates, delta-rule recurrence, gated RMSNorm, sigmoid gate) device tests vs a verbatim host port of the CPU fns at real 0.8B shapes; delta-rule mutant (scale×1.01) RED at relative L∞
7591ff1 head_dim-256 decode attention (positions-per-pass cap, multi-pass tested), partial NEOX RoPE (θ-scale on host — device pow measured 2.3e-4 > 1.4e-4 bar), [q|gate] split 12 device tests; stubbed kernels RED
58870fd, 6474c45, 1ed0adf executor wrappers (graph-recording push on every launch) + Qwen35CudaModel: DeltaNet and attention layers op-for-op, forward_single, config-sized device state, download_layer every DeltaNet/attention layer teacher-forced vs CPU; e2e 6-token: argmax identical at every position, cosine ≥ 0.996 (measured 0.9983–0.9996), per-op ≤ 1e-3 vs an exact-arithmetic reference; sigmoid-gate mutant RED (cos 0.766); DP4A falsifier RED (cos 0.466…−0.25)
d9459c7 refusal sites lifted in lockstep: capability gate admits qwen35 on GPU under cuda, parity/ptx-map stop refusing it, quant whitelist judges the SSM tensors, SSM predicate names the hybrid path 28 unit tests across apr-cli/serve
b2bd0c5 apr run/chat/qa route qwen35 to the GPU; F2 guard judges the hybrid against its own CPU forward (Serial probe); no silent CPU fallback; stale "GPU declined (#3090)" skip reasons removed apr run --gpu: CUDA backend line, "Paris"; apr qa: ALL GATES PASSED, Golden Output + Throughput on the GPU

Two measured facts that shaped it

Measured on lambda-vector (RTX 4090), pinned binary

Backend: GPU (CUDA, NVIDIA GeForce RTX 4090, 24035 MB VRAM) [qwen35 hybrid forward, #3090]
Output: The capital of France is **Paris**.
apr qa: Capability Match PASS · Golden Output PASS (GPU) · Throughput 50.5 tok/s PASS (GPU) · PTX Parity 6/6 · ALL GATES PASSED

gx10 (GB10) + ladder receipts: peer session runs both hosts at b2bd0c5; receipts land with the bump PR (#3498).

Process

Design grill 3/3 (refute-as-written → all four blocker classes applied: head_dim 256, partial rope, F2 reference, lockstep refusal sites). Pre-merge review quorum: in flight (docs/audits/quorum-PMAT-3477-pr.json to follow). Workers: G, H, I, J, J2, J3, K1, K2, K3 — every acceptance re-run by the orchestrator.

ont-delta: none this PR adds device kernels and a GPU forward; no ontology type, shape, reason or resolution changed

🤖 Generated with Claude Code

Issue refs (§6 R-2)

keep-open: #3477 — the release epic closes at the tag, not at this PR.
keep-open: #3513 — DP4A-through-recurrence root cause (0.69.0), mitigated here by pinning the float GEMV.
keep-open: #3346 #3510 — dense sizes 4B/9B/27B: the GQA-in-GDN CPU fix rides this branch; the CUDA half and the size rungs follow.

Two-host ladder at b2bd0c5 (peer session, main contract overlaid)

noahgift and others added 7 commits September 19, 2026 06:58
…orm, gates, delta-rule recurrence, gated RMSNorm, sigmoid gate — each against a host port of the CPU reference (#3090)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e split kernels for the Qwen3.5 attention layers (#3090)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s and the DeltaNet layer forward on the GPU, every DeltaNet layer matching the CPU forward on the real 0.8B file (#3090)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… forward — argmax matches the CPU at every position on the real 0.8B file (#3090)

The three attention-side kernels are wired through the executor (split q|gate, partial NEOX rope with a HOST-computed theta_scale, head_dim-256 decode attention), the attention layers upload their tensors and own device K/V caches, and forward_single runs a whole token through both layer kinds, the output norm and the lm_head.

Op-by-op against the CPU on the GPU's own inputs: every attention stage is inside 1e-3 relative. Skipping gdn_sigmoid_gate_into turns that test RED at 1.047e0, so the gate is load-bearing.

End to end over six tokens the argmax is IDENTICAL at every position (13, 198, 11, 198, 0, 908) but the logit L-inf is 2.0e-2..6.7e-2, not the 1e-3 the test asks for. The new parity-floor test says why: on attn_q the GPU float GEMV is 9.96e-8 from an exact dequantized f64 dot while the CPU fused_matmul_into is 2.95e-3 from it, because it quantizes the ACTIVATION to Q8_K. The reference is the noisier side, so no correct GPU implementation can sit inside 1e-3 of it. Two tests are left RED rather than have their tolerances moved; what the bar should be is a decision, not a fix.

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ate, parity, ptx-map, quant whitelist and the SSM predicate now agree with the hybrid forward (#3090)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…through the recurrence); parity restated to per-op exact ≤1e-3 plus e2e argmax equality with a measured cosine floor (#3090)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ard judges the hybrid against its own CPU forward; no silent CPU fallback (#3090)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@noahgift noahgift added this to the 0.68.2 milestone Sep 19, 2026
@github-actions

github-actions Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=3517 head=0f5aaab225105afa3aff64d977d18e91c7d84473 verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

… stream syncs, LM head on device, one logits download; floor test no longer re-pins (#3090 review)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
noahgift and others added 4 commits September 19, 2026 11:18
…recurrence) on the CPU forward — Qwen3.5 4B/9B/27B load and run (#3346, #3510)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…lt from main with the nine GDN variants appended, speedup.rs leading blank restored; review quorum artifact for #3517

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… in the recurrence) on the CPU forward — Qwen3.5 4B/9B/27B load and run (#3346, #3510)"

This reverts commit 1a97abb.
guyernest pushed a commit to guyernest/aprender that referenced this pull request Sep 29, 2026
… hosts — eight rungs with sha256, RED by construction on Qwen3.5 (paiml#3510) (paiml#3523)

* evidence(ladder): 0.68.1 receipts re-measured at main a6f54e8 on both hosts — all eight rungs with sha256, RED by construction on the Qwen3.5 rungs (paiml#3510, ONT-4c1 witness)

Both receipts produced by scripts/model_ladder.sh at a6f54e8 (the first main
with paiml#3507's sha256 rows and paiml#3511's four dense rungs), apr 0.68.1 (a6f54e8),
lambda RTX 4090 cc 8.9 and gx10 GB10 cc 12.1. Every rung present, sha_ok, and
carries the measured sha256 — the witness ONT-4c1's ladder-measured shape needs
on day one. Green: qwen2-1.5b, qwen3-1.7b (both hosts), qwen3-8b (gx10; lambda
optional rung: golden Empty output, paiml#3486). Red by construction: qwen35-0.8b/2b
fall back to CPU on main until paiml#3517 merges; qwen35-4b/9b/27b exit 101 at
forward_qwen35.rs:132 (GQA inside GDN, nk != nv — paiml#3346 root cause). The T-2
judge is not a PR check; this changes no gate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* audit: quorum artifact for PR paiml#3523 (3/3 PASS, PMAT-3524)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
guyernest pushed a commit to guyernest/aprender that referenced this pull request Sep 29, 2026
* release: 0.68.2

bump-version.sh 0.68.2 (every workspace, facades included; --check green) and the CHANGELOG [0.68.2] section,
per docs/specifications/06x-release-schedule.md §4.2. After this merges, rel-0682-autopilot runs APR-RELEASE-001 §4 T-1..T-4 + close:
deep (T-1, local), pre-publish dogfood, tag + release, clean-room.yml dispatched on the tag (T-3, run id recorded), assets by command, preflight, cascade (T-4, automated), install, host + installer receipts, close.

Pmat-Ticket: PMAT-3477
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* release: 0.68.2 CHANGELOG — Qwen3.5 on CUDA up to 27B (paiml#3517, paiml#3527) and every PR merged since v0.68.1

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* evidence(ladder): 0.68.2 receipts measured from the bump tree on lambda (sm_89) and gx10 (sm_121) — every required rung green on both hosts; qwen3-8b optional red on lambda (paiml#3532)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* contracts: arm ladder-measured + ladder-green shapes for 0.68.2; contracts.nt regenerated with the two 0.68.2 ladder receipts as focus nodes (ONT-4c1, paiml#3508)

Pmat-Ticket: PMAT-3477

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* CHANGELOG 0.68.2: rows for paiml#3533, paiml#3534 (merged since the cut)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* CHANGELOG 0.68.2: row for paiml#3528

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* CHANGELOG 0.68.2: rows for paiml#3526, paiml#3530 (merged-since-tag predicate now 0)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* CHANGELOG 0.68.2: row for paiml#3536

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Alfredo Deza <alfredo@deza.pe>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GPU: implement Gated DeltaNet / SSM inference for Qwen3.5 / Qwen3.8 hybrid GGUFs

1 participant