Repository navigation

qwen CUDA: a faster opt-in paged path (TF_QWEN_KV_PAGES=256), --parallel a cap with pages set - #2011

Merged

ashhart merged 16 commits into

ashhart:mainfrom

jschmied:qwen-pages-default

Nov 7, 2026

Merged

qwen CUDA: a faster opt-in paged path (TF_QWEN_KV_PAGES=256), --parallel a cap with pages set#2011
ashhart merged 16 commits into
ashhart:mainfrom
jschmied:qwen-pages-default

Commits on Nov 7, 2026
- cuda prefill_attention: a paged item's page looked up once, not every 16-byte chunk
  
  jschmied
  authored and
  ashhart
  committed
  df70a71
- cuda prefill_attention: hpattn_p256_kernel for pages of 256 positions, with the page size a constant
  
  jschmied
  authored and
  ashhart
  committed
  658083f
- qwen35 attention tiles: a decode fetch looks its page up once per page, not once per key
  
  jschmied
  authored and
  ashhart
  committed
  def0827
- qwen CUDA: streams on pages by default, the engine's own too
  
  jschmied
  authored and
  ashhart
  committed
  d6f3110
- qwen35 attention tiles: a decode thread keeps its last page across tiles, so its first row is looked up once a page, not once a tile
  
  jschmied
  authored and
  ashhart
  committed
  293b3da
- cuda prefill_attention: hpattn_p256_ids_kernel stages a chunk's page ids in shared memory (up to 768 pages), hpattn_p256_kernel past that
  
  jschmied
  authored and
  ashhart
  committed
  345d8e7
- qwen CUDA: a paged prompt-cache save and restore copy one 2D block a page, not one run a page and piece
  
  jschmied
  authored and
  ashhart
  committed
  b4eb3ea
- qwen CUDA: a MoE checkpoint stays on windows by default, pages opt-in; a dense one pages at 256 positions
  
  jschmied
  authored and
  ashhart
  committed
  a232423
- changelog: Qwen pages by default on the dense models, --parallel a cap (#2011)
  
  jschmied
  authored and
  ashhart
  committed
  5077a2a
- qwen CUDA: pages opt-in on a dense model too, TF_QWEN_KV_PAGES_ALL unset off again, both as on main
  
  jschmied
  authored and
  ashhart
  committed
  2321fe9
- qwen CUDA host tests: pages default (unset, 0, 1, a size; dense and MoE), _ALL unset off, --parallel a cap only with pages
  
  jschmied
  authored and
  ashhart
  committed
  964d6e4
- changelog: Qwen pages opt-in, --parallel a cap with TF_QWEN_KV_PAGES set (#2011)
  
  jschmied
  authored and
  ashhart
  committed
  8ce4f1d
- qwen CUDA: the two pages refusals stop before the weights load again, unset_pages a plain 0
  
  jschmied
  authored and
  ashhart
  committed
  cac3da2
- qwen pages contract: the PASS line names the ids kernel's control
  
  jschmied
  authored and
  ashhart
  committed
  9d688df
- changelog: Qwen pages cites (#2011, part of #1943)
  
  jschmied
  authored and
  ashhart
  committed
  3dbd911
- qwen contract records: tf_q35_attn_rows8_mt2 at 243 registers on sm_121, measured with the fp8 tiles' one page lookup
  
  jschmied
  authored and
  ashhart
  committed
  ac33c2e

Commits

Commits on Nov 7, 2026