Qwen3.8-27B CUDA: with TF_QWEN_KV_PAGES a batch takes the partial-only attention path, so its replies differ from the same replies alone (1.1.5, RTX 5090) #1344

Summary

Observed (the recipe's together check: 12 greedy chats, 300 tokens, each alone then four at a time; TensorFold 1.1.5 84e0d56,
Qwen3.8-27B MLX 4-bit + DFlash2, one RTX 5090, sm_120, CUDA 13.3; evidence in
https://github.com/BobClawblaw/qwen38-27b-vision-rtx-5090/tree/main/evidence/2026-10-25-engine-1.1.5, the
harness-together-16384x4-{whole,paged} and harness-streams-* folders):

Drafted == undrafted 12/12 in every configuration. The differing replies part at near-tied tokens and stay fluent.

Mechanism (from the code, zig/src/families/qwen3_5/cuda_forward.zig, attention layers):

if (shape.paged) {                       // any segment of the batch on pages
    partial(e, shape, ai);               // tf_q35_attn_partial_paged (or _contiguous), then attn_merge
} else if (prompt ...) { cuda_prompt.run } else {
    attn_partial + attn_rows (tensor cores, #1168) , then attn_merge
}

shape.paged is set when any segment in the forward has a page table (cuda_attention.hasPages), so a batch that
holds one paged stream computes attention for every row in it, the contiguous first stream's rows included, with the
partial-only kernels, while the same stream alone (a flat batch) takes the tensor-core attn_rows path. The two
paths accumulate in different orders, so their bf16 outputs differ in the last bits, and a greedy reply parts at the
first near-tie. #1170's note "flat batches keeping the contiguous tensor-core attention" describes exactly this split.
So this is not a page-table fault; the paged batches run a different attention arithmetic from flat ones.

Repro (lone stream, no batching): whole (1.1.5 as served) vs paged (TF_QWEN_KV_PAGES=256 TF_QWEN_KV_PAGES_ALL=1, one stream on pages) vs forced (a check build that takes the partial-only path for a flat
forward: TF_Q35_FORCE_PARTIAL=1). Results (32k window, fp8 caches, int8 prompts):

• forced vs whole: 7/12 equal — the partial-only path alone, on a flat lone stream, changes 5 of 12 greedy replies,
  the same order as the paged batches (4-6 of 12 changed). The five replies part from whole at characters 69, 345, 297,
  184 and 12; in the 65536x8 paged batch the replies parted from their alone runs at 69, 345, 297, 297, 219 and 12: four
  of the five divergence points coincide to the character.
• forced vs the earlier paged batches' replies (fp8 runs): 9/12, 10/12, 10/12, 6/12 (262144x2) equal, against whole's
  8, 6, 7, 6: the forced flat path lands closer to the paged batches than the served path does; the remainder is the
  paged partial kernel against the contiguous one, which upstream's check holds to a float64 bound, not bit-equality.
• the paged arm with one stream was not paged: TF_QWEN_KV_PAGES_ALL=1 needs streams beyond the first (plan line
  "0 more taking pages"), so it equalled whole 12/12 trivially.
• paged2 (two lanes, every stream on 256-row pages, one request at a time: a lone stream whose cache is paged):
  vs whole 7/12, parting at characters 69, 345, 297, 184 and 12; vs forced 12/12 identical. A lone paged stream
  gives exactly the replies of a flat stream forced onto the partial-only attention path.
• upstream's own tf-cuda-test qwen-pages on this card: every case PASS (paged == contiguous for queries, rows, stores
  and keep within its float64 bound, five repeats, guards), controls RED. The page table is sound; the divergence is the
  kernel choice above.

What would fix it: a paged variant of the tensor-core decode attention (tf_q35_attn_rows / _rows8 reading key
tiles through the page table), so paged batches keep #1168's arithmetic; or, cheaper, routing only the paged segments
through partial and the contiguous ones through attn_rows within one batch (keeps the first stream exact beside
paged ones; the paged streams still differ from their own flat runs).

Conclusion. The page table is exact (upstream's qwen-pages passes here; a lone paged stream equals the forced flat run
bit for bit on 12/12 replies). What changes the replies is that any forward holding a paged segment runs its attention
through the partial-only kernels (tf_q35_attn_partial_{paged,contiguous} + merge) instead of the tensor-core
tf_q35_attn_rows pass that flat decode takes, for every stream in the batch. The two accumulate in different orders;
greedy replies part at the first near-tie (5 of 12 of the suite's chats, at the same characters in every configuration).
It is a documented design split (#1170: "flat batches keeping the contiguous tensor-core attention"), but it breaks the
exactness rule "every stream equals its run alone" as soon as pages are on, and it also changes the contiguous first
stream's replies when a paged stream sits beside it. A paged attn_rows (key tiles through the page table) would close it.