Repository navigation

[Draft] kv paged kernels - #93

Draft

i-chaochen wants to merge 21 commits into

mainfrom

chao/kv_paged_kernels

Draft

[Draft] kv paged kernels#93
i-chaochen wants to merge 21 commits into
mainfrom
chao/kv_paged_kernels

Commits on Aug 20, 2026
- feat(inference): add the neutral vocabulary for a paged KV runtime
  
  i-chaochen
  and
  cursoragent
  committed
  1ba1ad4
- feat(inference): add a vendor-neutral gpu_paged attention path (M3)
  i-chaochen
  committed
  971db32
Commits on Aug 26, 2026
Commits on Aug 28, 2026
- perf(inference): scrub the whole pool in one dispatch, and stop the harness livelocking
  
  i-chaochen
  and
  cursoragent
  committed
  9edff85
- feat(inference): support the replicated KV regime, and size it correctly (M6)
  
  i-chaochen
  and
  cursoragent
  committed
  3dcfcb5
- refactor(inference): one implementation of the per-step order, not two
  
  i-chaochen
  and
  cursoragent
  committed
  0f4d36c
- feat(inference): join the paged driver to MaxEngine through a shared step seam
  
  i-chaochen
  and
  cursoragent
  committed
  8ddda20
- refactor(inference): both harnesses drive the driver, and the second copy of the livelock goes
  
  i-chaochen
  and
  cursoragent
  committed
  385545e
- feat(inference): give OfflineEngine a paged worker, completing the serving wiring
  
  i-chaochen
  and
  cursoragent
  committed
  4ec189c
- feat(inference): tokenise without JetStream or torch, so the native paged image needs neither
  
  i-chaochen
  and
  cursoragent
  committed
  a566f15
- Decouple the vLLM model adapter from tpu_inference
  
  i-chaochen
  and
  cursoragent
  committed
  f328d22
Commits on Aug 29, 2026
- Let the vLLM adapter and gpu_paged serve a non-TPU platform
  
  i-chaochen
  and
  cursoragent
  committed
  282428a
Commits on Sep 11, 2026
- M7: page staging to host, and the refusal that makes it safe
  
  i-chaochen
  and
  cursoragent
  committed
  d7ebb85
- M7: the host tier, and the capacity it actually buys
  
  i-chaochen
  and
  cursoragent
  committed
  30105a4