Repository navigation

[Draft] EXL3 Flash Next on CUDA: TP=2, BF16 paging, cache reuse, graphs - #1113

Closed

BlivionIaG wants to merge 126 commits into

ashhart:mainfrom

BlivionIaG:exl3-on-upstream-wip

Closed

[Draft] EXL3 Flash Next on CUDA: TP=2, BF16 paging, cache reuse, graphs#1113
BlivionIaG wants to merge 126 commits into
ashhart:mainfrom
BlivionIaG:exl3-on-upstream-wip

Commits on Oct 18, 2026
- feat: add EXL3 Flash Next execution and expert offload
  BlivionIaG
  committed
  2f648cf
Commits on Oct 19, 2026
- feat: add TP=2 expert-parallel serving and MTP
  BlivionIaG
  committed
  f2500d8
- feat: add deep prefill and long-context state placement
  BlivionIaG
  committed
  59a63c6
- feat: add BF16 KV paging and TP=2 graph capture
  BlivionIaG
  committed
  4304cb1
Commits on Oct 20, 2026
- perf: harden TP=2 graph replay and prefill overlap
  BlivionIaG
  committed
  7b960f2
- merge: integrate current upstream main for EXL3
  BlivionIaG
  committed
  09d7da9
- docs: keep current EXL3 paging, graph, and validation notes
  BlivionIaG
  committed
  b244063
- feat: add EXL3 Flash Next execution and expert offload
  BlivionIaG
  committed
  b8a60fd
- feat: add TP=2 expert-parallel serving and MTP
  BlivionIaG
  committed
  d9b4c9f
- feat: add deep prefill and long-context state placement
  BlivionIaG
  committed
  1bf973b
- feat: add BF16 KV paging and TP=2 graph capture
  BlivionIaG
  committed
  7f03189
- perf: harden TP=2 graph replay and prefill overlap
  BlivionIaG
  committed
  af3bb9c
- docs: keep current EXL3 paging, graph, and validation notes
  BlivionIaG
  committed
  b430974
- fix: retain validated merge resolutions after upstream rebase
  BlivionIaG
  committed
  656bcf9
- fix: dispatch EXL3 through retained CUDA block entrypoints
  BlivionIaG
  committed
  418ecdd
- fix: cover retained EXL3 dispatch with a host regression
  BlivionIaG
  committed
  620a886
- update: integrate latest-main rebase into the EXL3 draft
  BlivionIaG
  committed
  e0ca28b
- perf: omit unused classic MoE scratch for EXL3 engines
  BlivionIaG
  committed
  adefc26
- update: include EXL3 scratch efficiency in the draft
  BlivionIaG
  committed
  a07a4b3
Commits on Oct 21, 2026
Commits on Oct 22, 2026
- fix: open EXL3 profile directories with chmod-capable handles
  BlivionIaG
  committed
  babc976
- docs: record native TP2 expert-profile restart qualification
  BlivionIaG
  committed
  b46e273
- refactor: name the profile warm-start pool parameter explicitly
  BlivionIaG
  committed
  2016e6c
- perf: overlap prefill staging and coalesce release fences
  BlivionIaG
  committed
  ce10271
- feat: attribute expert transfers to demand and prefetch origins
  BlivionIaG
  committed
  090fabf
- perf: coalesce proven contiguous expert DMA ranges
  BlivionIaG
  committed
  b5b385d
- fix: bind prefill graphs to their actual GDN layer buffers
  BlivionIaG
  committed
  a25fa22
- docs: clarify transfer provenance and the prefill default
  BlivionIaG
  committed
  abc040f
- perf: size speculative prefetch from matching target-layer history
  BlivionIaG
  committed
  73d6065
- perf: integrate qualified target-layer prefetch history
  BlivionIaG
  committed
  0272aaf
- perf: accelerate Blackwell mul1 prompt fragment loads
  BlivionIaG
  committed
  5d5104b
- test: check nonfinite EXL3 prompt fragment classes
  BlivionIaG
  committed
  5a94359
- docs: record TP2 prefill overlap profile
  BlivionIaG
  committed
  dfdf27c
- perf: consume TP2 prefill reductions directly in writeback
  BlivionIaG
  committed
  ac22ec0
- docs: record reduced TP2 prefill writeback result
  BlivionIaG
  committed
  cb77e30
- docs: record rejected dense prompt tile experiment
  BlivionIaG
  committed
  c095205
- docs: record larger TP2 prefill chunk screen
  BlivionIaG
  committed
  d16c1d0
- docs: record TP2 chunk ABBA and AVX2 feasibility
  BlivionIaG
  committed
  84d202b
- docs: record TP2 decode critical path
  BlivionIaG
  committed
  73a27bd
- docs: record PCIe and no-prefetch measurements
  BlivionIaG
  committed
  be136f5
- docs: confirm prefetch result on both cards
  BlivionIaG
  committed
  6ce99f4
- docs: add DRAM counters for decode kernel
  BlivionIaG
  committed
  689c55a
- docs: record TP2 no-prefetch serving result
  BlivionIaG
  committed
  a258160
- perf: increase EXL3 prompt chunk size
  BlivionIaG
  committed
  8642d88
- docs: record TP2 decode profile and cumulative gains
  BlivionIaG
  committed
  9d6c0b5
- docs: record TP2 prefill optimization screens
  BlivionIaG
  committed
  a3246ef
- docs: record persisted expert-placement A/B
  BlivionIaG
  committed
  c5c53ef
- docs: record TP2 collective grouping regression
  BlivionIaG
  committed
  ab7ac3f
- docs: record matched TP2 prefill gain over 5a baseline
  BlivionIaG
  committed
  763e662
- docs: record matched TP2 graph decode comparison
  BlivionIaG
  committed
  54ee8b8
- docs: record TP2 paged decode miss profile
  BlivionIaG
  committed
  3c83cb1
- docs: record fragment kernel serving abba
  BlivionIaG
  committed
  54f9799
- docs: extend fragment abba to 512 tokens
  BlivionIaG
  committed
  6b042b2
Commits on Oct 23, 2026
- fix: preserve EXL3 dispatch in batched decode
  BlivionIaG
  committed
  df4ab21
- docs: record TP2 prefill overlap ABBA results
  BlivionIaG
  committed
  6604d23
- docs: record prefill expert lookahead measurements
  BlivionIaG
  committed
  c9fe688
- docs: record reduced TP2 prefill writeback result
  BlivionIaG
  committed
  ab030af
- docs: record shared paging graph budget failure
  BlivionIaG
  committed
  0be29f9
- fix: size EXL3 memory budgets for server concurrency
  BlivionIaG
  committed
  c2b669f
- perf: tune TP2 prefill expert lookahead
  BlivionIaG
  committed
  d073b22
- docs: record score-grid graph ABBA result
  BlivionIaG
  committed
  f9c38cd
- docs: record MTP verify-best ABBA result
  BlivionIaG
  committed
  0031f54
- docs: record clean TP2 graph decode ABBA
  BlivionIaG
  committed
  4bff271
- docs: record expert-copy overlap and cache miss results
  BlivionIaG
  committed
  c94226e
- docs: record concurrent long-context BF16 paging proof
  BlivionIaG
  committed
  09bf5e7
- docs: record matched expert profile miss A/B
  BlivionIaG
  committed
  f61e68b
Commits on Oct 24, 2026
Commits on Oct 25, 2026

Commits

Commits on Oct 18, 2026

Commits on Oct 19, 2026

Commits on Oct 20, 2026

Commits on Oct 21, 2026

Commits on Oct 22, 2026

Commits on Oct 23, 2026

Commits on Oct 24, 2026

Commits on Oct 25, 2026