Skip to content

CUDA: #1928's load read-ahead on Nemotron, Bolt, MiniMax M3, Kolibri-1, Spark 2.5 and DeepSeek-V4.1, opt-in (#2003) - #2033

Merged
ashhart merged 3 commits into
mainfrom
lead/2003-load-prefetch-port
Nov 5, 2026
Merged

ashhart merged 3 commits into
mainfrom
lead/2003-load-prefetch-port

Conversation

@ashhart

@ashhart ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Part of #2003. Stacked on #1928 (@jschmied) and #1887: the first two commits are #1928's own, and the series sits on #1887's TP page-drop deferral. It lands after both, and #1928's commits drop out then.

#1928's read-ahead (TF_LOAD_PREFETCH: reader threads read a layer's checkpoint intervals while its loader packs the weights) is ported to the other CUDA loaders. On these families it's opt-in (unset is 0) until each one's GB10 load receipts are in. GLM keeps its existing default, and #1928's own two families are unchanged.

Reviewed independently three times. The host suite, the Linux CUDA build and lean (424) pass. GB10 load-time cells per family come before any default changes.

@ashhart
ashhart force-pushed the lead/2003-load-prefetch-port branch from f7ecb13 to d7833cc Compare November 5, 2026 04:08
Extend the shared interval planner from @jschmied's #1928 to the other CUDA loaders. Keep upload and release order, join readers before borrowed sources close, retain GLM's legacy option as a fallback, and skip pack reads on prepared-image cache hits (#2003).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant