Repository navigation

cuda/norms: one tree a width, IEEE 1/sqrt; Nemotron's add_rmsnorm on the shared kernel - #796

Merged

ashhart merged 1 commit into

ashhart:mainfrom

jschmied:cuda-nemotron-shared-norms

Oct 16, 2026

Merged

cuda/norms: one tree a width, IEEE 1/sqrt; Nemotron's add_rmsnorm on the shared kernel#796
ashhart merged 1 commit into
ashhart:mainfrom
jschmied:cuda-nemotron-shared-norms

Commits on Oct 14, 2026
- cuda/norms: add_rms_h issues every load before any is used, and each warp folds the warps' sums itself (no broadcast, one block sync where one norm runs), the same bits
  jschmied
  committed
  b3ad1b9