Skip to content

Perf/kernel optimizations - #1

Merged
barhanc merged 39 commits into
mainfrom
perf/kernel-optimizations
Oct 21, 2026
Merged

barhanc merged 39 commits into
mainfrom
perf/kernel-optimizations

Conversation

@barhanc

@barhanc barhanc commented Oct 20, 2026

Copy link
Copy Markdown
Owner

No description provided.

@barhanc barhanc self-assigned this Oct 20, 2026
… tiling

- Upgrade mm, bmm, conv2d and conv-with-clamp 64x64 kernels to 256-thread (16x16) workgroups
- Reduce per-thread accumulators from 64 floats to 16 floats (4 vec4s) to eliminate register spilling and maximize GPU occupancy
- Adopt 100% coalesced 128-bit memory transactions with zero inner-loop divisions/modulos
- Add dedicated FP16 1x1 tile64 compute shaders for convolution and conv_with_clamp
- Benchmarked on real GPU: DepthART at 2.69x, YOLO26-N at 2.0x, Conv3x3 at 2.66x speedup
…e64 kernels

- Eliminate 8 integer divisions and 8 modulos per thread from the inner tile loop
- Achieves 3.35x speedup on ConvWithClamp 3x3 (0.170ms) and 2.63x on Conv2D 3x3 (0.230ms)
- Push YOLO26-N to 2.10x and DepthART to 2.30x speedup on real GPU
Rebalance the final decoder convolutions before half conversion, add photographic-range regression coverage, and specialize single-channel pointwise convolution with FP32 accumulation. Replace the example precision dropdown with an accessible sliding switch and record Chrome benchmark evidence.
@barhanc
barhanc merged commit 7237ca4 into main Oct 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant