Skip to content

perf(qwen35): greedy decode allocates nothing and downloads 4 bytes per token (Refs #4215) - #4332

Closed
noahgift wants to merge 5 commits into
mainfrom
perf/qwen35-decode-4215
Closed

perf(qwen35): greedy decode allocates nothing and downloads 4 bytes per token (Refs #4215)#4332
noahgift wants to merge 5 commits into
mainfrom
perf/qwen35-decode-4215

perf(qwen35): a greedy decode token allocates nothing on host or devi…

f80b3f8
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar