You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
apr serve stop handling: stop on every EOS and stop string, on every route and backend #5313
Scope: Stop-token handling in apr serve: /v1/completions running past stop on CPU GGUF, GPU paths stopping on a single EOS or none, native routes leaking the im_end control token into text, and one shared sampling/stop function instead of several.
Why it matters: Clients see run-on text and control tokens; the same model stops correctly in llama-server and ollama.
Done when: On the CRUX prompt set, every apr serve route on CPU and GPU stops where llama-server stops and returns no control tokens, through one shared stop function.
Epic: #3999. Folded under triage rule 7b (2026-10-14): each row below was its own issue, closed as not planned and kept here as a sub-ticket. Reopen an issue when its work starts or its PR opens.
changed the title [-]MASTER E6 0.73: folded sub-tickets (epic #3999)[/-][+]apr serve stop handling: stop on every EOS and stop string, on every route and backend[/+]on Oct 14, 2026
Activity
[-]MASTER E6 0.73: folded sub-tickets (epic #3999)[/-][+]apr serve stop handling: stop on every EOS and stop string, on every route and backend[/+]noahgift commentedon Oct 22, 2026
Tracked in #5348 (triage 2026-10-22).
noahgift commentedon Oct 22, 2026
tracked in #5348