Skip to content

apr serve stop handling: stop on every EOS and stop string, on every route and backend #5313

Closed as not planned

Description

@noahgift

Scope: Stop-token handling in apr serve: /v1/completions running past stop on CPU GGUF, GPU paths stopping on a single EOS or none, native routes leaking the im_end control token into text, and one shared sampling/stop function instead of several.

Why it matters: Clients see run-on text and control tokens; the same model stops correctly in llama-server and ollama.

Done when: On the CRUX prompt set, every apr serve route on CPU and GPU stops where llama-server stops and returns no control tokens, through one shared stop function.

Epic: #3999. Folded under triage rule 7b (2026-10-14): each row below was its own issue, closed as not planned and kept here as a sub-ticket. Reopen an issue when its work starts or its PR opens.

Sub-tickets

Activity

  1. added this to the 0.73.0 milestone on Oct 14, 2026
  2. changed the title [-]MASTER E6 0.73: folded sub-tickets (epic #3999)[/-] [+]apr serve stop handling: stop on every EOS and stop string, on every route and backend[/+] on Oct 14, 2026
  3. added
    bugSomething isn't working
    P2Medium priority
    on Oct 14, 2026
  4. noahgift commented on Oct 22, 2026

    @noahgift
    MemberAuthor

    Tracked in #5348 (triage 2026-10-22).

  5. noahgift commented on Oct 22, 2026

    @noahgift
    MemberAuthor

    tracked in #5348

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium prioritybugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions