Repository navigation
apr serve: /v1/chat/completions stream leaks <|im_end|> into delta content (1/14 CRUX prompts) #4340
Description
Binary: apr v0.69.3-rc.2 x86_64 cpu asset (sha256 faa473e339f81744…). Model: qwen2.5-coder-1.5b-instruct-q4_k_m.gguf (sha cc324af070c2). Found by the #3797 CRUX serve run #2 on lambda (rows under /mnt/nvme-raid0/tmp/tmp.ZtvuBEgtRL/cc324af070c2/off/serve/).
On prompt ctl-recall-7, stream mode, the SSE deltas carry a literal <|im_end|>. The nonstream response for the same prompt is clean. One occurrence in 14 prompts: this is an anecdote until it is reproduced on neighbouring prompts.
done_when: special tokens are never emitted in stream deltas. A test pins it.
Refs #3797
Activity
comparator: none+ reason #3774noahgift commentedon Oct 12, 2026
evidence for close: 989cb01 (#4655), ancestor of main 6bd89b0. Done-when is "special tokens never emitted in stream deltas, a test pins it". Checked at main: ChatStopFilter (crates/aprender-serve/src/api/chat_completions_stream.rs) filters the live SSE path in openai_handlers.rs, and the test module sse_stream_stop_markers_4340 is wired in api/tests/mod.rs.
noahgift commentedon Oct 12, 2026
Cop batch close (FLOW-004 drain, ≤43/d): fix verified on main — merged #4655 ; ancestor of origin/main: 989cb01 . Evidence: #4340 (comment)