Description
FIM autocompletion over the raw /v1/completions endpoint (added in #1317) fails on every QAIRT model. Editor autocomplete clients always send a stop list with the FIM sentinel strings, the handler forwards that list to the plugin unchanged, and the qairt plugin rejects any generate call that carries stop sequences. The result is ErrorCode[-100016] and an empty completion, so /v1/completions is currently unusable with QAIRT models even though the same request works on llama_cpp GGUF models.
This was reported by a user testing FIM autocompletion from a VS Code extension against qualcomm/Qwen3-1.7B:W4A16.
Steps to reproduce
geniex serve- Send a completion request for a QAIRT model with a
stop array, the way FIM clients do:
curl http://127.0.0.1:8080/v1/completions -H "Content-Type: application/json" -d "{\"model\":\"qualcomm/Qwen3-1.7B:W4A16\",\"prompt\":\"<fim_prefix>def main():\\n print(<fim_suffix>)\\n<fim_middle>\",\"stop\":[\"<fim_prefix>\",\"<fim_suffix>\",\"<fim_middle>\",\"<|endoftext|>\"],\"max_tokens\":256,\"stream\":true}"
- The completion comes back empty and the server log shows the plugin rejection.
Observed behavior
Aug 13 16:42:19.331 INF handler\completion.go:111 Completions param="{CompletionNewParams:{Prompt:{OfString:<fim_prefix>\n# part2_agents/testcomplete.py\nimport requests\n\n\ndef main():\n print(\"hello World<fim_suffix>\")\n\n\nif __name__ == \"__main__\":\n main()\n<fim_middle> ...} Model:qualcomm/Qwen3-1.7B:W4A16 ... MaxTokens:4096 ... Stop:{OfStringArray:[<fim_prefix> <fim_suffix> <fim_middle> <file_sep> <|endoftext|> </fim_middle> </code> /src/ #- coding: utf-8 ```]} ...} Stream:true NCtx:4096 Ngl:-1 Compute:npu ...}"
Aug 13 16:42:19.395 DBG go\llm.go:422 Generate called promptLen=163 inputIDsLen=0
Aug 13 16:42:19.395 ERR go\ml.go:212 [ML] [plugins/qairt/src/llm.cpp:220:generate] --stop / --stop-file (stop sequences) is not supported by the qairt plugin
Aug 13 16:42:19.395 DBG go\ml.go:214 [ML] [src/llm.cpp:143:geniex_llm_generate] ErrorCode[-100016](Parameter not supported by this plugin): LlmGenerateOutput(full_text: nullptr, profile_data: ProfileData(ttft: 0 us, prompt_time: 0 us, decode_time: 0 us, prompt_tokens: 0, generated_tokens: 0, prefill_speed: 0 tokens/s, decoding_speed: 0 tokens/s, stop_reason: nullptr))
[GIN] 2026/08/13 - 16:42:19 | 200 | 65.1308ms | 127.0.0.1 | POST "/v1/completions"
Where it happens:
cli/server/handler/completion.go builds GenerationConfig with Stop: completionStop(req.Stop) and passes it to the plugin for every runtime.sdk/plugins/qairt/src/llm.cpp:219 rejects any generate input with stop_count > 0 and returns GENIEX_ERROR_COMMON_PARAM_NOT_SUPPORTED.
Two side effects worth noting:
- The HTTP response is
200 even though generation failed, so editor clients silently show nothing instead of surfacing an error.
- The failure is request-shaped, not model-shaped: the same model works if the client omits
stop, but real FIM clients never omit it.
Expected behavior
Stop sequences on /v1/completions should work regardless of which plugin serves the model. For plugins that do not implement stop sequences natively, the server can enforce them host side:
- Do not forward
Stop to a plugin that rejects it.
- Match the stop strings against the streamed text inside the
OnToken callback.
- Cancel generation when a stop string matches (the qairt pipeline already supports cancellation through the callback return value, reported as
stop_reason: "user").
- Truncate the returned text at the match so the client receives the completion up to, but not including, the stop sequence.
With that in place the request above returns the FIM completion on QAIRT models exactly as it does on llama_cpp models today, and no plugin changes are required.
Additional context
With GGUF models the same setup works, which confirms the failure is specific to the qairt plugin parameter rejection. I am happy to submit a PR implementing the host side stop handling.
Environment
- GenieX: current main build
- Model:
qualcomm/Qwen3-1.7B:W4A16 (qairt runtime, NPU)
- Client: VS Code FIM autocomplete extension via
/v1/completions, stream: true
- OS: Windows 11
Description
FIM autocompletion over the raw
/v1/completionsendpoint (added in #1317) fails on every QAIRT model. Editor autocomplete clients always send astoplist with the FIM sentinel strings, the handler forwards that list to the plugin unchanged, and the qairt plugin rejects any generate call that carries stop sequences. The result isErrorCode[-100016]and an empty completion, so/v1/completionsis currently unusable with QAIRT models even though the same request works on llama_cpp GGUF models.This was reported by a user testing FIM autocompletion from a VS Code extension against
qualcomm/Qwen3-1.7B:W4A16.Steps to reproduce
geniex servestoparray, the way FIM clients do:Observed behavior
Where it happens:
cli/server/handler/completion.gobuildsGenerationConfigwithStop: completionStop(req.Stop)and passes it to the plugin for every runtime.sdk/plugins/qairt/src/llm.cpp:219rejects anygenerateinput withstop_count > 0and returnsGENIEX_ERROR_COMMON_PARAM_NOT_SUPPORTED.Two side effects worth noting:
200even though generation failed, so editor clients silently show nothing instead of surfacing an error.stop, but real FIM clients never omit it.Expected behavior
Stop sequences on
/v1/completionsshould work regardless of which plugin serves the model. For plugins that do not implement stop sequences natively, the server can enforce them host side:Stopto a plugin that rejects it.OnTokencallback.stop_reason: "user").With that in place the request above returns the FIM completion on QAIRT models exactly as it does on llama_cpp models today, and no plugin changes are required.
Additional context
With GGUF models the same setup works, which confirms the failure is specific to the qairt plugin parameter rejection. I am happy to submit a PR implementing the host side stop handling.
Environment
qualcomm/Qwen3-1.7B:W4A16(qairt runtime, NPU)/v1/completions,stream: true