Repository navigation
switch to openai compatible endpoints - #171
Conversation
Signed-off-by: va <va@us.ibm.com>
Juan Cruz-Benito (cbjuan)
left a comment
There was a problem hiding this comment.
Overall the endpoint mapping and request bodies look correct. The refactor is clean. However, there are two response-handling bugs that may break the model selection and disclaimer acceptance flows, plus a likely streaming issue. I know you have tested this on your local, so dismiss those comments if they are not a real issue
Bug 1: getModel response not transformed
GET /v1/models/{model_name} returns a ModelObject:
{ "id": "string", "object": "model", "created": 0, "owned_by": "string" }The code casts this directly as ModelInfo, but downstream in selectModel.ts:85:
currentModel = await apiService.getModel(selectedModel._id);…then accesses currentModel._id, currentModel.display_name, currentModel.disclaimer?.accepted — all of which would be undefined from a ModelObject.
Note that modelTransform won't fully solve this either, since it hardcodes disclaimer: { accepted: true }, while the caller (selectModel.ts:77) checks disclaimer?.accepted to decide whether to show the disclaimer panel. Consider whether this endpoint should keep using the old /model/{model_id} route (which returns the full ModelDetail with real disclaimer state) or whether modelTransform needs to be enhanced.
Bug 2: getModelDisclaimer response field name mismatch (id vs _id)
GET /v1/models/{model_name}/disclaimer returns ModelDisclaimerOutput:
{ "id": "string", "object": "disclaimer", "version": "...", "title": "...", "body": "...", "model": "...", "accepted": false }The ID field is id, not _id. The code casts this as ModelDisclaimer (which has _id), so in acceptDisclaimer.ts:105:
await apiService.postDisclaimerAcceptance(model._id, disclaimer._id, true);disclaimer._id would be undefined, causing the POST body to send { disclaimer: undefined, accepted: true }.
Fix: map id → _id in the response handling, or update ModelDisclaimer and its consumers.
Likely bug 3: streaming delta field not handled for /v1/chat/completions
The standard OpenAI streaming protocol for chat completions uses delta in each SSE chunk, not message:
{"choices": [{"index": 0, "delta": {"content": "token..."}, "finish_reason": null}]}The OpenAIChoice type only has text? and message?, and toModelPromptResponse maps:
{ "generated_text": c.text ?? c.message?.content }If the QCA service follows the standard protocol, every streaming chunk would produce { generated_text: undefined }.
Suggested fix — add delta defensively to OpenAIChoice and the transform:
interface OpenAIChoice {
index: number,
text?: string
message?: OpenAIMessage
delta?: OpenAIMessage
}{ "generated_text": c.text ?? c.message?.content ?? c.delta?.content }If the QCA service intentionally deviates from the standard and uses message even in streaming, a comment noting that would be helpful.
Signed-off-by: va <va@us.ibm.com>
Signed-off-by: va <va@us.ibm.com>
Signed-off-by: va <va@us.ibm.com>
|
the issues have been addressed
while not exactly as described here, investigating this did lead to uncovering another issue related to the improper schema of the streaming response data (and missing
good catch! 26866fc
fixed: b093e69
along with updating existing tests (a53d1e0) accordingly, new tests (8093881) have also been added. |
Juan Cruz-Benito (cbjuan)
left a comment
There was a problem hiding this comment.
All substantive issues from the previous review have been addressed. The [DONE] marker handling in SSEParser is a solid addition with good test coverage. CI passes. LGTM!
Description
this PR updates the extension to use the OpenAI compatible endpoints of the Qiskit Code Assistant service. it replaces the calls to the custom endpoint to calls the OpenAI endpoints (i.e.,
/v1/chat/completions). in addition the/modelsrelated endpoints are also update to more closely align with the OpenAI endpoint patternsLinked Issue(s)
Fixes #108
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
this has been tested by running the extension against a local Ollama deployment of the Qiskit Code Assistant as well the remote Qiskit Code Assistant service deployment
Checklist: