Skip to content

Responses API: remaining per-upstream discriminators after nan-devops#637 #122

Closed

Description

@sre-helmcode

After nan-devops#637, the Responses API body and event fields are scrubbed (allowlist, nested model, provider-only fields, resp_ ids encrypted). These discriminators remain, so a caller can still tell which upstream serves a model (owner requirement: they must not):

  1. Output item and delta ids keep each upstream's format. Novita/DeepSeek rs_/msg_0217…ffff<ipv4> embed an upstream-internal IPv4 (172.21.x.x). The vLLM delta item_id is the raw upstream resp_<16hex>, unencrypted. On glm the message item id equals the upstream response id. Needs a reversible mapping on the request path, so that previous_response_id and item_reference still work.
  2. Reasoning item shape differs per upstream. deepseek {id, summary, type, status}; glm content[output_text, annotations] + status; vLLM summary: [] + content[reasoning_text] with no status; gemma4 message content carries logprobs: null. The "identical output item key sets" test uses stubs, so add real-shape fixtures.
  3. Encrypted response id length varies per upstream and model_id (over 1000 chars on deepseek). On glm, response.created and response.completed carry different ids.
  4. Error bodies differ per upstream. A made-up previous_response_id returns invalid response ID … trace_id on Novita and Response with id '…' not found. on vLLM.
  5. Stream shape differs. vLLM groups emit only output_text.delta and completed, and sequence_number is present only on some upstreams. This is behavioural, so it can only be closed with a normalising stream re-emitter.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions