Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
d211c6b
feat(genai-openai): instrument Responses.parse and AsyncResponses.par…
RichardoMrMu Sep 24, 2026
dbb1281
test(genai-openai): cover Responses.parse instrumentation (#659)
RichardoMrMu Sep 24, 2026
cc44c8c
test(genai-openai): cover AsyncResponses.parse instrumentation (#659)
RichardoMrMu Sep 24, 2026
f97acb5
fix(openai): point Responses.parse test at its own cassette (#786)
RichardoMrMu Sep 25, 2026
4864c66
fix(openai): point AsyncResponses.parse test at its own cassette (#786)
RichardoMrMu Sep 25, 2026
9008412
test(openai): add Responses.parse cassette with structured JSON body …
RichardoMrMu Sep 25, 2026
404936f
test(openai): add AsyncResponses.parse cassette with structured JSON …
RichardoMrMu Sep 25, 2026
f689b5c
docs: regenerate instrumentation README (tox -e generate) (#786)
RichardoMrMu Sep 25, 2026
5bfb4d9
chore(openai): add changelog fragment for Responses.parse (#786)
RichardoMrMu Sep 25, 2026
f721a08
fix(openai): point Responses.parse test at its own cassette (#786)
RichardoMrMu Sep 25, 2026
bdd4a28
fix(openai): point AsyncResponses.parse test at its own cassette (#786)
RichardoMrMu Sep 25, 2026
831db95
test(openai): add Responses.parse cassette with structured JSON body …
RichardoMrMu Sep 25, 2026
be38cc7
test(openai): add AsyncResponses.parse cassette with structured JSON …
RichardoMrMu Sep 25, 2026
05d2550
docs: regenerate instrumentation README (tox -e generate) (#786)
RichardoMrMu Sep 25, 2026
070e951
chore(openai): add changelog fragment for Responses.parse (#786)
RichardoMrMu Sep 25, 2026
72714f0
fix(genai-openai): map Responses.parse text_format to gen_ai.output.type
RichardoMrMu Sep 27, 2026
a64381d
test(genai-openai): assert gen_ai.output.type for Responses.parse and…
RichardoMrMu Sep 27, 2026
3982601
test(genai-openai): restore Responses test file content lost to a bad…
RichardoMrMu Sep 27, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,12 @@ All instrumentations use [opentelemetry-util-genai](./util/opentelemetry-util-ge

| Instrumentation | Supported Package | Version | Status |
| --------------- | ----------------- | ------- | ------ |
| [opentelemetry-instrumentation-genai-bedrock](./instrumentation/opentelemetry-instrumentation-genai-bedrock) | boto3 >= 1.40.46, < 2 | 1.2b0.dev | to be released |
| [opentelemetry-instrumentation-genai-bedrock](./instrumentation/opentelemetry-instrumentation-genai-bedrock) | boto3 >= 1.40.46, < 2 | 1.2b0 | to be released |
| [opentelemetry-instrumentation-genai-claude-agent-sdk](./instrumentation/opentelemetry-instrumentation-genai-claude-agent-sdk) | claude-agent-sdk >= 0.1.14, < 1 | 1.2b0.dev | skeleton |
| [opentelemetry-instrumentation-genai-crewai](./instrumentation/opentelemetry-instrumentation-genai-crewai) | crewai >= 1.10.1, < 2 | 1.2b0.dev | skeleton |
| [opentelemetry-instrumentation-genai-dspy](./instrumentation/opentelemetry-instrumentation-genai-dspy) | dspy >= 3.3.0, < 4 | 1.2b0.dev | to be released |
| [opentelemetry-instrumentation-genai-llama-index](./instrumentation/opentelemetry-instrumentation-genai-llama-index) | llama-index-core >= 0.14.19, < 1, llama-index-instrumentation >= 0.4.3, < 1, llama-index-workflows >= 2.17.1, != 2.24.0, < 3 | 1.2b0.dev | to be released |
| [opentelemetry-instrumentation-genai-portkey](./instrumentation/opentelemetry-instrumentation-genai-portkey) | portkey-ai >= 1.0.0, < 3 | 1.2b0.dev | to be released |
| [opentelemetry-instrumentation-genai-dspy](./instrumentation/opentelemetry-instrumentation-genai-dspy) | dspy >= 3.3.0, < 4 | 1.2b0 | to be released |
| [opentelemetry-instrumentation-genai-llama-index](./instrumentation/opentelemetry-instrumentation-genai-llama-index) | llama-index-core >= 0.14.19, < 1, llama-index-instrumentation >= 0.4.3, < 1, llama-index-workflows >= 2.17.1, != 2.24.0, < 3 | 1.2b0 | to be released |
| [opentelemetry-instrumentation-genai-portkey](./instrumentation/opentelemetry-instrumentation-genai-portkey) | portkey-ai >= 1.0.0, < 3 | 1.2b0 | to be released |
| [opentelemetry-instrumentation-genai-weaviate-client](./instrumentation/opentelemetry-instrumentation-genai-weaviate-client) | weaviate-client >= 3.0.0, <5.0.0 | 1.2b0.dev | skeleton |
<!-- end -->

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Instrument `Responses.parse` and `AsyncResponses.parse`, so structured-output calls emit GenAI spans like `Responses.create`, reporting `gen_ai.output.type` as `json` for the `text_format` argument.
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,27 @@ def _is_parse_supported():
return False


def _is_responses_parse_supported():
"""Check if parse() is available on the Responses class.

The Responses API structured-output helper ``parse()`` calls the SDK's
request path directly rather than delegating to the instrumented
``Responses.create()``, so it must be wrapped separately (issue #659).
"""
try:
from openai.resources.responses.responses import ( # pylint: disable=import-outside-toplevel
Responses,
)

return hasattr(Responses, "parse")
except ImportError:
return False


class OpenAIInstrumentor(BaseInstrumentor):
def __init__(self):
self._parse_supported = False
self._responses_parse_supported = False

def instrumentation_dependencies(self) -> Collection[str]:
return _instruments
Expand Down Expand Up @@ -207,6 +225,25 @@ def _instrument(self, **kwargs):
async_responses_retrieve(handler),
)

# parse() is the Responses API structured-output helper. Like
# chat.completions.parse it maps to the same inference operation
# as create() -- the telemetry-relevant request/response fields
# are identical and its ParsedResponse result is already handled
# by the create wrappers -- but it does not delegate to the
# instrumented create(), so it must be wrapped separately (#659).
self._responses_parse_supported = _is_responses_parse_supported()
if self._responses_parse_supported:
wrap_function_wrapper(
"openai.resources.responses.responses",
"Responses.parse",
responses_create(handler),

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is covered in the latest version of the PR. extract_params() now accepts text_format and response_extractors._extract_output_type_from_text_format() maps a Pydantic/dataclass type (what Responses.parse(text_format=...) receives) to gen_ai.output.type=json, mirroring the chat.completions.parse path. The mapping is asserted by both parse tests via _assert_request_attrs(span, output_type="json"), and test_responses_create_output_type_unchanged_by_parse keeps a plain create() (no text_format) from spuriously recording an output type.

)
wrap_function_wrapper(
"openai.resources.responses.responses",
"AsyncResponses.parse",
async_responses_create(handler),
)

def _uninstrument(self, **kwargs):
import openai # pylint: disable=import-outside-toplevel

Expand All @@ -225,6 +262,9 @@ def _uninstrument(self, **kwargs):
unwrap(responses_module.AsyncResponses, "stream")
unwrap(responses_module.Responses, "retrieve")
unwrap(responses_module.AsyncResponses, "retrieve")
if self._responses_parse_supported:
unwrap(responses_module.Responses, "parse")
unwrap(responses_module.AsyncResponses, "parse")


def _get_responses_module():
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,21 @@ def _extract_output_type_from_value(text_config: object) -> str | None:
return None


def _extract_output_type_from_text_format(text_format: object) -> str | None:
"""Map ``Responses.parse(text_format=...)`` onto an output type.

``parse()`` takes the caller's Pydantic model (or dataclass) as
``text_format`` and only turns it into a JSON-schema ``text.format``
inside the SDK, after this wrapper has read the request kwargs, so the
``text`` mapping alone misses it (issue #659). Mirror the
``chat.completions.parse`` handling: a structured-output type means JSON.
Only the format metadata is recorded -- never the caller's schema.
"""
if isinstance(text_format, type):
return GenAIAttributes.GenAiOutputTypeValues.JSON.value
return None


def _extract_conversation_id(conversation: object) -> str | None:
"""Return the conversation id the ``conversation`` parameter names."""
if isinstance(conversation, str):
Expand All @@ -203,6 +218,7 @@ def extract_params(
service_tier: str | None = None,
temperature: float | None = None,
text: object | None = None,
text_format: object | None = None,
tools: Iterable[ToolParam] | None = None,
top_p: float | None = None,
**_kwargs: object,
Expand Down Expand Up @@ -230,7 +246,10 @@ def extract_params(
else None
),
temperature=_get_float(temperature),
output_type=_extract_output_type_from_value(text),
output_type=(
_extract_output_type_from_value(text)
or _extract_output_type_from_text_format(text_format)
),
tools=_get_tools(tools),
top_p=_get_float(top_p),
)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
interactions:
- request:
body: |-
{
"input": "Say this is a test",
"instructions": "You are a helpful assistant.",
"model": "gpt-4o-mini",
"stream": false,
"text": {
"format": {
"type": "json_schema",
"name": "_AsyncParseCalendarEvent",
"strict": true,
"schema": {
"properties": {
"name": {
"title": "Name",
"type": "string"
},
"date": {
"title": "Date",
"type": "string"
},
"participants": {
"items": {
"type": "string"
},
"title": "Participants",
"type": "array"
}
},
"required": [
"name",
"date",
"participants"
],
"title": "_AsyncParseCalendarEvent",
"type": "object",
"additionalProperties": false
}
}
}
}
headers:
Accept:
- application/json
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
Content-Type:
- application/json
Host:
- api.openai.com
User-Agent:
- OpenAI/Python 1.109.1
authorization:
- Bearer test_openai_api_key
method: POST
uri: https://api.openai.com/v1/responses
response:
body:
string: |-
{
"id": "resp_0f4faba17dcd0f1e0069e2f3e4907881909179832ba1237100",
"object": "response",
"created_at": 1776481253,
"status": "completed",
"background": false,
"error": null,
"frequency_penalty": 0.0,
"incomplete_details": null,
"instructions": "You are a helpful assistant.",
"max_output_tokens": null,
"max_tool_calls": null,
"model": "gpt-4o-mini-2024-07-18",
"output": [
{
"id": "msg_0f4faba17dcd0f1e0069e2f3e7b2b88190bff23981628ac400",
"type": "message",
"status": "completed",
"content": [
{
"type": "output_text",
"annotations": [],
"logprobs": [],
"text": "{\"name\":\"science fair\",\"date\":\"Friday\",\"participants\":[\"Alice\",\"Bob\"]}"
}
],
"role": "assistant"
}
],
"parallel_tool_calls": true,
"presence_penalty": 0.0,
"previous_response_id": null,
"reasoning": {
"effort": null,
"summary": null
},
"safety_identifier": null,
"service_tier": "default",
"store": true,
"temperature": 1.0,
"text": {
"format": {
"type": "json_schema",
"name": "_AsyncParseCalendarEvent"
},
"verbosity": "medium"
},
"tool_choice": "auto",
"tools": [],
"top_logprobs": 0,
"top_p": 1.0,
"truncation": "disabled",
"usage": {
"input_tokens": 22,
"input_tokens_details": {
"cached_tokens": 0
},
"output_tokens": 6,
"output_tokens_details": {
"reasoning_tokens": 0
},
"total_tokens": 28
},
"user": null,
"metadata": {}
}
headers:
Content-Type:
- application/json
openai-organization: test_openai_org_id
openai-project:
- test_openai_project_id
openai-version:
- '2020-10-01'
set-cookie:
- test_set_cookie
status:
code: 200
message: OK
version: 1
Loading