Use shared retrieval document model across instrumentations - #775
Open
JacksonWeber wants to merge 3 commits into
Open
JacksonWeber wants to merge 3 commits into
JacksonWeber wants to merge 3 commits into
Conversation
Add RetrievalDocument with optional id and score fields and adopt it in DSPy, LangChain, and LlamaIndex. Stop capturing document text while preserving query capture and legacy dictionary serialization. Cover the behavior with regression tests, README updates, and changelog fragments. Refs open-telemetry#743 Assisted-by: GPT-6 Astra Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pull request dashboard statusWaiting on the author · refreshed 2026-09-23 14:13 UTC Respond to 1 review item (e.g. link a commit, explain why not, ask a follow-up):
Status above doesn't look right?
|
Assisted-by: GPT-6 Astra Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
LlamaIndex extraction needs exception guarding and non-finite score normalization.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (2)
What changed in this PR
Introduces a shared RetrievalDocument model across DSPy, LangChain, and LlamaIndex while stopping document-text capture.
Changes:
- Adds optional IDs and scores with legacy serialization support.
- Updates instrumentation, tests, documentation, and changelogs.
- Preserves query capture.
| File | Description |
|---|---|
util/opentelemetry-util-genai/tests/test_handler_retrieval.py |
Tests typed and legacy document serialization. |
util/opentelemetry-util-genai/src/opentelemetry/util/genai/types.py |
Adds RetrievalDocument. |
util/opentelemetry-util-genai/src/opentelemetry/util/genai/_retrieval_invocation.py |
Supports typed and legacy documents. |
util/opentelemetry-util-genai/README.rst |
Documents the shared model. |
util/opentelemetry-util-genai/.changelog/+retrieval-documents.added |
Records the utility change. |
instrumentation/opentelemetry-instrumentation-genai-llama-index/tests/test_retrieval.py |
Tests ID and score extraction. |
instrumentation/opentelemetry-instrumentation-genai-llama-index/src/opentelemetry/instrumentation/genai/llama_index/_handler.py |
Emits typed retrieval documents. |
instrumentation/opentelemetry-instrumentation-genai-llama-index/README.rst |
Documents retrieval capture changes. |
instrumentation/opentelemetry-instrumentation-genai-llama-index/.changelog/+retrieval-documents.changed |
Records instrumentation changes. |
instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_retriever.py |
Updates retrieval assertions. |
instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_callback_handler.py |
Tests typed document conversion. |
instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/callback_handler.py |
Converts documents to RetrievalDocument. |
instrumentation/opentelemetry-instrumentation-genai-langchain/README.rst |
Documents ID and score-only capture. |
instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/+retrieval-documents.changed |
Records instrumentation changes. |
instrumentation/opentelemetry-instrumentation-genai-dspy/tests/test_retrieve.py |
Tests passage-count and content suppression behavior. |
instrumentation/opentelemetry-instrumentation-genai-dspy/src/opentelemetry/instrumentation/genai/dspy/patch.py |
Emits typed documents for text-only passages. |
instrumentation/opentelemetry-instrumentation-genai-dspy/README.rst |
Documents DSPy retrieval behavior. |
instrumentation/opentelemetry-instrumentation-genai-dspy/.changelog/+retrieval-documents.changed |
Records instrumentation changes. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Assisted-by: GPT-6 Astra Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
lmolkova
approved these changes
Sep 23, 2026
|
|
||
| invocation.documents = [{"content": str(psg)} for psg in passages] | ||
| # Retrieve returns passage text, without document IDs or scores. | ||
| invocation.documents = [RetrievalDocument() for _ in passages] |
Member
There was a problem hiding this comment.
Unfortunate we end up with empty docs here, but that's what semconv does right now. Created open-telemetry/semantic-conventions-genai#537 to track, let's follow up once it's resolved in some way.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add the shared
RetrievalDocumentmodel with optionalidandscorefields and adopt it in DSPy, LangChain, and LlamaIndex forgen_ai.retrieval.documents.Document text is no longer captured; unavailable IDs and scores serialize as
null. Query capture, content-capture gating, SDK return values, and legacy dictionary serialization in the util remain unchanged. Includes regression tests and README updates.Fixes #743.
Type of change
How has this been tested?
333 targeted tests passed in an isolated Python 3.12 environment. Run
python -m pytestagainst these files with the corresponding package/test dependencies installed:util/opentelemetry-util-genai/tests/test_handler_retrieval.py— 32 passed.instrumentation/opentelemetry-instrumentation-genai-dspy/tests/test_retrieve.py— 12 passed.instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_retriever.pyandtests/test_callback_handler.pyin the same package — 278 passed.instrumentation/opentelemetry-instrumentation-genai-llama-index/tests/test_retrieval.py— 11 passed.Scoped Pyright, Ruff, formatting, rstcheck, and PyPI README rendering checks passed. Full tox gates (oldest/latest, conformance, precommit, and typecheck) were attempted but blocked during dependency setup by PyPI TLS handshake failures.
Checklist
+retrieval-documentsfilenames still need to be renamed to775Assisted-by: GPT-6 Astra