feat(instrumentation): Add litellm package instrumentation#4319
Open
nina-kollman wants to merge 10 commits into
Open
feat(instrumentation): Add litellm package instrumentation#4319nina-kollman wants to merge 10 commits into
nina-kollman wants to merge 10 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughIntroduces a new LiteLLM OpenTelemetry instrumentation
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
nina-kollman
changed the title
pyproject declares `readme = "README.md"`, but the file was absent, so the hatchling editable build failed with `OSError: Readme file does not exist`, breaking the Lint, Build, and Test CI jobs at the install step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/opentelemetry-instrumentation-litellm/tests/test_nesting.py (1)
72-87: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winMake uninstrument cleanup robust to partial setup failures.
At Line 74-76, both
.instrument(...)calls happen before thetry. If Line 75 raises, Line 86 won’t run for the already-instrumented OpenAI instrumentor. Move instrumentation insidetry(or track success flags) to guarantee teardown.Suggested fix
openai_instrumentor = OpenAIInstrumentor() litellm_instrumentor = LiteLLMInstrumentor() - openai_instrumentor.instrument(tracer_provider=tracer_provider, meter_provider=meter_provider) - litellm_instrumentor.instrument(tracer_provider=tracer_provider, meter_provider=meter_provider) try: + openai_instrumentor.instrument( + tracer_provider=tracer_provider, meter_provider=meter_provider + ) + litellm_instrumentor.instrument( + tracer_provider=tracer_provider, meter_provider=meter_provider + ) litellm.completion( model="gpt-3.5-turbo", messages=MESSAGES, mock_response="hello", ) spans = span_exporter.get_finished_spans() assert [span.name for span in spans] == ["litellm.chat"] finally: litellm_instrumentor.uninstrument() openai_instrumentor.uninstrument()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/opentelemetry-instrumentation-litellm/tests/test_nesting.py` around lines 72 - 87, The test cleanup in test_nesting.py is not robust if one instrument() call fails after the other has already succeeded. Move the OpenAIInstrumentor.instrument and LiteLLMInstrumentor.instrument setup inside the try block in test_nesting, or track whether each instrumentor was successfully enabled before calling uninstrument in finally, so partial setup failures still guarantee teardown.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/__init__.py`:
- Around line 696-705: Make LiteLLMInstrumentor.__init__() side-effect free: it
currently mutates process-wide Config state (exception_logger,
use_legacy_attributes, and get_common_metrics_attributes) as soon as the
instrumentor is constructed. Move those Config assignments out of __init__ and
into the actual instrumentation path (or guard them so they only run when
instrumentation is applied), so creating a new LiteLLMInstrumentor before the
is_instrumented_by_opentelemetry check does not change already-installed
wrappers or swap callbacks unexpectedly.
- Around line 115-122: The system resolution in _resolve_system_from_kwargs() is
returning raw LiteLLM provider prefixes instead of the published gen_ai.system
values. Update this helper to normalize aliases like azure and bedrock to the
constants used in opentelemetry.semconv_ai, and make sure the same normalized
value is what _start_span(), _record_metrics(), and _record_exception_metric()
use when custom_llm_provider is absent. Keep the fallback behavior for unknown
providers, but ensure the returned system key matches the repo’s semantic
convention mapping.
In `@packages/opentelemetry-instrumentation-litellm/tests/conftest.py`:
- Around line 71-83: The fixture setup leaks TRACELOOP_TRACE_CONTENT if
LiteLLMInstrumentor.instrument() fails before the yield, because the cleanup
after yield is skipped. Update the conftest.py fixture around
LiteLLMInstrumentor.instrument to guarantee cleanup with a try/finally block or
monkeypatch so os.environ.pop(TRACELOOP_TRACE_CONTENT, None) always runs even
when setup raises, and keep instrumentor.uninstrument() in the same guaranteed
cleanup path.
In `@packages/sample-app/sample_app/litellm_async_streaming.py`:
- Around line 173-178: The tool execution loop in litellm_async_streaming should
not abort when tc["arguments"] is malformed or when impl(**args) receives
invalid model-supplied parameters. Update the tool-call handling around the
tool_calls loop to catch JSON parsing and invocation errors, and convert them
into structured tool error messages (similar to the unknown tool fallback) so
tool_messages can still be returned and the final model turn can recover.
- Around line 110-118: The calculate() helper still uses eval(), and the current
character whitelist does not prevent expensive expressions like exponentiation
from model-controlled tool args. Replace the eval-based execution in calculate()
with a bounded AST-based parser that only allows the intended arithmetic
nodes/operators, and add explicit length/node limits so the expression is
rejected before evaluation if it is too large or complex.
- Around line 129-148: The _consume_stream function assumes every streamed chunk
has at least one choice, but the usage-only metadata chunk can have an empty
choices array and crash on chunk.choices[0]. Update _consume_stream in
litellm_async_streaming.py to guard against empty chunk.choices before accessing
delta, skipping those chunks while still accumulating content and tool_calls
from normal chunks.
In `@packages/sample-app/sample_app/litellm_completion.py`:
- Around line 153-158: The tool-call loop in litellm_completion should not abort
when tc.function.arguments is malformed or when impl(**args) receives bad
model-supplied parameters. Update the tool processing in the tool_calls loop to
catch json.loads and tool invocation failures, then return an error
payload/message for that specific tool call instead of raising, so the loop can
continue processing remaining calls. Use the existing tool registry flow around
TOOL_REGISTRY.get, tc.function.name, and the result assignment to centralize the
error handling.
- Line 16: The sample app dependency note only mentions OPENAI_API_KEY, but the
default Traceloop.init() flow also requires TRACELOOP_API_KEY for spans to be
emitted. Update the requirement/documentation in litellm_completion.py to
include TRACELOOP_API_KEY alongside OPENAI_API_KEY so the sample clearly states
all needed credentials. Refer to Traceloop.init and the existing “Requires” note
when making the change.
---
Outside diff comments:
In `@packages/opentelemetry-instrumentation-litellm/tests/test_nesting.py`:
- Around line 72-87: The test cleanup in test_nesting.py is not robust if one
instrument() call fails after the other has already succeeded. Move the
OpenAIInstrumentor.instrument and LiteLLMInstrumentor.instrument setup inside
the try block in test_nesting, or track whether each instrumentor was
successfully enabled before calling uninstrument in finally, so partial setup
failures still guarantee teardown.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 4b11ec10-2133-4153-9039-151f86f44fb6
⛔ Files ignored due to path filters (3)
packages/opentelemetry-instrumentation-litellm/uv.lockis excluded by!**/*.lockpackages/sample-app/uv.lockis excluded by!**/*.lockpackages/traceloop-sdk/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (21)
packages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/__init__.pypackages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/config.pypackages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/event_emitter.pypackages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/event_models.pypackages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/utils.pypackages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/version.pypackages/opentelemetry-instrumentation-litellm/project.jsonpackages/opentelemetry-instrumentation-litellm/pyproject.tomlpackages/opentelemetry-instrumentation-litellm/pytest.inipackages/opentelemetry-instrumentation-litellm/tests/conftest.pypackages/opentelemetry-instrumentation-litellm/tests/test_completion.pypackages/opentelemetry-instrumentation-litellm/tests/test_custom_provider.pypackages/opentelemetry-instrumentation-litellm/tests/test_embeddings.pypackages/opentelemetry-instrumentation-litellm/tests/test_nesting.pypackages/opentelemetry-instrumentation-litellm/tests/test_streaming.pypackages/opentelemetry-semantic-conventions-ai/opentelemetry/semconv_ai/__init__.pypackages/sample-app/sample_app/litellm_async_streaming.pypackages/sample-app/sample_app/litellm_completion.pypackages/traceloop-sdk/pyproject.tomlpackages/traceloop-sdk/traceloop/sdk/instruments.pypackages/traceloop-sdk/traceloop/sdk/tracing/tracing.py
Comment on lines
+696
to
+705
| def __init__( | ||
| self, | ||
| exception_logger=None, | ||
| use_legacy_attributes=True, | ||
| get_common_metrics_attributes=lambda: {}, | ||
| ): | ||
| super().__init__() | ||
| Config.exception_logger = exception_logger | ||
| Config.use_legacy_attributes = use_legacy_attributes | ||
| Config.get_common_metrics_attributes = get_common_metrics_attributes |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Make LiteLLMInstrumentor.__init__() side-effect free.
These assignments mutate process-wide Config immediately. Because packages/traceloop-sdk/traceloop/sdk/tracing/tracing.py, Lines 807-812 constructs a new LiteLLMInstrumentor before checking is_instrumented_by_opentelemetry, a second init call can silently flip already-installed wrappers between legacy-attribute mode and event mode, or replace the common metric attributes callback, without re-instrumenting anything.
Suggested fix
class LiteLLMInstrumentor(BaseInstrumentor):
@@
def __init__(
self,
exception_logger=None,
use_legacy_attributes=True,
get_common_metrics_attributes=lambda: {},
):
super().__init__()
- Config.exception_logger = exception_logger
- Config.use_legacy_attributes = use_legacy_attributes
- Config.get_common_metrics_attributes = get_common_metrics_attributes
+ self._exception_logger = exception_logger
+ self._use_legacy_attributes = use_legacy_attributes
+ self._get_common_metrics_attributes = get_common_metrics_attributes
@@
def _instrument(self, **kwargs):
+ Config.exception_logger = self._exception_logger
+ Config.use_legacy_attributes = self._use_legacy_attributes
+ Config.get_common_metrics_attributes = self._get_common_metrics_attributes
+
tracer_provider = kwargs.get("tracer_provider")
tracer = get_tracer(__name__, __version__, tracer_provider)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def __init__( | |
| self, | |
| exception_logger=None, | |
| use_legacy_attributes=True, | |
| get_common_metrics_attributes=lambda: {}, | |
| ): | |
| super().__init__() | |
| Config.exception_logger = exception_logger | |
| Config.use_legacy_attributes = use_legacy_attributes | |
| Config.get_common_metrics_attributes = get_common_metrics_attributes | |
| def __init__( | |
| self, | |
| exception_logger=None, | |
| use_legacy_attributes=True, | |
| get_common_metrics_attributes=lambda: {}, | |
| ): | |
| super().__init__() | |
| self._exception_logger = exception_logger | |
| self._use_legacy_attributes = use_legacy_attributes | |
| self._get_common_metrics_attributes = get_common_metrics_attributes |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/__init__.py`
around lines 696 - 705, Make LiteLLMInstrumentor.__init__() side-effect free: it
currently mutates process-wide Config state (exception_logger,
use_legacy_attributes, and get_common_metrics_attributes) as soon as the
instrumentor is constructed. Move those Config assignments out of __init__ and
into the actual instrumentation path (or guard them so they only run when
instrumentation is applied), so creating a new LiteLLMInstrumentor before the
is_instrumented_by_opentelemetry check does not change already-installed
wrappers or swap callbacks unexpectedly.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/opentelemetry-instrumentation-litellm/README.md`:
- Around line 3-5: The README badge image is missing alt text, which triggers
markdownlint and hurts accessibility. Update the badge markup in the README so
the img element used for the PyPI badge includes descriptive alt text that
identifies the package badge, keeping the link and image placement unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b8e25a91-898c-42db-9f11-7407c9c3d20f
📒 Files selected for processing (1)
packages/opentelemetry-instrumentation-litellm/README.md
Comment on lines
+3
to
+5
| <a href="https://pypi.org/project/opentelemetry-instrumentation-litellm/"> | ||
| <img src="https://badge.fury.io/py/opentelemetry-instrumentation-litellm.svg"> | ||
| </a> |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add alt text to the badge image.
This trips markdownlint and leaves the badge inaccessible to screen readers.
♿ Proposed fix
<a href="https://pypi.org/project/opentelemetry-instrumentation-litellm/">
- <img src="https://badge.fury.io/py/opentelemetry-instrumentation-litellm.svg">
+ <img
+ src="https://badge.fury.io/py/opentelemetry-instrumentation-litellm.svg"
+ alt="PyPI version"
+ >
</a>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| <a href="https://pypi.org/project/opentelemetry-instrumentation-litellm/"> | |
| <img src="https://badge.fury.io/py/opentelemetry-instrumentation-litellm.svg"> | |
| </a> | |
| <a href="https://pypi.org/project/opentelemetry-instrumentation-litellm/"> | |
| <img | |
| src="https://badge.fury.io/py/opentelemetry-instrumentation-litellm.svg" | |
| alt="PyPI version" | |
| > | |
| </a> |
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 4-4: Images should have alternate text (alt text)
(MD045, no-alt-text)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/opentelemetry-instrumentation-litellm/README.md` around lines 3 - 5,
The README badge image is missing alt text, which triggers markdownlint and
hurts accessibility. Update the badge markup in the README so the img element
used for the PyPI badge includes descriptive alt text that identifies the
package badge, keeping the link and image placement unchanged.
Source: Linters/SAST tools
Bring the LiteLLM instrumentation up to the same semconv level as the openai/anthropic packages: - emit gen_ai.input.messages / gen_ai.output.messages (parts-based JSON schema) instead of the retired flat gen_ai.prompt.* / gen_ai.completion.* attributes - set gen_ai.response.finish_reasons (top-level, ungated, deduped) and map provider finish reasons to the OTel vocabulary (tool_calls -> tool_call); per-message finish_reason always a string - carry tool_calls + mapped finish_reason into gen_ai.choice events - replace deprecated gen_ai.system with gen_ai.provider.name and llm.request.type with gen_ai.operation.name, using upstream well-known values (fixes azure.ai.openai, splits gcp.vertex_ai vs gcp.gemini) - drop the now-unused GenAISystem.LITELLM enum member Tests updated to assert the JSON schema, provider/operation attributes, and finish-reason mapping/dedup; added event-mode tool_calls coverage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Gal Zilberstein seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
packages/opentelemetry-instrumentation-litellm/tests/test_events.py (1)
13-36: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThis test doesn't actually cover the
use_legacy_attributes=Falsebranch.
_emit_choice_events()is invoked directly here, and the supplied implementation forpackages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/__init__.py:427-441does not readConfig.use_legacy_attributes. That means Lines 13-36 still pass even if the public-path gate for event mode regresses. Either drop the config-specific framing from this test or drive the public instrumentation path that consults the flag.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/opentelemetry-instrumentation-litellm/tests/test_events.py` around lines 13 - 36, The test is incorrectly framed as covering Config.use_legacy_attributes=False because it calls _emit_choice_events() directly, which bypasses the public path that checks the flag. Either remove the config-specific assertion from test_events.py or rewrite the test to exercise the public instrumentation entrypoint in opentelemetry/instrumentation/litellm/__init__.py that actually consults Config.use_legacy_attributes, so the test truly validates the branch.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/__init__.py`:
- Around line 583-597: The _finalize() path in the litellm instrumentation can
leak spans if _model_as_dict(), _handle_response(), or _record_metrics() raises
before span.end() is reached. Update _finalize() so the finalization work is
wrapped defensively and span.end() always executes, while preserving the
existing provider resolution, GenAIAttributes.GEN_AI_PROVIDER_NAME setting, and
Status(StatusCode.OK) behavior when the span is recording.
---
Nitpick comments:
In `@packages/opentelemetry-instrumentation-litellm/tests/test_events.py`:
- Around line 13-36: The test is incorrectly framed as covering
Config.use_legacy_attributes=False because it calls _emit_choice_events()
directly, which bypasses the public path that checks the flag. Either remove the
config-specific assertion from test_events.py or rewrite the test to exercise
the public instrumentation entrypoint in
opentelemetry/instrumentation/litellm/__init__.py that actually consults
Config.use_legacy_attributes, so the test truly validates the branch.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 1ac59e8b-e88a-4f4b-8712-089c075c15a1
📒 Files selected for processing (8)
packages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/__init__.pypackages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/event_emitter.pypackages/opentelemetry-instrumentation-litellm/tests/test_completion.pypackages/opentelemetry-instrumentation-litellm/tests/test_custom_provider.pypackages/opentelemetry-instrumentation-litellm/tests/test_embeddings.pypackages/opentelemetry-instrumentation-litellm/tests/test_events.pypackages/opentelemetry-instrumentation-litellm/tests/test_provider_resolution.pypackages/opentelemetry-instrumentation-litellm/tests/test_streaming.py
🚧 Files skipped from review as they are similar to previous changes (5)
- packages/opentelemetry-instrumentation-litellm/tests/test_embeddings.py
- packages/opentelemetry-instrumentation-litellm/tests/test_streaming.py
- packages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/event_emitter.py
- packages/opentelemetry-instrumentation-litellm/tests/test_custom_provider.py
- packages/opentelemetry-instrumentation-litellm/tests/test_completion.py
Comment on lines
+583
to
+597
| def _finalize(span, metrics, event_logger, request_type, response, model, kwargs, duration, accumulated_provider=None): | ||
| provider = ( | ||
| accumulated_provider | ||
| or _resolve_provider_from_response(response) | ||
| or _resolve_provider_from_kwargs(kwargs) | ||
| ) | ||
| # Serialize the response once and reuse it for attributes, events, and metrics — | ||
| # model_dump() on a ModelResponse is the most expensive step in the wrapper. | ||
| response_dict = _model_as_dict(response) | ||
| _set_span_attribute(span, GenAIAttributes.GEN_AI_PROVIDER_NAME, provider) | ||
| _handle_response(span, request_type, response_dict, event_logger) | ||
| _record_metrics(metrics, provider, model, duration, response_dict, request_type) | ||
| if span.is_recording(): | ||
| span.set_status(Status(StatusCode.OK)) | ||
| span.end() |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Guarantee span closure when finalization work fails.
_finalize() ends the span only after response serialization, response/event handling, and metrics recording all succeed. If any instrumentation-only step raises, the span leaks and the successful LiteLLM response path can be turned into an application error. Wrap finalization work so span.end() always runs.
Suggested fix
def _finalize(span, metrics, event_logger, request_type, response, model, kwargs, duration, accumulated_provider=None):
- provider = (
- accumulated_provider
- or _resolve_provider_from_response(response)
- or _resolve_provider_from_kwargs(kwargs)
- )
- # Serialize the response once and reuse it for attributes, events, and metrics —
- # model_dump() on a ModelResponse is the most expensive step in the wrapper.
- response_dict = _model_as_dict(response)
- _set_span_attribute(span, GenAIAttributes.GEN_AI_PROVIDER_NAME, provider)
- _handle_response(span, request_type, response_dict, event_logger)
- _record_metrics(metrics, provider, model, duration, response_dict, request_type)
- if span.is_recording():
- span.set_status(Status(StatusCode.OK))
- span.end()
+ try:
+ provider = (
+ accumulated_provider
+ or _resolve_provider_from_response(response)
+ or _resolve_provider_from_kwargs(kwargs)
+ )
+ # Serialize the response once and reuse it for attributes, events, and metrics —
+ # model_dump() on a ModelResponse is the most expensive step in the wrapper.
+ response_dict = _model_as_dict(response)
+ _set_span_attribute(span, GenAIAttributes.GEN_AI_PROVIDER_NAME, provider)
+ _handle_response(span, request_type, response_dict, event_logger)
+ _record_metrics(metrics, provider, model, duration, response_dict, request_type)
+ if span.is_recording():
+ span.set_status(Status(StatusCode.OK))
+ finally:
+ span.end()📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def _finalize(span, metrics, event_logger, request_type, response, model, kwargs, duration, accumulated_provider=None): | |
| provider = ( | |
| accumulated_provider | |
| or _resolve_provider_from_response(response) | |
| or _resolve_provider_from_kwargs(kwargs) | |
| ) | |
| # Serialize the response once and reuse it for attributes, events, and metrics — | |
| # model_dump() on a ModelResponse is the most expensive step in the wrapper. | |
| response_dict = _model_as_dict(response) | |
| _set_span_attribute(span, GenAIAttributes.GEN_AI_PROVIDER_NAME, provider) | |
| _handle_response(span, request_type, response_dict, event_logger) | |
| _record_metrics(metrics, provider, model, duration, response_dict, request_type) | |
| if span.is_recording(): | |
| span.set_status(Status(StatusCode.OK)) | |
| span.end() | |
| def _finalize(span, metrics, event_logger, request_type, response, model, kwargs, duration, accumulated_provider=None): | |
| try: | |
| provider = ( | |
| accumulated_provider | |
| or _resolve_provider_from_response(response) | |
| or _resolve_provider_from_kwargs(kwargs) | |
| ) | |
| # Serialize the response once and reuse it for attributes, events, and metrics — | |
| # model_dump() on a ModelResponse is the most expensive step in the wrapper. | |
| response_dict = _model_as_dict(response) | |
| _set_span_attribute(span, GenAIAttributes.GEN_AI_PROVIDER_NAME, provider) | |
| _handle_response(span, request_type, response_dict, event_logger) | |
| _record_metrics(metrics, provider, model, duration, response_dict, request_type) | |
| if span.is_recording(): | |
| span.set_status(Status(StatusCode.OK)) | |
| finally: | |
| span.end() |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/opentelemetry-instrumentation-litellm/opentelemetry/instrumentation/litellm/__init__.py`
around lines 583 - 597, The _finalize() path in the litellm instrumentation can
leak spans if _model_as_dict(), _handle_response(), or _record_metrics() raises
before span.end() is reached. Update _finalize() so the finalization work is
wrapped defensively and span.end() always executes, while preserving the
existing provider resolution, GenAIAttributes.GEN_AI_PROVIDER_NAME setting, and
Status(StatusCode.OK) behavior when the span is recording.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: TLP-2379
feat(instrumentation): ...orfix(instrumentation): ....Summary by CodeRabbit
New Features
Bug Fixes
Documentation