You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Improve OTLP telemetry span names with dataset context #58
Improve the readability of OTLP evaluation spans by including dataset row context (input text) in span names and attributes.
Current Behavior
Item spans are named generically:
eval_item 1, eval_item 2, eval_item 3
This makes it hard to identify which dataset row a span corresponds to when viewing traces in AI Toolkit, Jaeger, or other collectors.
Desired Behavior
Item spans should include a truncated version of the input query:
eval_item 1: What is the weather in Seattle today?
eval_item 2: Convert 100 USD to EUR
And the input_text and expected_text attributes should be populated on item spans.
Implementation Notes
Thread dataset rows (or at minimum input_field values) from the runner to _emit_item_spans()
Pass input_text to eval_item_span() — the parameter already exists but is not used by the caller
Truncate input text in span name to ~80 chars to avoid excessively long span names
The eval_item_span() function in utils/telemetry.py already accepts input_text and expected_text — only the caller in services/runner.py needs updating
Files to Change
src/agentops/services/runner.py — pass dataset row data to _emit_item_spans
tests/unit/test_telemetry.py — update tests if span name format changes
Activity
feat: include dataset input text in OTLP eval_item span names (#58)