Skip to content

Improve OTLP telemetry span names with dataset context #58

Closed

Description

Objective

Improve the readability of OTLP evaluation spans by including dataset row context (input text) in span names and attributes.

Current Behavior

Item spans are named generically:

  • eval_item 1, eval_item 2, eval_item 3

This makes it hard to identify which dataset row a span corresponds to when viewing traces in AI Toolkit, Jaeger, or other collectors.

Desired Behavior

Item spans should include a truncated version of the input query:

  • eval_item 1: What is the weather in Seattle today?
  • eval_item 2: Convert 100 USD to EUR

And the input_text and expected_text attributes should be populated on item spans.

Implementation Notes

  • Thread dataset rows (or at minimum input_field values) from the runner to _emit_item_spans()
  • Pass input_text to eval_item_span() — the parameter already exists but is not used by the caller
  • Truncate input text in span name to ~80 chars to avoid excessively long span names
  • The eval_item_span() function in utils/telemetry.py already accepts input_text and expected_text — only the caller in services/runner.py needs updating

Files to Change

  • src/agentops/services/runner.py — pass dataset row data to _emit_item_spans
  • tests/unit/test_telemetry.py — update tests if span name format changes

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions