docs(agent-tracing): document evaluation spans for JS and Python

docs/platforms/javascript/common/agent-tracing/manual-instrumentation.mdx

@@ -287,6 +287,102 @@ Marking failed tools with an error status populates the Tool Errors widget.
   <Include name="tracing/ai-agents-module/execute-tool-span" />
 </Expandable>
 
+## Evaluation Span
+
+This span represents an evaluation of generated content against a set of questions, such as grading a model's reply for accuracy or tone. Models like <PlatformLink to="/configuration/integrations/typesafe/">TypeSafe Jev</PlatformLink> produce these spans.
+
+Sentry identifies an evaluation by `gen_ai.operation.name: "evaluate"`. Evaluations share the `ai_client` operation type with ordinary LLM calls, so the operation name is what sets them apart.
+
+```javascript
+const questions = {
+  resolves_issue: {
+    type: "boolean",
+    instructions: "Does the reply resolve the customer's issue?",
+  },
+  accuracy: {
+    type: "score",
+    instructions: "Does the reply correctly answer the user's question?",
+    // For a score question, criteria is an ordered array; the index is the score.
+    criteria: ["incorrect", "partially correct", "correct"],
+  },
+  tone: {
+    type: "choice",
+    instructions: "Classify the tone of the reply.",
+    // For a choice question, criteria maps each label to its description.
+    criteria: {
+      friendly: "Warm and helpful",
+      neutral: "Plain and factual",
+      rude: "Dismissive or hostile",
+    },
+  },
+};
+
+const evaluationInput = [
+  {
+    type: "evaluation",
+    state: "Your refund has been processed and will arrive in 3-5 days.",
+    questions,
+  },
+];
+
+await Sentry.startSpan(
+  {
+    op: "gen_ai.evaluate",
+    name: "evaluate typesafe/jev-1.13",
+    attributes: {
+      "gen_ai.operation.name": "evaluate",
+      "gen_ai.request.model": "typesafe/jev-1.13",
+      "gen_ai.provider.name": "typesafe",
+      "gen_ai.agent.name": "Support Agent", // when this evaluation runs under an agent
+      "gen_ai.input.messages": JSON.stringify(evaluationInput),
+    },
+  },
+  async (span) => {
+    // Call your evaluator; map its response into span attributes below
+    const result = await yourEvaluator.evaluate({
+      state: evaluationInput[0].state,
+      questions,
+    });
+
+    span.setAttributes({
+      "gen_ai.response.model": result.model, // e.g. "typesafe/jev-1.13-20260917"
+      "gen_ai.output.messages": JSON.stringify([
+        {
+          type: "evaluation",
+          answers: {
+            resolves_issue: { type: "boolean", probability: 0.98 },
+            accuracy: { type: "score", score: 2, confidence: 0.91 },
+            tone: {
+              type: "choice",
+              choice: "friendly",
+              probabilities: { friendly: 0.87, neutral: 0.1, rude: 0.03 },
+              confidence: 0.87,
+            },
+          },
+        },
+      ]),
+      "gen_ai.usage.input_tokens": result.usage.inputTokens,
+      "gen_ai.usage.output_tokens": result.usage.outputTokens,
+    });
+    return result;
+  }
+);
+```
+
+The input is a single `{type: "evaluation", state, questions}` object. `state` is the text being evaluated, and each entry in `questions` describes one question. The output is a single `{type: "evaluation", answers}` object keyed by those same question names. Answers come in a few types (`boolean`, `noul`, `score`, and `choice`), each with its own fields. See the attributes below for the full shape.
+
+Pass the raw provider model string in `gen_ai.request.model` (and `gen_ai.response.model`) so Sentry can resolve the model icon and any [model costs](/product/agents/costs/).
+
+<Alert>
+
+If you call Jev through the <PlatformLink to="/configuration/integrations/typesafe/">`@typesafe-ai/sdk`</PlatformLink> package or Vercel AI SDK's <PlatformLink to="/configuration/integrations/vercelai/#trace-jev-evaluations">`experimental_evaluate()`</PlatformLink>, these spans are created automatically, so you don't need to instrument them by hand.
+
+</Alert>
+
+<Expandable title="Evaluation span attributes">
+  <Include name="tracing/ai-agents-module/evaluation-span" />
+</Expandable>
+
 ## Streaming Responses
 
 When the model streams tokens, keep the span open until the stream finishes (including when you yield chunks to the client). Set response attributes when you have the final usage and text:

docs/platforms/python/agent-tracing/manual-instrumentation.mdx

@@ -261,6 +261,94 @@ Marking failed tools with an error status populates the Tool Errors widget.
   <Include name="tracing/ai-agents-module/execute-tool-span" />
 </Expandable>
 
+## Evaluation Span
+
+This span represents an evaluation of generated content against a set of questions, such as grading a model's reply for accuracy or tone. Models like <PlatformLink to="/agent-tracing/typesafe/">TypeSafe Jev</PlatformLink> produce these spans.
+
+Sentry identifies an evaluation by `gen_ai.operation.name: "evaluate"`. Evaluations share the `ai_client` operation type with ordinary LLM calls, so the operation name is what sets them apart.
+
+```python
+import json
+import sentry_sdk
+
+questions = {
+    "resolves_issue": {
+        "type": "boolean",
+        "instructions": "Does the reply resolve the customer's issue?",
+    },
+    "accuracy": {
+        "type": "score",
+        "instructions": "Does the reply correctly answer the user's question?",
+        # For a score question, criteria is an ordered list; the index is the score.
+        "criteria": ["incorrect", "partially correct", "correct"],
+    },
+    "tone": {
+        "type": "choice",
+        "instructions": "Classify the tone of the reply.",
+        # For a choice question, criteria maps each label to its description.
+        "criteria": {
+            "friendly": "Warm and helpful",
+            "neutral": "Plain and factual",
+            "rude": "Dismissive or hostile",
+        },
+    },
+}
+
+evaluation_input = [
+    {
+        "type": "evaluation",
+        "state": "Your refund has been processed and will arrive in 3-5 days.",
+        "questions": questions,
+    }
+]
+
+with sentry_sdk.start_span(
+    op="gen_ai.evaluate",
+    name="evaluate typesafe/jev-1.13",
+    attributes={
+        "gen_ai.operation.name": "evaluate",
+        "gen_ai.request.model": "typesafe/jev-1.13",
+        "gen_ai.provider.name": "typesafe",
+        "gen_ai.agent.name": "Support Agent",  # when this evaluation runs under an agent
+        "gen_ai.input.messages": json.dumps(evaluation_input),
+    },
+) as span:
+    result = your_evaluator.evaluate(state=evaluation_input[0]["state"], questions=questions)
+
+    span.set_data("gen_ai.response.model", result.model)  # e.g. "typesafe/jev-1.13-20260917"
+    span.set_data("gen_ai.output.messages", json.dumps([
+        {
+            "type": "evaluation",
+            "answers": {
+                "resolves_issue": {"type": "boolean", "probability": 0.98},
+                "accuracy": {"type": "score", "score": 2, "confidence": 0.91},
+                "tone": {
+                    "type": "choice",
+                    "choice": "friendly",
+                    "probabilities": {"friendly": 0.87, "neutral": 0.1, "rude": 0.03},
+                    "confidence": 0.87,
+                },
+            },
+        }
+    ]))
+    span.set_data("gen_ai.usage.input_tokens", result.usage.input_tokens)
+    span.set_data("gen_ai.usage.output_tokens", result.usage.output_tokens)
+```
+
+The input is a single `{"type": "evaluation", "state": ..., "questions": ...}` object. `state` is the text being evaluated, and each entry in `questions` describes one question. The output is a single `{"type": "evaluation", "answers": ...}` object keyed by those same question names. Answers come in a few types (`boolean`, `noul`, `score`, and `choice`), each with its own fields. See the attributes below for the full shape.
+
+Pass the raw provider model string in `gen_ai.request.model` (and `gen_ai.response.model`) so Sentry can resolve the model icon and any [model costs](/product/agents/costs/).
+
+<Alert>
+
+If you call Jev through the <PlatformLink to="/agent-tracing/typesafe/">TypeSafe Python SDK</PlatformLink>, these spans are created automatically, so you don't need to instrument them by hand.
+
+</Alert>
+
+<Expandable title="Evaluation span attributes">
+  <Include name="tracing/ai-agents-module/evaluation-span" />
+</Expandable>
+
 ## Streaming Responses
 
 When the model streams tokens, keep the span open until the stream finishes (including when you yield chunks to the client). Set response attributes when you have the final usage and text:

includes/tracing/ai-agents-module/evaluation-span.mdx

@@ -0,0 +1,31 @@
+Describes an evaluation of generated content against a set of questions (for example a TypeSafe Jev evaluation).
+
+- The span `op` (transaction mode) or the span's `sentry.op` attribute (stream mode) MUST be `"gen_ai.evaluate"`.
+- The span `name` SHOULD be `"evaluate {gen_ai.request.model}"`. (e.g. `"evaluate typesafe/jev-1.13"`)
+- The `gen_ai.operation.name` attribute MUST be `"evaluate"`. Sentry uses this attribute, not the span `op`, to tell evaluations apart from ordinary LLM calls (both share the `ai_client` operation type).
+- The `gen_ai.request.model` attribute SHOULD be the requested evaluation model. (e.g. `"typesafe/jev-1.13"`)
+- The `gen_ai.response.model` attribute SHOULD be the concrete model that responded. (e.g. `"typesafe/jev-1.13-20260917"`)
+- The `gen_ai.provider.name` attribute SHOULD be set to the provider. (e.g. `"typesafe"`)
+- The `gen_ai.agent.name` attribute SHOULD be set to the agent's name when the evaluation runs under an agent. (e.g. `"Support Agent"`)
+
+Additional attributes on the span:
+
+| Data Attribute               | Type   | Requirement Level | Description                                                                                                   | Example                                                                                                                          |
+| :--------------------------- | :----- | :---------------- | :------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------------------------------------------------------------------- |
+| `gen_ai.input.messages`      | string | optional          | A single evaluation object holding the evaluated `state` and the `questions` (stringified JSON). **[0]**      | `'[{"type": "evaluation", "state": "...", "questions": {"accuracy": {"type": "score", "criteria": ["incorrect", "correct"]}}}]'` |
+| `gen_ai.output.messages`     | string | optional          | A single evaluation object holding the `answers`, keyed by the same question keys (stringified JSON). **[1]** | `'[{"type": "evaluation", "answers": {"accuracy": {"type": "score", "score": 1}}}]'`                                             |
+| `gen_ai.usage.input_tokens`  | int    | optional          | The number of tokens used in the evaluation input.                                                            | `154`                                                                                                                            |
+| `gen_ai.usage.output_tokens` | int    | optional          | The number of tokens used in the evaluation output.                                                           | `28`                                                                                                                             |
+
+**[0]** Each entry in `questions` has an `instructions` string and a `type`. For a `score` question, `criteria` is an ordered string array whose index is the score (`["low", "medium", "high"]`). For a `choice` question, `criteria` maps each label to its description (`{"positive": "...", "negative": "..."}`).
+
+**[1]** Each answer carries a `type` that matches its question:
+
+| Answer `type` | Fields                                                                              |
+| :------------ | :---------------------------------------------------------------------------------- |
+| `boolean`     | `probability`                                                                       |
+| `noul`        | `noul` holds the probability (Jev's variant of `boolean`).                          |
+| `score`       | `score`, plus optional `legend` (`{"0": "low"}`), `probabilities`, and `confidence` |
+| `choice`      | `choice`, plus optional `probabilities` and `confidence`                            |
+
+Optional fields may be omitted, but a field present with the wrong type makes that question or answer unreadable and it's shown raw in the span detail.