docs(agent-tracing): document evaluation spans for JS and Python by ArthurKnaus · Pull Request #19825 · getsentry/sentry-docs

ArthurKnaus · 2026-10-06T12:25:22Z

DESCRIBE YOUR PR

Documents evaluation (gen_ai.evaluate) spans in the agent tracing manual instrumentation guides, so users whose evaluation library isn't auto-instrumented can report these spans themselves. Follows the structure of the existing span sections on each page.

Add an "Evaluation Span" section to the JavaScript manual instrumentation page
Add the same section to the Python manual instrumentation page
Add a shared evaluation-span attributes include used by both pages
Cover the input (state + questions) and output (answers) shapes, the boolean/noul/score/choice answer types, and the two criteria shapes
Point users to the TypeSafe integrations and Vercel AI SDK's experimental_evaluate() for automatic instrumentation

Closes TET-3057
Closes TET-3058

IS YOUR CHANGE URGENT?

Help us prioritize incoming PRs by letting us know when the change needs to go live.
Select exactly one option. For deadlines, replace YYYY-MM-DD with the due date. You can update this information later by editing the PR description.

Urgent deadline (GA date, etc.): YYYY-MM-DD
Other deadline: YYYY-MM-DD
No deadline: Not urgent, can wait up to 1 week+

SLA

Teamwork makes the dream work, so please add a reviewer to your PRs.
Please give the docs team up to 1 week to review your PR unless you've supplied a deadline.

Thanks in advance for your help!

PRE-MERGE CHECKLIST

Make sure you've checked the following before merging your changes:

Checked Vercel preview for correctness, including links
PR was reviewed and approved by any necessary SMEs (subject matter experts)
PR was reviewed and approved by a member of the Sentry docs team

🤖 Generated with Claude Code

Add an Evaluation Span section to the JS and Python manual instrumentation pages covering gen_ai.evaluate spans (e.g. jev), plus a shared attributes include. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

vercel · 2026-10-06T12:25:25Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
sentry-docs	Ready	Preview	Oct 6, 2026 12:32pm UTC

1 Skipped Deployment

Project	Deployment	Actions	Updated
develop-docs	Ignored		Oct 6, 2026 12:32pm UTC

linear-code · 2026-10-06T12:25:31Z

TET-3058

TET-3057

shellmayr · 2026-10-06T12:27:52Z

+      "gen_ai.response.model": result.model, // e.g. "typesafe/jev-1.13-20260917"
+      "gen_ai.output.messages": JSON.stringify([
+        {
+          type: "evaluation",


Doesn't this conflict with the whole evaluate vs evaluation thing?

yeah, it is a bit confusing...
getsentry/sentry-conventions#650 added the conventions

cursor · 2026-10-06T12:49:07Z

The plan checks the new heading on the changed pages. It also checks client errors on those pages and on the catch-all docs route. An issue escalates when page errors appear or catch-all errors rise above the measured band.

Monitoring in production

Services: sentry-docs.

Mention @change-monitor in a comment to update the plan.

Plan

What changed

The JavaScript and Python manual instrumentation pages now teach evaluation spans. Those pages render a new Evaluation Span section with example code and attributes. This is live after sentry-docs deploys.

Risk

A broken Include or MDX block can fail those pages. Shared chrome such as code samples and expanders can throw on the catch-all route. JavaScript common content also ships to every JavaScript guide URL.

Intended effect

This is a docs edit. There is no Sentry counter for the new heading. Confirmed means the live pages contain the heading Evaluation Span. Absent means that heading is missing while the pages still load.

Signal	Baseline	Window	Source	Rule
Evaluation Span heading	absent on live pages	2026-10-05T12:49:00Z to 2026-10-06T12:49:00Z	live HTML on `/platforms/python/agent-tracing/manual-instrumentation/` and any `/platforms/javascript/guides/*/agent-tracing/manual-instrumentation/` page	Hold absent until deploy. Confirmed when the heading is present.

Regression watch

A bad MDX render can throw on the changed pages or in shared page chrome.

Signal	Baseline	Window	Source	Rule
Client errors on manual instrumentation pages	0 events	2026-10-05T12:49:00Z to 2026-10-06T12:49:00Z	Sentry org `sentry` project `docs`, `environment:production AND transaction:agent-tracing/manual-instrumentation`	Hold at zero. Detected if any error appears.
Catch-all page errors	0-29 events per hour, 137 events in 24h	2026-10-05T12:00:00Z to 2026-10-06T12:00:00Z	Sentry org `sentry` project `docs`, `environment:production transaction:/:path*`	Hold inside 0-29 events per hour. Detected if an hour exceeds 29 or the 24h count rises far above 137.

Page-specific pageload volume is too sparse to use as a watch. Sampled pageload and navigation on url:*agent-tracing/manual-instrumentation* were 6-7 in 24h. Most hours were 0. A drop to zero is normal here.

Not observable

Python-only URL views were 0 in this window, so Python traffic cannot be split from JavaScript. HTTP status codes for these URLs were not queried. Traces are sampled at 30 percent, so page-view counts are not full traffic.

cursor · 2026-10-08T11:28:53Z

Issue detected in production

Live docs omit the Evaluation Span heading after this ship

The live Python and JavaScript pages load without the Evaluation Span heading.

) ## DESCRIBE YOUR PR Documents evaluation (`gen_ai.evaluate`) spans in the agent tracing manual instrumentation guides, so users whose evaluation library isn't auto-instrumented can report these spans themselves. Follows the structure of the existing span sections on each page. - Add an "Evaluation Span" section to the JavaScript manual instrumentation page - Add the same section to the Python manual instrumentation page - Add a shared `evaluation-span` attributes include used by both pages - Cover the input (`state` + `questions`) and output (`answers`) shapes, the `boolean`/`noul`/`score`/`choice` answer types, and the two `criteria` shapes - Point users to the TypeSafe integrations and Vercel AI SDK's `experimental_evaluate()` for automatic instrumentation Closes TET-3057 Closes TET-3058 ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. Select exactly one option. For deadlines, replace `YYYY-MM-DD` with the due date. You can update this information later by editing the PR description. - [ ] Urgent deadline (GA date, etc.): YYYY-MM-DD - [ ] Other deadline: YYYY-MM-DD - [x] No deadline: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've supplied a deadline. Thanks in advance for your help! ## PRE-MERGE CHECKLIST _Make sure you've checked the following before merging your changes:_ - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

cursor · 2026-10-09T10:27:55Z

Verified in production

The live pages now show the Evaluation Span heading.
The live pages still show the Evaluation Span heading.

docs(agent-tracing): document evaluation spans for JS and Python

0ac3fab

Add an Evaluation Span section to the JS and Python manual instrumentation pages covering gen_ai.evaluate spans (e.g. jev), plus a shared attributes include. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

github-actions Bot added the Priority: Normal Docs review has no urgent deadline label Oct 6, 2026

codeowner-assignment Bot requested a review from a team October 6, 2026 12:25

ArthurKnaus requested a review from a team October 6, 2026 12:26

shellmayr reviewed Oct 6, 2026

View reviewed changes

vercel Bot deployed to Preview – sentry-docs October 6, 2026 12:32

View deployment

ArthurKnaus requested a review from a team October 6, 2026 12:48

ArthurKnaus marked this pull request as ready for review October 6, 2026 12:48

codeowner-assignment Bot requested a review from a team October 6, 2026 12:48

obostjancic approved these changes Oct 8, 2026

View reviewed changes

ArthurKnaus merged commit 18c1f54 into master Oct 8, 2026
27 checks passed

ArthurKnaus deleted the arthurknaus/tet-3057-agent-evaluation-spans branch October 8, 2026 10:16

Navigation Menu

Repository navigation

docs(agent-tracing): document evaluation spans for JS and Python - #19825

docs(agent-tracing): document evaluation spans for JS and Python#19825
ArthurKnaus merged 1 commit into
masterfrom
arthurknaus/tet-3057-agent-evaluation-spans

ArthurKnaus commented Oct 6, 2026

Uh oh!

vercel Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

linear-code Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

shellmayr Oct 6, 2026

Uh oh!

ArthurKnaus Oct 6, 2026

Uh oh!

cursor Bot commented Oct 6, 2026 •

edited

Loading

What changed

Risk

Intended effect

Regression watch

Not observable

Uh oh!

Uh oh!

cursor Bot commented Oct 8, 2026

Uh oh!

cursor Bot commented Oct 9, 2026 •

edited

Loading

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ArthurKnaus commented Oct 6, 2026

DESCRIBE YOUR PR

IS YOUR CHANGE URGENT?

SLA

PRE-MERGE CHECKLIST

Uh oh!

vercel Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linear-code Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shellmayr Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

ArthurKnaus Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Risk

Intended effect

Regression watch

Not observable

Uh oh!

Uh oh!

cursor Bot commented Oct 8, 2026

Uh oh!

cursor Bot commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vercel Bot commented Oct 6, 2026 •

edited

Loading

linear-code Bot commented Oct 6, 2026 •

edited

Loading

cursor Bot commented Oct 6, 2026 •

edited

Loading

cursor Bot commented Oct 9, 2026 •

edited

Loading