Skip to content

fix(tool): record locally executed tools in OpenCode V2 - #134

Open
pedropombeiro wants to merge 1 commit into
DEVtheOPS:mainfrom
pedropombeiro:fix/local-tool-telemetry
Open

pedropombeiro wants to merge 1 commit into
DEVtheOPS:mainfrom
pedropombeiro:fix/local-tool-telemetry

Conversation

@pedropombeiro

Copy link
Copy Markdown

Description

With OpenCode V2, the plugin drops telemetry for every tool that OpenCode runs locally, such as shell, read, edit, MCP tools, Code Mode, and subagents:

  • opencode.tool.duration is never recorded.
  • No opencode.tool.* spans are created.
  • opencode.commit.count and the commit log event never fire.
  • Every tool_result log event reports duration_ms: 0.

Root cause: In V2 tool events, executed means the model provider ran the tool, not that OpenCode ran it. OpenCode publishes session.tool.called with executed: tool.providerExecuted (publish-llm-event.ts#L458-L476). It then publishes the terminal event for a locally executed tool with the same false value (#L573-L591). Only provider-hosted tools, such as OpenAI web_search or Anthropic code_execution, report executed: true (#L477-L513). handleToolCalled and finishTool gated spans, durations, and commit detection on executed, so the plugin measured only hosted tools.

Fix: A tool counts as executed once the plugin observes session.tool.called. OpenCode starts a local execution only after that event is published (step.ts#L100-L128). Calls that never ran don't publish it:

Neither case is recorded as an execution.

  • Failures after the call: A tool that fails after session.tool.called, including a permission rejection or user decline (step.ts#L199-L207), is recorded as a failed execution (success=false). Its duration includes any permission wait.
  • Provider execution: The flag is now the tool.provider_executed span attribute. It accepts both the current top-level executed and the nested provider.executed shape, which OpenCode dev uses (session-event.ts#L312-L372). That covers only the payload shape: dev also renames the events to session.next.tool.*, which this PR doesn't address.
  • Subagents: Subagent dispatch spans now start at session.tool.called instead of at the first progress event that carries a child session ID. Child-session linking is unchanged.

Tests now cover local success and failure, local git commit detection, permission rejection, malformed input without a call, provider-hosted success and failure, the nested provider.executed shape, and a subagent that fails before creating a child. Two existing tests encoded the old assumption that executed: false meant the tool didn't run, so I updated them.

I also checked it end to end with OpenCode 2.0.20 and a local OTLP/HTTP JSON sink. A read call produced a 33 ms span, histogram sample, and duration_ms. A shell call running sleep 2 recorded 2,834 ms. A git commit through shell incremented opencode.commit.count and emitted the commit log event.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Chore (dependency updates, etc.)

Checklist

  • I have read the CONTRIBUTING.md document
  • My code follows the style guidelines of this project
  • bun run lint passes with no errors
  • bun run check:jsdoc-coverage passes with no errors
  • bun run typecheck passes with no errors
  • bun test passes with no errors
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the documentation accordingly
  • My commits follow the Conventional Commits specification

Related issues

Independent of #112, which covers V1 message.part.updated timestamp restamping and orphan spans. This PR fixes V2 tool events that were never measured.

Additional context

The executed gating was introduced in #132, which treated executed: false calls as invalid. That holds for malformed calls but also matches every successful local call.

In OpenCode V2 tool events, `executed` means the model provider ran the
tool. Locally executed tools always report `executed: false`, so the
plugin skipped their spans, duration histogram, and commit detection, and
reported `duration_ms: 0` in `tool_result`.

Treat a tool as executed when the plugin observes `session.tool.called`.
Malformed-input rejections and calls cancelled before publication never
emit that event, so they are still not recorded as executions. Record
provider execution as the `tool.provider_executed` span attribute, and
accept both the top-level `executed` field and the nested
`provider.executed` shape.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant