Skip to content

feat(browser): let the agent read the user's own Browser tab - #2287

Merged
chuks-qua merged 3 commits into
mainfrom
feat/issue-2219
Oct 23, 2026
Merged

chuks-qua merged 3 commits into
mainfrom
feat/issue-2219

Conversation

@chuks-qua

@chuks-qua chuks-qua commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

What

Before, browser_inspect read only the agent's current tab. While the user held any tab, browser_tabs select was refused, so the agent could not read a user tab that wasn't already its current tab. The inspect tab listing carried no holder, and readiness looked only at the current tab.

Now:

  • browser_inspect takes an optional tabId and reads any Browser tab in the Thread, including one the user holds. A named read never claims control and never moves the agent's current tab.
  • The inspect listing shows every tab in the Thread as { tabId, active, focused, holder }. Holder is user, agent, or none. A user hold on any tab shows on every tab, matching the thread-wide hold in ADR-0029.
  • Readiness reports human-control and the guidance says "do not act" while the user holds any tab in the Thread, because that hold blocks every effect.
  • An act or evaluate whose observation came from a different tab is refused. Tabs count revisions separately, so two tabs could share them. Without this check, refs read from the user's tab could run against the agent's tab.
browser_inspect { tabId }
  broker.resolveExecutionAssignment
    inspect + tabId -> resolveExplicitTarget   # sticky assignment untouched
    unknown tabId   -> TAB_UNAVAILABLE, recovery inspect
    busy host       -> TAB_UNAVAILABLE, retryable
  host inspects dispatch.target                # no host change needed
  broker.shapeNegotiatedResponse
    tabs     <- holder the broker enforces (host.targets)
    readiness/guidance <- user hold anywhere in the Thread
  toolkit.projectTabs -> { tabId, active, focused, holder }

Acceptance criteria, each covered by broker.user-tab.test.ts:

Criterion Test
User tab, no agent tab: "what's on this page" returns its content reads the user's tab when the agent has no tab of its own
Reading a held tab succeeds and keeps the holder keeps the user's hold after the agent reads the held tab
An act on a held tab is refused, then succeeds after Hand back refuses an action on the user's held tab and admits it after Hand back
The listing shows user tabs with their holder lists every thread tab with the holder its host published, plus the toolkit holder test

Verification:

  • broker, toolkit, contract, and web automation suites: 220 server tests, 58 contract tests, and 222 web automation tests pass.
  • tsc --noEmit is clean in contracts, server, and web.
  • bun run lint exits 0. The only warnings are 2 pre-existing ones in untouched scripts.

Why

Users ask about "this page" while a tab they opened is in front of them. The agent should be able to read it without taking it from them (#2219, part of #2206).

Review Notes

  • Waiting on a held tab is not added. The issue says the agent can "wait for state" on the user's tab. wait and assert are browser_act steps, and ADR-0029 refuses every effect while the user holds. Repeated browser_inspect is the read-only way to watch state. Adding a read-only wait would need its own operation.
  • Spec vs ADR conflict. The issue says that while the user holds, the agent "reads it or opens its own tab". Under ADR-0029 the hold covers the whole Thread, and the broker refuses browser_open with HUMAN_INTERRUPTED (verified). This PR follows the ADR. Allowing open during a hold would need an ADR amendment.
  • Hosts disagree on holders. Electron marks every tab in a held Thread as human, and the web host marks only the held tab. The toolkit projection normalizes this for the model. Making both hosts publish the same shape is a possible follow-up.
  • Reading another tab and then acting costs one STALE_TARGET_GENERATION refusal unless the agent first selects that tab and inspects it again. The tool description says so.
  • No live end-to-end provider run. Coverage is at the broker, toolkit, and driver seams.
  • Docs: none needed. docs/internals/runtime/browser-v2-rollout.md stays accurate.

Closes #2219


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…ervation

Observation revisions count per tab, so two tabs can share them. An
observation from one tab could then pass the currency check on another
and run its refs there. Bind act and evaluate observations to the tab
they were taken from.
browser_inspect takes an optional tabId to read any tab in the thread,
including one the user holds. A named read never claims control and
never moves the agent's current tab.

The inspect tab listing now shows who holds each tab, taken from the
holder the broker enforces. Readiness and guidance reflect a user hold
anywhere in the thread, since that hold blocks every effect.

Closes #2219
…t from missing ones

The Electron host marks every tab in a held thread as held by the user,
while the web host marks only the tab the user took. browser_inspect now
lists each tab as { tabId, active, focused, holder } and reports a user
hold on any tab for every tab, which matches the thread-wide hold the
broker enforces. The listing also drops routing fields and control epochs
the model never uses.

A named read of a listed tab on a busy host now fails as retryable
instead of claiming the tab is not open.
@chuks-qua
chuks-qua merged commit 4720061 into main Oct 23, 2026
11 checks passed
@chuks-qua
chuks-qua deleted the feat/issue-2219 branch October 23, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: let the agent read the user's own Browser tab

1 participant