[FEATURE] RPA-backed trusted browser action connector for Gods Eye View

[FenjuFu]

Problem

Astron Agent can orchestrate browser/RPA work, but integrations commonly fall back to brittle DOM selectors or unrestricted page evaluation. Applications that expose a structured same-page action runner should be callable through a narrower, auditable contract.

bilawalsidhu/gods-eye-view is a concrete public-data fixture. Its voice agent and deterministic QA harness both use:

window.__gevVoiceCommands.runner(toolName, args)

The runner dispatches the same 28 bounded actions used by the OpenAI Realtime voice surface. Astron has no action allowlist, confirmation policy, timeout/cancellation, or honest-result adapter for this kind of browser surface.

Proposed Solution

Add an RPA-backed trusted browser action connector, with Gods Eye View as the first acceptance fixture:

1. Configure a localhost application URL and explicit action allowlist. Never accept arbitrary JavaScript from an Agent prompt.
2. Open/attach through the existing Astron RPA boundary, wait for the fixed runner, and invoke only runner(actionName, JSON_args).
3. Serialize calls per page and add bounded readiness/action timeouts, cancellation, and stale-page detection.
4. Preserve result honesty: exception, timeout, missing runner, page close, or { ok: false } must remain failures.
5. Separate read-only actions from state-changing actions. Require confirmation for camera/layer changes and anything that may activate metered providers.
6. Do not copy cookies, local storage, API keys, screenshots, browser globals, or arbitrary DOM content into model context.
7. Keep it local-first: no LAN exposure, unauthenticated HTTP bridge, generic remote eval, submodules, or hidden background control.
8. Audit action name, normalized args, duration, result status, target origin, and correlation ID with secret/large-payload redaction.

Initial Policy

Read-only candidates: get_current_view_state, get_entity_context, and bounded analyst_query.

Confirmation-required candidates: fly_to_location, zoom_to_globe, set_layer_visibility, set_map_stack, annotations, radio, scenes, tracking, and cockpit actions.

Named-person search, face recognition, private/paywalled data, and safety-critical operational decisions remain out of scope. Gods Eye View documents that its public data can be delayed, incomplete, modeled, inferred, or wrong.

Acceptance Fixture

Pin bilawalsidhu/gods-eye-view@314a0e1c2ef668cb110674b737e19a44ff6fc1ef on localhost.

• Wait for window.__gevVoiceCommands.runner without requiring an OpenAI voice session.
• get_current_view_state succeeds with structured JSON.
• Confirmed fly_to_location and set_layer_visibility calls preserve runner status.
• Non-allowlisted actions are rejected before page evaluation.
• Missing runner, timeout, page close, thrown exception, and { ok: false } remain failures.
• Calls are serialized; cancelled/superseded work cannot overwrite newer results.
• No LAN/public exposure or browser/provider credentials are required or leaked.

Why not MCP first?

Gods Eye View has no network MCP/HTTP action server. Its existing trusted automation seam is inside the browser and is already exercised by scripts/qa-voice-routing.mjs. RPA can reuse that boundary without adding a remote-control surface; a future MCP bridge can reuse the same policy.

Related

• https://github.com/bilawalsidhu/gods-eye-view
• Runner: https://github.com/bilawalsidhu/gods-eye-view/blob/314a0e1c2ef668cb110674b737e19a44ff6fc1ef/src/voice/gevActions.js
• Behavior QA: https://github.com/bilawalsidhu/gods-eye-view/blob/314a0e1c2ef668cb110674b737e19a44ff6fc1ef/scripts/qa-voice-routing.mjs
• Astron RPA orchestration request: ✨ Feature Request: 支持 OpenClaw 触发 Astron Agent 调度 RPA 机器人执行安全操作 #1067