Skip to content

compat(agent): track built-in Harness Room-approval behavior after #286 #297

Closed

Description

@madawei2699

Context

#286 is complete and closed. Agent v0.5.24 now has a production-verified generic Room-native ACP approval path using the built-in Claude Harness:

real Claude tool request
→ ACP session/request_permission
→ production Room approval card
→ Human Allow / Deny
→ exact native option returned
→ same Claude turn continues or is denied

This follow-up does not reopen the Room approval architecture. It tracks truthful per-Harness behavior and public documentation after #286.

Current compatibility matrix

Harness Current status Product interpretation
Claude PRODUCTION VERIFIED Real Room approval works end-to-end.
Codex UPSTREAM LIMITATION Native Codex can prompt under untrusted, but tested codex-acp 1.6.2 and 1.10.0 did not expose that path under current ACP mode presets. Track upstream rather than patching Free4Chat core.
OpenCode ROOM APPROVAL VERIFIED OpenCode 1.18.18 with an isolated ask policy emitted real ACP session/request_permission; production Room card, ordinary yes no-op, Allow once, Reject, same-turn continuation, and post-denial recovery all passed.
Hermes NATIVE POLICY / NO CARD REQUIRED FOR TESTED PATH Hermes Agent v0.21.0 contains ACP permission relay code, but the tested harmless command executed directly under its native/default policy with no local prompt and no ACP permission request. This is valid Harness behavior.
Pi NATIVE/DEFAULT POLICY; NO ROOM APPROVAL REQUIRED IN CURRENT DOGFOOD Pi executed the tested actions without interactive approval. This is acceptable: Free4Chat does not require every Harness to emit ACP approval requests.
custom ACP implementation-defined If it emits standard ACP session/request_permission, the generic Free4Chat path should work.

Codex upstream references:

Immediate documentation follow-up

Update the public Agent permissions and approvals documentation so it states the shipped behavior truthfully:

  • Claude is production-verified for Room-native ACP approval;
  • other Harnesses may execute under their own native/default policy and therefore never show a Room card;
  • absence of a Room approval card is not automatically a Free4Chat failure;
  • Codex currently has a known upstream bridge/policy limitation;
  • Pi's current no-prompt/high-trust behavior is acceptable and does not need a fake approval layer;
  • OpenCode/Hermes remain compatibility evidence, not blockers for the generic feature.

Keep the docs user-oriented. Do not turn the public page into an implementation matrix dump.

Durable permission choices

The Claude production test also exposed an important presentation boundary:

Deny
Allow Once
Always Allow

A durable native option may have Harness-defined lifetime/storage semantics (session, project-local settings, user settings, etc.).

Free4Chat must not infer those semantics from the button label or kind. Public docs should say so explicitly.

The current generic feature remains correct because it forwards the exact native option id unchanged. A future UI improvement may display native scope/effect metadata when the Harness already provides a bounded safe presentation, but this issue does not require inventing a Free4Chat permission taxonomy.

Final probe results

OpenCode

OpenCode 1.18.18 was tested with opencode acp --pure and a disposable per-process OPENCODE_CONFIG_CONTENT policy setting bash, edit, and external_directory to ask.

A harmless command produced a real production Room approval card with the full command and native options Allow once / Always allow / Reject. Ordinary Human chat yes did not approve the pending request. Allow once resumed the same turn and produced the expected marker contents; Reject prevented a second marker from being created; a later ordinary turn succeeded.

Classification: ROOM APPROVAL VERIFIED.

Hermes

Hermes Agent v0.21.0 (2026.8.31) was verified to contain make_approval_callback and make_acp_edit_approval_requester ACP relay code. Under the tested default/native policy, a harmless shell write executed directly, produced the expected marker, and emitted neither a local approval prompt nor ACP session/request_permission.

Classification: NATIVE POLICY / NO ROOM APPROVAL REQUIRED FOR THE TESTED PATH.

Non-goals

  • changing ux/security(agent): Room-native ACP approval for resident Harnesses #286 Room approval semantics;
  • requiring every built-in Harness to ask before tool execution;
  • adding a Free4Chat universal permission DSL;
  • adding RBAC/owners/approval routing;
  • adding a CLI approval inbox;
  • patching/forking Codex ACP merely to force prompts;
  • weakening Pi/Hermes/OpenCode native policies to manufacture approval cards.

Acceptance

  • public permission docs reflect the production-verified Claude path;
  • docs explain that Harnesses may legitimately run under native/default policies without Room approval;
  • durable native choices are documented as Harness-defined in scope/effect;
  • OpenCode is classified by one narrow real probe;
  • Hermes is classified by one narrow real probe;
  • Codex remains tracked upstream unless a concrete Free4Chat-specific defect appears;
  • Pi remains acceptable under its observed native/default no-prompt behavior;
  • no changes are made to the generic Room approval architecture without new evidence.

Activity

  1. madawei2699 commented on Sep 8, 2026

    @madawei2699
    CollaboratorAuthor

    Final Harness compatibility probes — 2026-09-08

    OpenCode — ROOM APPROVAL VERIFIED

    • Version: OpenCode 1.18.18
    • Launcher: opencode acp --pure
    • Isolated policy: OPENCODE_CONFIG_CONTENT via process-local --agent-env, with bash, edit, and external_directory set to ask
    • Real production ACP session/request_permission: observed
    • Room approval card: observed with full command and native options
    • Ordinary Human chat yes: did not approve
    • Allow once: same turn resumed and marker contents matched
    • Reject: denied marker remained absent
    • Post-denial ordinary turn: healthy

    No persistent/global OpenCode config was changed.

    Hermes — NATIVE POLICY / NO CARD REQUIRED FOR TESTED PATH

    • Version: Hermes Agent v0.21.0 (2026.8.31)
    • Launcher: hermes acp
    • Installed ACP relay code: present (make_approval_callback, make_acp_edit_approval_requester)
    • Tested harmless shell write: executed and verified
    • Local native prompt: none
    • ACP session/request_permission: none
    • Room approval card: none

    This is valid Harness-native/default policy behavior, not a Free4Chat approval failure.

    Final matrix

    Harness Final status
    Claude ROOM APPROVAL VERIFIED
    OpenCode ROOM APPROVAL VERIFIED
    Hermes NATIVE POLICY / NO CARD REQUIRED FOR TESTED PATH
    Codex UPSTREAM LIMITATION
    Pi NATIVE POLICY / NO CARD REQUIRED

    No Free4Chat code change is required from these probes. #297 acceptance is complete.

  2. madawei2699 commented on Sep 20, 2026

    @madawei2699
    CollaboratorAuthor

    Historical matrix follow-up: #426 now contains the current-version re-verification. codex-acp 1.12.0 / Codex 0.154.0 passed the file-change Runtime/Room decision path, but the ordinary command path is still partial because Runtime rejects a 175-character native option label (#429). The 1.6.2 pin emits a command permission without a title/presentation metadata. #297 remains closed.

  3. madawei2699 commented on Sep 20, 2026

    @madawei2699
    CollaboratorAuthor

    Historical matrix update — 2026-09-20:

    Codex is no longer accurately described as a blanket upstream Room-approval limitation.

    #426 re-verified released @agentclientprotocol/codex-acp@1.12.0 / Codex 0.154.0 through a REAL browser Room on current Free4Chat:

    The production pin was updated by merged PR #432 (c6ec2ab812167f1e9299ad0e3ff3f6c950de9a6d).

    Classification remains deliberately PARTIAL, not universal: openai/codex#21982, codex-acp#310, and codex-acp#401 still cover sandbox/mode/MCP paths not certified by this probe. Codex remains SERIAL and Task Session Continuation remains disabled.

    Public permission docs were updated with the same scoped wording.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions