Repository navigation
test: stabilize e2e approval and cancellation tests - #624
Merged
Merged
Conversation
Switch the default e2e model to gpt-6-luna: gpt-5.2 is no longer in the Codex catalog, so it ran on fallback metadata without the apply_patch tool and edited files through the shell instead. Ask for escalated permissions up front in the shell approval prompts so the model does not run the command sandboxed and skip the approval.
Reuse the escalation hint for read-only shell tests that expect an approval, and wait for the execute tool call before cancelling so the interrupt does not race Codex registering the process.
Codex keeps a running unified exec command as a background terminal when the turn is interrupted. Assert that cancel ends the turn, then stop the command with _session/async_task/stop and check it exits.
The SDK client rejects the async_task_spawned update, so the test cannot read the task id from it. A root command's async task id is its tool call id, so retry the stop request with it until the task is published.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The E2E run on #620 (run) failed on two model-dependent tests. #620 didn't cause it: the run before it shows the same behavior and passed only by luck. This PR changes test code only.
Default e2e model
gpt-5.2→gpt-6-luna[low].gpt-5.2isn't in the Codex model list anymore, so Codex used fallback metadata. In 0.160.1 that hasapply_patch_tool_type: None, so the model ranapply_patchthrough the shell. A malformed patch came in as anexecuteapproval, notedit, and brokeapplies an approved workspace file edit.Shell approval prompts ask for elevated permissions on the first try. In read-only mode the model sometimes ran the command in the sandbox, hit
Read-only file system, and ended the turn without asking for approval (cancels every command when cancel is selectedgotend_turn). This applies only to the tests that expect an approval prompt.Cancellation test follows how Codex handles interrupts.
gpt-6-lunaruns commands through unified exec, andturn/interruptdoesn't kill those processes: they keep running as background terminals. The test now:cancelledand the process is still running;_session/async_task/stopand asserts it exits.The ACP SDK client rejects the
async_task_spawnedupdate because its schema doesn't include it, so the test uses the tool call id as the task id. For a root command they're the same.Fixture helpers: the fixture records tool calls, and
createAuthenticatedFixturecan declare AIR capabilities.Test plan
npx tsc --noEmit