Repository navigation
Conversation
A custom Handoff can pick the receiving agent when it is invoked: the
upstream Runner hands off to the agent its callback returns. convertAgent
cloned each Handoff with only a new agent, and Handoff.clone then returns
that agent after running the callback, so TemporalOpenAIRunner always
handed off to the Handoff's agent field and dropped the callback's choice.
The cloned Handoff now converts the agent the callback returns, through the
same seen-map. When the callback returns its own agent nothing changes.
Runs that already handed off to the agent field keep doing so on replay:
the new path is behind patched('openai-agents.handoff-returned-agent'),
which is only asked when the callback returns another agent.
…sandbox config An agent first reached through a custom Handoff's callback is converted when the handoff runs. The runner set runConfig.sandbox on agentAsTool tools only over the initial graph, so such an agent's agentAsTool tools never got it, and a nested SandboxAgent failed with "runConfig.sandbox is not configured" although the run had a valid Temporal sandbox client. Move that step into convertAgent, which now takes runConfig.sandbox and passes it to each converted agent's agentAsTool tools, so agents converted at handoff time get it too. Before converting such an agent, the callback path also runs the runner's missing-sandbox check, now shared as requireSandboxRunConfig, so a callback that returns a SandboxAgent without runConfig.sandbox fails with SandboxConfigurationError.
Contributor
Author
|
Replaced by #270. It runs agents as written instead of cloning them, so a custom Handoff hands off to the agent its callback returns without the conversion code here. This PR's tests are carried over there and pass, with agentAsTool replaced by agent.asTool(). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was changed
TemporalOpenAIRunnernow hands off to the agent a customHandoff's callback returns, as the upstreamRunnerdoes.convertAgentclones eachHandoffwith a newonInvokeHandoff. It runs the original callback and converts the agent it returns, through the same seen-map, so a returned agent that is also reachable throughhandoffsgets the same converted clone.Handoff's own agent, which is always the case forhandoff(agent)and plainAgenthandoffs, the result is the convertedagentfield, as before.runConfig.sandboxtoagentAsTooltools moved intoconvertAgent, which now takesrunConfig.sandbox, so agents converted at handoff time get it too. Before converting such an agent, the callback path also runs the runner's missing-sandbox check, now shared asrequireSandboxRunConfig, so a callback that returns aSandboxAgentwithoutrunConfig.sandboxfails withSandboxConfigurationError.src/__tests__/test-openai-agents.ts: a customHandoffthat picksviporstandardfrom the run context, and a replay of a History recorded onmainbefore this change (src/__tests__/histories/custom-handoff-before-fix.json). New sandbox tests insrc/__tests__/test-openai-sandbox.tsfor agents found through the callback.API change: none.
convertAgentis not exported, and no option or signature changes.Replay and rollback: a run that already handed off to the
agentfield, when the callback returned another agent, would replay into a different agent, with other tools and other commands. So the new path is behindpatched('openai-agents.handoff-returned-agent'), and the gate is only asked when the callback returns another agent. Runs recorded before this change keep the old target on replay. Runs that never hit that case record no marker and can roll back to the previous release. Runs that took the new path record the marker, so an earlier plugin version can't replay them ([TMPRL1100]on the marker). To roll back, keep this version and decline the patch with the Worker'spatchActivationCallback(experimental in SDK 1.24), or use Worker Versioning.Why?
See #237.
convertAgentreplaced eachHandoffwithh.clone({ agent: convertedAgent }). Given onlyagent, the framework'sHandoff.clonewraps the callback asawait this.onInvokeHandoff(context, args); return agent;, so the agent the callback returned was dropped and the run always went to theagentfield: the wrong agent answered, with its own tools, and nothing warned.Checklist
Closes [Bug] openai-agents (TypeScript): TemporalOpenAIRunner ignores the agent a custom Handoff's callback returns #237
How was this tested:
A custom Handoff hands off to the agent its callback returns. Withmain'sconvert-agent.tsit fails on its results assertion ('standard: vip answer'instead of'vip: vip answer'). With this change it passes. Thestandardrun is the control.A custom Handoff keeps its agent field when replaying a run recorded before the fix. The History was recorded onmain(c664a34) with the test Workflow: the callback returnedvip, the run went tostandard, andstandard's model called itsstandardDeskActivity. It replays onmainand with this change. Without the gate it fails with[TMPRL1100] Nondeterminism error: Fail workflow machine does not handle this event: HistoryEvent(id: 17, ActivityTaskScheduled).Handoff's callback returnsvip, whoseagentAsToolasks a nestedSandboxAgentto run a command through the fake Temporal sandbox client:A custom Handoff target returned only by the callback gets the sandbox config: on this PR's first commit it fails withA SandboxAgent was provided but runConfig.sandbox is not configured, although the run setsrunConfig.sandbox. With the second commit it passes and the nested command runs.A custom Handoff target also in the handoff graph gets the sandbox configis the control:vipis also a router handoff. It passes on both commits.A custom Handoff returning a SandboxAgent without runConfig.sandbox fails the Workflow: on the first commit the framework'sSandboxAgent execution requires RunConfig.sandboxerror ends the run; with the second it fails withSandboxConfigurationError.make lintpasses (8 existing warnings, none in files this PR touches),make buildpacks, andcheck_npm_package.py,smoke.mjsandcheck_conventions.pypass.make test TEST_ARGS="--concurrency=2"passes 212 tests (207 onmainplus the 5 new ones). With the default concurrency, one unrelated test (read-only handler tracing: repeated query/update-validation ...) timed out while the machine was busy; it passes on its own onmainand with this change.@temporalio/openai-agents1.24.0 and onmain, and 0 with this change.vip(changed) and astandard(unchanged) callback result, finished and held at the desk's last model call:main's histories replay with this change and onmain(6 of 6 each). They include two heldmainruns finished on this change: both kept thestandardtarget, with no failed Workflow Task.main, the 3 that took the new path fail with[TMPRL1100] ... Non-deprecated patch marker encountered, as expected for a gate; the other 4 replay.patchActivationCallbackdeclining new patches. The one that had taken the new path keptvip, with no failed Workflow Task; a newviprun on that build went tostandardand recorded no marker.viphistories frommainwith[TMPRL1100]; a renamed desk Activity in a history fails with[TMPRL1100].CI covers the other Node versions and platforms.
Any docs updates needed?
No. The README lists handoffs as supported; this makes the custom
Handoffform behave as upstream documents it.Not covered:
Modelobject and aSandboxAgentwithoutrunConfig.sandboxsee it only at handoff time. The sandbox client checks run up front wheneverrunConfig.sandboxis set, so they already cover it.Related: the open drafts for
agent-hooks-never-fireandagent-tool-run-model-lostalso changeconvertAgentand its call inrunner.ts, and other open drafts for this plugin (#209, #210, #215, #216–#218) add tests at the end ofsrc/__tests__/test-openai-agents.tsandsrc/__tests__/workflows/openai-agents.ts. Whichever merges later may need a small rebase there. With the hook-forwarding change, agents converted at handoff time should get the same hook forwarding.Open questions:
Draft while this question is open.