Repository navigation
Conversation
…agentAsTool (breaking) TemporalOpenAIRunner now gives its inner Runner a model provider that sends every model name to the model Activity, plus the run's model. Upstream passes the runner's config to nested agent.asTool() runs, so the framework's own asTool() works in a Workflow: its model calls are model Activities with the run's model and model settings and the runner's Activity options. agentAsTool and AgentAsToolOptions are removed. Use agent.asTool().
|
Let's take option 2: keep native For now, wrap The main benefit is the developer experience: applications moving from native execution to Temporal can keep their agent-tool declarations. We also reuse upstream's tool behavior rather than maintain a separate implementation. The plugin still needs its Temporal runner, Worker setup and Activity boundaries; native syntax does not remove those requirements. Existing plugin users would still migrate from The tradeoff is that the plugin owns a temporary patch to a shared upstream class in the Workflow runtime. That makes initialization order and SDK compatibility our responsibility. The implementation must install interception before tools are created, handle repeated initialization safely, and account for subclasses and duplicate SDK module copies. Tools whose source was not recorded must be rejected with a clear error. This choice does not accept the two review findings as documented limitations. Before merging, we need to:
In parallel, request a supported upstream way to inspect an agent tool's source and options and rebuild or rebind it while preserving identity and approval restoration. Once that exists, we can replace the interception inside the plugin while keeping the same native API and tested behavior. Callers should not need another migration for that internal change. |
TemporalOpenAIRunner no longer clones the agent graph to swap in Activity-backed models. Its inner Runner's model provider already sends every model name to the model Activity, so agents run as the caller wrote them. That also fixes two defects of the clones: agent.on(...) listeners on the caller's agents now fire (#235), and a custom Handoff now hands off to the agent its callback returns (#237). convertAgent becomes checkAgent, which runs the same checks, including the nested agent.asTool() checks, without changing the graph. asTool() tools are no longer rebuilt, so the asTool() record keeps only the agent and the runConfig and runOptions the tool was made with. The same-name check goes: without clones, handoff and nested agents keep their own string models. A ModelSummaryProvider still gets the running agent, tracked from the inner Runner's agent_start and agent_handoff events; a model call during a tool call belongs to a nested agent.asTool() run and gets none. As upstream, runConfig.model now applies only to agents that set no model of their own.
brianstrauch
left a comment
There was a problem hiding this comment.
Lint, build, and all 261 existing tests pass on Node 24. A focused Worker regression probe reproduces the nested-streaming restriction bypass noted inline.
| const innerRunner = new Runner({ | ||
| ...runnerConfigOverrides, | ||
| model: modelOverride, | ||
| modelProvider: new ActivityBackedModelProvider(this.modelParams, () => |
There was a problem hiding this comment.
[P2] Enforce the local Activity streaming restriction for nested tools
With useLocalActivity: true and a nested agent.asTool({ onStream }), the child run goes directly through upstream Runner, bypassing the StreamingLocalActivityUnsupported guard in TemporalOpenAIRunner.run(). This provider inherits the local Activity options and schedules invokeModelStreamActivity locally. I reproduced this by running the existing agentToolStreamWorkflow with modelParams.useLocalActivity: true: the provider's getStreamedResponse() runs once, both stream handlers receive the answer, and the parent succeeds instead of rejecting the unsupported combination. Please enforce the restriction in ActivityBackedModel.getStreamedResponse() or the shared run check, and add nested regression coverage.
There was a problem hiding this comment.
Good catch, thanks. Fixed in 3f11f6c: ActivityBackedModel.getStreamedResponse() now rejects useLocalActivity, next to its streaming-topic check, so every streamed model call has the check, including nested agent.asTool() runs. The runner's own check stays, so a top-level streamed run still fails before it starts.
One difference from the top-level case: as upstream, asTool() returns a nested run's failure to the parent model as the tool's output, so the parent run goes on. The nested model is never called. This matches the other checks that run inside a nested run. Rejecting the parent before it starts would need every on() handler up front, and handlers can be added after the run starts.
The new Worker test, A nested agent.asTool() run fails to stream when model calls are local Activities, runs agentToolStreamWorkflow with useLocalActivity: true. It fails without the check (the nested model streams) and passes with it.
…agent.asTool() runs A nested agent.asTool() run with onStream or on() handlers streams through upstream's Runner, so it skipped the runner's useLocalActivity check and ran the streamed model call as a local Activity. Check in ActivityBackedModel.getStreamedResponse(), which every streamed model call reaches. As upstream, the nested failure is the tool's output.
DABH
left a comment
There was a problem hiding this comment.
First review, at head 3f11f6c. On a worktree at the head with pnpm 10.27.0: make lint is clean (the 8 warnings already on main) and make test passes 262 of 262. I reviewed from four angles: line by line against @openai/agents-core 0.14.3, Temporal replay semantics, removed behaviour and documentation, and tests. I checked every candidate with a probe or traced it to file:line. The probes were:
- the upstream
Runnerwith a fake provider, for model precedence; - real Workers against a dev server, for the nested
SandboxAgent, the removed export and a nested 401; - histories recorded on
main(3488fbe), replayed against this head withWorker.runReplayHistory, and runs held onmainthen continued here with no Workflow cache; RunState.fromStringon a state saved from a graph built the old way;- a mutation pass over the new checks: removing the
getSystemPromptcheck body, the run wrapper, the frozen sandbox copy, the cancellation re-raise, the nested local-Activity stream check or the other-copy installs each fails at least one test.
Refuted and dropped:
- the
getSystemPrompthook misses no model call: its only caller is#prepareModelCall(run.mjs:1544), after model selection, in both loops and on resume, and it applies to theSandboxAgentexecution clone too; - the shallow
Object.freezeof the sandbox config is safe, because upstream only reads it; - the
toolCallscounter stays balanced, because upstream emitsagent_tool_endon the error path; - interceptor modules load before the Workflow module (
importInterceptors()runs beforeimportWorkflows()), so a module-scopeasTool()is recorded; - cancellation two
asTool()levels deep, followed by cleanup inCancellationScope.nonCancellable, completes; - the second-copy test is real: it bundles a copied package and guards against
copy.Agent === Agent; - #209, #210 and #166 don't depend on anything this removes. #209 and #210 conflict with it only in the two test files, #166 merges cleanly, and #209's README line about callable
toolFiltergetting the real agent needs this PR merged first.
The nested local-Activity streaming bypass from the earlier review is fixed in 3f11f6c, and its test fails without the check. This branch now conflicts with main in src/__tests__/test-openai-agents.ts and src/__tests__/workflows/openai-agents.ts after #215. Not run locally: the Node 22, macOS and Windows cells, which are green in CI.
Severity: P1 = wrong result or stuck run in realistic use; P2 = correctness or robustness in an edge case; P3 = docs, tests, cleanup.
Where the problems cluster: removing the clones is sound, and the new checks are well covered. The problems are in what changes for someone upgrading a Generally Available 1.x package:
- runs already in flight switch models with no version gate;
- the replay guidance promises loud failures where the actual outcome is silent wrong answers;
agentAsTooldisappears without a deprecation release;- two fail-fast behaviours become tool output that the parent model is told to retry: a nested
SandboxAgentwithout sandbox config, and non-retryable nested Activity failures.
Requesting changes for two items: the runConfig.model precedence flip changes runs already in flight on a Generally Available plugin with no version gate and diverges from the Python plugin, and dropping the nested SandboxAgent check turns a configuration error into a completed run. The other items can be follow-ups or documentation.
| // Nested `agent.asTool()` runs take the model, provider and settings from this config, as upstream. | ||
| const innerRunner = new Runner({ | ||
| ...runnerConfigOverrides, | ||
| model: modelOverride, |
There was a problem hiding this comment.
P2: runConfig.model now loses to any model the agent sets, with no version gate. Runs already in flight switch models silently after the upgrade, and the TypeScript and Python plugins now give opposite answers for the same config.
Upstream selectModel (runner/modelSettings.mjs:15) returns the agent's model whenever it is a non-empty string or a Model. Probe with the upstream Runner and a fake provider: an agent with own-model and runConfig.model: 'run-model' calls own-model, and a model-less agent calls run-model. On main, convertAgent replaced every agent's model with the override.
Replay doesn't compare Activity inputs. So a run that started with runConfig.model: 'gpt-5' and agents that declare gpt-4o-mini continues on gpt-4o-mini from its next model call, with no error. The PR body says this, but users read the README, and release notes are built from the commit subject only (scripts/release/release_tool.py:456).
"As upstream" cuts both ways here:
- Upstream's own contract (
run.d.ts:61, still the same on upstreammainatpackages/agents-core/src/run.ts:318) saysRunConfig.model"will override the model set on every agent". - The Python plugin keeps the run's model winning:
test_run_config_modelsasserts that only the run model is used. This is the point raised on Add a rule to match the upstream framework's behavior to AGENTS.md #272. - In fairness, upstream JS already lets an agent's
modelSettingswin over the run's, andmainpassed that through. So this change makes model and settings consistent with each other.
Fix, one of:
- Keep the run's model winning for the outer run, matching Python and the documented upstream contract. The
Runner.prototype.runwrapper already sees each run's config and can tell the outer run from nested ones. - Keep the agent winning, but gate it with
patched('openai-agents/agent-model-precedence'), read once at the top ofrun()and captured in the provider. In TS SDK 1.24,patched()returns false when replaying history recorded before the marker, so runs started before the upgrade keep the old rule. The Python plugin gates its own changes the same way (_openai_runner.py:281).
Either way, state the rule and the upgrade effect in the README next to line 100, and either align with the Python plugin or document the difference.
There was a problem hiding this comment.
Thanks for checking both sides. We're keeping upstream's code here: selectModel and its tests in test/run.test.ts let an agent's own model win, and our rule for these fixes is to match upstream's code and tests, even where its docs say otherwise. The Python plugin follows the Python SDK, whose code lets the run's model win, so the two plugins differ because the two SDKs do. We're also not adding version gates to fixes before 1.0, so no patched() marker.
In 0ca1882, the README's runner section states the rule, says upstream's docs disagree with its code, and notes the Python difference. The new "Upgrading open runs" section says runs in progress with runConfig.model switch to the agent's own model from their next model call. The runConfig.model JSDoc says the same, and the PR body's breaking-changes entry names the docs mismatch.
| ``` | ||
|
|
||
| Nested approval interruptions are not supported. If a nested run pauses for approval, the tool invocation fails with `ApplicationFailure` type `NestedAgentInterruption`. | ||
| As in the framework, the nested run takes the run's `model` (for agents that set none) and `modelSettings`. Its model Activities use the same options as the runner's. |
There was a problem hiding this comment.
P2: the upgrade guidance overstates what a replay check catches, and it exists only in the PR body.
The PR body lists five kinds of run that "fail replay with [TMPRL1100]". That holds when the added or missing step is not a model call, for example the Timer in the PR's own check. When it is another model call, replay passes: every model call is the same Activity type, and replay compares only type and ID.
Verified with a history recorded on main:
agentAsTool's nested call gets HTTP 400, and the Workflow catches it and runs a recovery agent (threeinvokeModelActivityevents).- Replaying it against this head with
Worker.runReplayHistorygives no error. - The same run, held on
mainafter the recovery and continued on this head, completes withcompleted:recovery-text. The recovery agent's recorded answer became the Triage agent's final output, and thecatchbranch was skipped. - Likewise, a run that called native
agent.asTool()onmainand was held afterrun()continues on this head with the parent's recorded second response delivered as the nested agent's answer.
Two more cases are missing from the list:
defaultModelParams.cancellationType: ABANDONnow applies to nested calls. A run cancelled during a nested call that is still open no longer sendsRequestCancelActivityTask, which gives TMPRL1100 (by trace).- A
RunStatesaved onmaincan resume on the wrong agent when anasTool()source agent has the same name as a handoff target. Upstream now addsasTool()sources to the RunState identity map, andagentAsTooltools were never in it. Probe: a savedcurrentAgent: {name: 'X'}resolves to the tool'sX, and resuming fails withTool confirm not found. If the two agents had the same tool names, the run would continue silently as the wrong agent. The one migration test (legacyFirstRun) uses the same object as both handoff target and tool source, which hides this.
Fix: add a short "Upgrading" subsection here saying that:
- open runs that used
agentAsTool,asTool()orrunConfig.modelshould finish on the old Worker or be pinned with Worker Versioning; - a passing replay test of old histories is not evidence of safety;
- agent names must be unique across handoffs and
asTool()sources before resuming an oldRunState.
Committing one cross-version history recorded on 3488fbe would also help. The Python plugins keep such fixtures under tests/histories/.
There was a problem hiding this comment.
Agreed, and thanks for the probes. In 0ca1882 the README has an "Upgrading open runs" section. It lists the runs to let finish or pin with Worker Versioning: agentAsTool and asTool() runs, runConfig.model with agents that set their own model, useLocalActivity or cancellationType now applying to nested calls, and hooks and custom Handoff callbacks. It says a passing replay test of old histories doesn't show an upgrade is safe, because a different model call replays without an error. It also says to give agents unique names across handoffs and asTool() tools before resuming an old RunState.
I rewrote the PR body's replay section the same way, with the ABANDON and RunState cases and the silent model-call outcomes you found.
I didn't commit a cross-version history. These runs replay without an error, so a fixture would mostly pin that, and the README now says it. Happy to add one if you'd still like it.
| } | ||
| return (await innerRunner.run(converted, preparedInput as any, sharedRunOptions)) as RunResult<any, TAgent>; | ||
| const result = (await innerRunner.run(agent, preparedInput as any, sharedRunOptions)) as RunResult<any, TAgent>; | ||
| if (scope.consideredCancelled) await scope.cancelRequested; |
There was a problem hiding this comment.
P2: nested failures that main treated as terminal, including non-retryable model Activity failures, now go back to the parent model as "Please try again". The run can spend model Activities until maxTurns and then fails without the cause.
Upstream asTool() builds its tool with tool() and exposes no errorFunction option (agent.mjs:183, :209), so every thrown error becomes the tool's output (tool.mjs:505).
Verified with a nested model that returns 401, which the Worker classifies as non-retryable (src/worker/activities.ts:96, :101), and a parent that calls the tool again:
mainschedules 2 Activities and fails withFailed to run function tools: ActivityFailure;- this head schedules 20 (10 parent, 10 nested) and fails with
Agent workflow failed: Max turns (10) exceeded, with no trace of the auth failure.
This PR also adds a realistic trigger: nested runs now inherit the run's modelSettings, so a setting the nested model rejects (temperature on a reasoning model, for example) gives a 400 on every call.
The same path also catches:
- a
TimeoutFailureafter retries run out; - the plugin's own checks;
- a
ModelSummaryProviderthat readsagent.name. It is now called for nested calls withagent: undefined; onmainthe nested runner had nodefaultModelParams, so it never ran for them.
The README's nested section mentions none of this; only the cancellation bullet at line 102 touches it.
Fix: this line already re-raises a cancellation that a tool turned into output. Do the same for terminal failures: in the Runner.prototype.run wrapper, record a nested run's non-retryable ApplicationFailure, TimeoutFailure, or ActivityFailure with a non-retryable cause, and rethrow it from run(). If upstream's behaviour is the intended contract, document the maxTurns cost and the lost cause in the nested section instead.
There was a problem hiding this comment.
This is upstream's asTool() contract: its tool has no errorFunction option, so every nested error becomes the tool's output, and a native run with a 401 on the nested model loops the same way. Our rule for these fixes is to match upstream's behavior, so I took your second option. In 0ca1882, the README's nested section says which failures come back as tool output: model Activity failures that aren't retried, timeouts, the plugin's checks in the nested run, and a nested SandboxAgent without sandbox config. It also says the parent may call the tool again until maxTurns, that the final error doesn't carry the cause, and that a ModelSummaryProvider gets agent: undefined for nested calls. The cancellation re-raise stays the one exception, because a cancelled scope must not complete.
| @@ -7,8 +7,6 @@ export { WorkflowSafeMemorySession } from './workflow/session'; | |||
| export type { WorkflowSafeMemorySessionOptions } from './workflow/session'; | |||
| export { nexusOperationAsTool } from './workflow/nexus-tools'; | |||
| export type { NexusOperationToolDefinition, NexusOperationAsToolOptions } from './workflow/nexus-tools'; | |||
There was a problem hiding this comment.
P2: agentAsTool is removed from a Generally Available 1.x package without a deprecation release. A Workflow bundled without a type check ends up in a Workflow Task retry loop instead of a build error.
1.18.0 through 1.24.0 export agentAsTool and AgentAsToolOptions. The next release will be above 1.24.0, so ^1.x ranges pick up the removal.
Verified: bundleWorkflowCode with the plugin and a .ts Workflow that calls agentAsTool (swc with no type check, which is the samples' dev path) builds with only an INFO-level webpack warning. Every Workflow Task then fails with agentAsTool is not a function until the execution timeout. Projects that run tsc get TS2305 instead.
Fix: for one release, keep a @deprecated agentAsTool(agent, { toolName, toolDescription, customOutputExtractor }) that returns agent.asTool({ ... }), plus the type alias. The tool is recorded automatically, because loading agent-tools.ts installs the interception, and upstream's customOutputExtractor type is compatible. Also add the one-line migration to the README.
There was a problem hiding this comment.
Done in 0ca1882: agentAsTool and AgentAsToolOptions are back as @deprecated aliases for one release. agentAsTool returns agent.asTool({ toolName, toolDescription, customOutputExtractor }), so the tool is recorded and runs like any other asTool() tool. The README's "Upgrading open runs" section has the migration, including that a nested approval now pauses the parent run instead of failing with NestedAgentInterruption. A unit test checks the alias.
| */ | ||
| /** A reachable `SandboxAgent` needs `runConfig.sandbox`, with a Temporal sandbox client. */ | ||
| function validateSandboxRunConfig(agent: Agent<any, any>, sandbox: SandboxRunConfig | undefined): void { | ||
| if (!hasSandboxAgent(agent) && sandbox === undefined) return; |
There was a problem hiding this comment.
P2: a SandboxAgent behind an asTool() tool, in a run without runConfig.sandbox, no longer fails fast. The run completes.
hasSandboxAgent (lines 137-146) follows only handoffs, and checkAgent walks the asTool() records without asking whether a sandbox is needed. On main, the nested TemporalOpenAIRunner.run threw SandboxConfigurationError, and agent-tools.test.ts covered it ("agentAsTool: nested SandboxAgent without parent sandbox config fails"). The PR deletes that test without a replacement.
Verified with a real Worker: new SandboxAgent({...}).asTool() on a plain agent, run with no sandbox, ends COMPLETED. The tool output is An error occurred while running the tool. Please try again. Error: UserError: SandboxAgent execution requires RunConfig.sandbox. A configuration error now reaches the model as a prompt to retry.
Fix: in hasSandboxAgent, also follow getAgentToolRecord(tool)?.agent when the record sets neither runConfig.sandbox nor runOptions.sandbox, and restore the test using asTool().
There was a problem hiding this comment.
Thanks, the probe is right. Here we're following upstream: a native run with the same graph gets the same UserError as the tool's output and completes, and our rule for these fixes is not to add plugin rules that depart from upstream's behavior. On main the fail-fast came from agentAsTool's own runner, which this PR replaces. The handoff check stays, because a handoff to a SandboxAgent without sandbox config fails the run upstream too.
In 0ca1882, the README lists this case with the other nested failures that become the tool's output, and Sandbox configuration errors are raised inside the Workflow now pins it: the nested SandboxAgent fails before calling its model, and the run ends on that tool output.
| if (rawModel !== undefined && rawModel !== null && typeof rawModel !== 'string') { | ||
| throw ApplicationFailure.create({ | ||
| message: | ||
| `Agent '${agent.name}' has a Model object instead of a string model name. ` + |
There was a problem hiding this comment.
P3: this message and a few other places still describe the run's model winning.
Now that the agent wins, runConfig.model cannot override an agent's Model object. Probe: the upstream Runner with runConfig.model set still calls the agent's inline model. So "use runConfig.model to override" points the user the wrong way; the fix is to replace the Model object with a model name.
The same wording appears in:
- the JSDoc of
TemporalRunOptions.runConfig.model: "Model name override" (runner.ts:67, outside the diff); - the README feature-table row label "Model override per run" (line 744).
Older, but nearby: runner.ts:69 says the run's modelSettings "override agent-specific settings", but upstream merges the agent's settings over the run's (run.mjs:1539-1540).
Also, the README drops the nested-approval note without saying that a nested approval now pauses the parent run.
There was a problem hiding this comment.
Fixed in 0ca1882. The Model object message now says to give the agent a model name. The runConfig.model and modelSettings JSDoc say the agent's own model and settings win, as upstream. The feature-table row is now "Run model". The nested section says a nested approval pauses the parent run.
| function asTool(this: Agent<any, any>, options: AnyAgentToolOptions): AnyAgentTool { | ||
| // Upstream reads its options when the tool is made, so record them then too. | ||
| const tool = state.original.call(this, options); | ||
| state.records.set(tool, { agent: this, runConfig: options.runConfig, runOptions: options.runOptions }); |
There was a problem hiding this comment.
P3: the record is keyed by the tool object, so upstream's own toolNamespace() makes a valid asTool() tool look unrecorded, and the run is rejected.
toolNamespace() (tool.mjs:892) returns copies built with Object.create plus the original property descriptors (tool.mjs:513-519). A copy is a new object that this WeakMap doesn't know, but it keeps on, so isUnrecordedAgentTool matches and checkToolOptions throws.
Probe: a plain asTool() tool is accepted. The same tool passed through toolNamespace({ name, description, tools: [it] }) is rejected with "looks like an agent.asTool() tool that @temporalio/openai-agents did not record ... pass it as is". Upstream would run the copy, since it has the same invoke and on closures.
Fix: key the record by a closure that copies keep, such as tool.on, in both set and get. That also covers spread copies.
Separately, asTool() tools made by an agent from another copy of @openai/agents-core are always rejected, because only this copy's asTool is wrapped. README line 430 says such agents are "checked the same way". Add that case to the list at line 421.
There was a problem hiding this comment.
Good catch. In 0ca1882 the record is keyed by the tool's on closure, so spread copies and toolNamespace() copies keep it. The unit test covers both, and a namespaced tool passes checkAgent. The Worker test's unrecorded case is now an asTool() tool from a second copy of @openai/agents-core, which is rejected before any model call. The README says such tools are rejected.
| const provider = modelSummary as ModelSummaryProvider; | ||
| const systemInstructions = request.systemInstructions; | ||
| const summary = provider.provide(this.agent, systemInstructions, request.input); | ||
| const summary = provider.provide(this.agent ?? this.runningAgent?.(), systemInstructions, request.input); |
There was a problem hiding this comment.
P3 (non-blocking cleanup): setAgent and the agent field have no production caller now that convertAgent is gone.
Only activity-backed-model.test.ts:67 uses them, so this.agent ?? ... always falls through to runningAgent, here and at line 170.
Similarly, the "no model declared" branch at check-agent.ts:22 can't fire for an agents-core Agent:
- the constructor sets
model = config.model ?? ''(agent.mjs:81); rejectModelObjecthas already thrown for any non-string model;- the unit test has to force
model = undefinedto reach the branch (check-agent.test.ts:32).
main behaved the same, but in this PR the modelOverride and nestedModel plumbing exists only to feed that branch. Remove both.
There was a problem hiding this comment.
Removed in 0ca1882: setAgent and the agent field (the summary unit test passes the agent through the constructor), the unreachable no-model branch, and the modelOverride and nestedModel plumbing.
|
|
||
| // The Worker loads interceptor modules before the Workflow module, so agent.asTool() tools that | ||
| // Workflow code creates at module scope are recorded too. | ||
| installAgentToolInterception(); |
There was a problem hiding this comment.
P3: three of the new checks have no test. Removing any one of them still passes the suite, 262 of 262.
- This install. Removing the import and the call passes. But a probe with a module-scope
nested.asTool(), in a module imported before@temporalio/openai-agents/workflow, is accepted at the head and rejected as unrecorded without the install. Commit that probe as a test. - The run-start
MemorySessioncheck (check-agent.ts:161-169). No test sets aMemorySessionafter the pre-run check, for example from a listener or aninputBuilder. Atool-start-memory-sessioncase inrunningAgentRejectedWorkflowwould cover it. - The run-start
checkAgentCopy(agent)(check-agent.ts:140). It looks redundant with line 20 and theagent_startlistener, which are both covered. Test it or drop it.
The agent_handoff listener at runner.ts:264 can also be removed with no failure. It is redundant, because upstream emits agent_start on the handoff target.
There was a problem hiding this comment.
Done in 0ca1882:
- The options-reuse Worker test now also passes
ask_early, a tool made at module scope in a module imported before@temporalio/openai-agents/workflow. Without the install in the interceptor module, the run is rejected as unrecorded. runningAgentRejectedWorkflowhas atool-start-memory-sessioncase: anagent_tool_startlistener sets aMemorySessionin the tool'srunOptions. Without the run-start check, the nested run uses it.- I dropped the run-start
checkAgentCopy(agent)and theagent_handofflistener as redundant.
Both new cases fail with their check removed.
…escript-native-as-tool # Conflicts: # typescript/openai-agents/src/__tests__/test-openai-agents.ts # typescript/openai-agents/src/__tests__/workflows/openai-agents.ts
…ed failures From review of 3f11f6c: - agentAsTool and AgentAsToolOptions come back as deprecated aliases for one release; agentAsTool returns agent.asTool({ toolName, toolDescription, customOutputExtractor }). - The asTool() record is keyed by the tool's on closure, so copies, such as the ones toolNamespace() makes, keep it. - The README gets an "Upgrading open runs" section, documents which nested failures become the tool's output, the nested approval, and the runConfig.model rule next to upstream's docs and the Python plugin. - Wording that described the run's model winning is fixed. - setAgent, the unreachable no-model branch, the run-start copy check and the agent_handoff listener are removed. - Tests cover the Workflow interceptor module's install, the run-start MemorySession check, another copy's asTool() tool and a nested SandboxAgent without sandbox config.
…escript-native-as-tool # Conflicts: # typescript/openai-agents/src/__tests__/test-openai-agents.ts # typescript/openai-agents/src/__tests__/workflows/openai-agents.ts # typescript/openai-agents/src/workflow/runner.ts
DABH
left a comment
There was a problem hiding this comment.
Second round, at head 2c2dc14 (0ca1882 plus a merge of main at 88745dc). On a worktree at the head with pnpm 10.27.0: make lint is clean (the 8 warnings already on main), make test passes 270 of 270, and CI is green. I re-ran the reproductions behind each item from the last round, then reviewed the whole diff again from four angles: line by line against @openai/agents-core 0.14.3, Temporal replay, removed behaviour and parity with the Python plugin, and tests and docs. Each finding below was reproduced with real Workers against a dev server, running the same Workflow code on main (88745dc) and on this head, or with a direct probe of the upstream code, or traced to file:line.
Replay: histories recorded on main were replayed here with Worker.runReplayHistory, and runs held on main were continued here with no Workflow cache. Plain agentAsTool runs replay and continue. So do runs with runner defaults (timeout, retry, summary string, task queue), a parallel activityAsTool, MCP, handoffs, RunState resumes in place and across continueAsNew, and a streamed parent. Every case in the PR body's Replay list behaves as described there. The undisclosed breaks are below.
Tests: a mutation pass over the new checks (45 mutants) kills the interceptor-module install, the run-start MemorySession check, the on-keyed record and the agent_start listener; the two survivors are below. The headline tests for #243, #235 and #237, ported to main, fail there.
Refuted and dropped:
- false positives from
isUnrecordedAgentToolon upstream tools (in 0.14.3 onlyasTool()addson); - a model call that skips the
getSystemPrompthook (every call goes through#prepareModelCallor reuses its prepared model); - nondeterminism from the prototype patches (each execution has its own module cache, and 15 Workflow types recorded concurrently on one Worker self-replay);
- a new
ApplicationFailurelanding as a Workflow Task failure (each fails the Workflow or becomes tool output; the one task-failure path is theTypeErrorbelow); - identity dependencies on the removed clones in the MCP tool cache (keyed by names), Nexus tools,
activityAsTooland sessions; - output differences in the deprecated alias beyond whitespace in structured output.
Not verified: the strict-endpoint failure below uses a fake provider that rejects the fields, not a live endpoint.
Severity: P1 = wrong result or stuck run in realistic use; P2 = correctness or robustness in an edge case; P3 = docs, tests, cleanup.
Previous rounds. Addressed, each re-checked at this head: agentAsTool and AgentAsToolOptions are back as deprecated aliases, and a Workflow calling agentAsTool completes through one nested model Activity; the record is keyed by tool.on, so the toolNamespace() reproduction is accepted, its nested agent is still checked (a Model object in it is rejected) and spread copies pass; the README has the "Upgrading open runs" section, the nested-failure paragraph and the nested-approval note; the Model-object message, the runConfig.model and modelSettings JSDoc and the table row are corrected; setAgent, the no-model branch, the run-start copy check and the agent_handoff listener are gone; the interceptor-module install and the run-start MemorySession check now have tests that fail without them. Accepted declines: no version gate for the runConfig.model precedence (the README now states the rule, upstream's docs mismatch, the Python difference and the effect on open runs), the nested SandboxAgent failure becoming tool output (upstream behaviour, documented and pinned by a test), and no committed cross-version history. brianstrauch's nested local-Activity streaming item stays fixed. Every review thread has an answer.
Where the problems cluster: removing the clones is sound, the alias restores source compatibility with 1.24, and the new checks are well covered. What is left comes from running the caller's agents as written:
- Upstream now sees string model names. It adds implicit GPT-5 settings and prompt-template flags that
mainnever sent, and the plugin forwards them without what the Worker needs to treat them as implicit. - Sharing the caller's objects exposes upstream rules the clones hid: one active run per
SandboxAgent, andRunStatelookup by declared agents. - Several of these, and two replay effects (command order within a turn, cancelled runs), change open or closed runs silently. None of them is in the upgrade section.
Requesting changes for one item: an agent that names no model now sends GPT-5 settings computed in the Workflow to whatever model the Worker resolves, which fails the first model call on a strict OpenAI-compatible endpoint, the failure #178 fixed in the Python plugin. The other items can be fixed here or documented.
PR body and title (P3). Release notes are built from the commit subject only (scripts/release/release_tool.py:456, :473). "run agents as written, support agent.asTool() and deprecate agentAsTool (breaking)" doesn't say what breaks. Recent breaking subjects name the change (#204 "Breaking: refuse RunConfig.model_provider in OpenAI Agents workflows"). Something like "Breaking: run TypeScript OpenAI Agents as written: agent.asTool() works, runConfig.model no longer overrides an agent's own model, agentAsTool is deprecated" would carry it. Stale or inaccurate after 0ca1882:
- "checkAgent checks ... a missing model": there is no such check;
- "tracked from the inner Runner's agent_start and agent_handoff events": only
agent_startremains; - "Not covered: Usage across nested runs": usage now accumulates in the parent;
- "Not covered: a Handoff object from another copy": the plugin's
checkAgentrejects it first, with "Agent 'undefined' is not an @openai/agents-core Agent" (check-agent.ts:26,:74-79), which could name the handoff instead.
| private readonly runningAgent?: () => Agent<any, any> | undefined | ||
| ) {} | ||
|
|
||
| getModel(modelName?: string): Model { |
There was a problem hiding this comment.
P2 (blocking): implicit model settings computed in the Workflow now reach the Worker as if they were explicit. An agent that names no model fails its first model call on a strict OpenAI-compatible endpoint, and a run-level providerData.reasoning_effort is overwritten.
On main the agent's model was the ActivityBackedModel object, so #prepareModelCall (run.mjs:1524-1531) had no resolved name and added no implicit settings. Now upstream sees the name and adds getImplicitModelSettingsForResolvedModel(...) (runner/runConfig.mjs:10-18). For an agent with no model, that means the defaults for the Workflow's own default model, gpt-5.4-mini. The Workflow sandbox doesn't see the Worker's OPENAI_DEFAULT_MODEL, and the Worker's provider picks the real model later.
Probe, real Workers recording the Activity input, same Workflow on main and this head, agent new Agent({ name, instructions }):
mainsendsmodelSettings: {}. This head sends{"reasoning":{"effort":"none"},"text":{"verbosity":"low"}}. It does so also withOPENAI_DEFAULT_MODEL=gpt-4.1on the Worker. Native upstream with that variable sends{}, so this is not upstream's behaviour either.- Against a provider that rejects fields its model doesn't take (Fireworks answered
400 Extra inputs are not permitted, field: 'verbosity'in openai_agents: align implicit model settings with the model a custom provider resolves #178):mainCOMPLETED; this head FAILED on the first model call withModelInvocationError.BadRequest, non-retryable. In-flight runs hit it at their next model call.
Second effect: toSerializedModelRequest (lines 38-53) drops request._internal. The Worker's Chat Completions model therefore can't apply upstream's reasoningEffortImplicit guard (openaiChatCompletionsModel.mjs:386-389). Probe of OpenAIChatCompletionsModel with gpt-5, the implicit reasoning.effort: 'low' and a run-level providerData.reasoning_effort: 'high': with the marker it sends high; without it, as the plugin calls it, it sends low. A gpt-5.6 agent with tools likewise gets an effort that upstream omits.
The Python plugin fixed the first case in #178. The stub flags implicit settings when no model is named (_temporal_model_stub.py:184-196). The Activity then re-derives them for the model it resolved (_invoke_model_activity.py:335-356).
Fix:
- Carry
reasoningEffortImplicitin the Activity input and restore it as_internalon the Worker. - For an empty model name, port openai_agents: align implicit model settings with the model a custom provider resolves #178: send whether the settings are upstream's implicit defaults, plus the run's
modelSettings. The Activity replaces them withgetDefaultModelSettings(<resolved model name>)and re-applies the run settings. - Test with a Worker default that isn't GPT-5, and with a Chat Completions model.
| - use `agentAsTool` or `agent.asTool()`; | ||
| - set `runConfig.model` and have agents with a model of their own: those agents use their own model from their next model call; | ||
| - set `useLocalActivity` or `cancellationType` in the runner's options: nested model calls now use the runner's Activity options; | ||
| - have agent hooks (`agent.on(...)`), or a custom `Handoff` whose callback returns another agent: they now take effect. |
There was a problem hiding this comment.
P2: running agents as written changes more than this list says. Each item was checked with the same Workflow on main and on this head (the last by trace). The first five follow upstream, so they need disclosure rather than code; open runs pick them up as they continue, with no replay error:
- a gpt-5-family agent without
modelSettingsnow sends upstream's implicit settings:gpt-5gets{"reasoning":{"effort":"low"},"text":{"verbosity":"low"}}andgpt-5.4-minigets effortnone, wheremainsent{}. A nestedagentAsToolagent ongpt-5gets the same.modelSettings: {}on the agent opts out (probe:{}again); - explicit GPT-5
providerData.reasoningon agpt-5agent, whichmainstripped ({"providerData":{}}), now reaches the model; - an agent with a
prompttemplate and no model or tools now gets the template's model and tools.mainsentoverridePromptModel: trueandtoolsExplicitlyProvided: true, so the Worker's default model andtools: []replaced them (run.mjs:711-714,openaiResponsesModel.mjs:2417,:2455-2457); Agentsubclass overrides such asgetSystemPromptnow run (main's clone built a plainAgent), and aSandboxAgent's compaction threshold now follows its model;- the parent's
result.state.usagenow includes nested runs (2 requests onmain, 3 here), which contradicts the PR body's "Not covered: Usage across nested runs".
Two are the plugin's own:
- a
ModelSummaryProviderthat readsagent.namewithout a check worked on 1.24, because nestedagentAsToolcalls never called it. It now throws on them, so the nested call is never scheduled, and a run held onmainafterrun()loops on[TMPRL1100]; - an upstream
Runnerwith the user's own Workflow-safe provider worked on 1.24. It is now rejected at run start in any bundle that imports/workflow(check-agent.ts:123-131). The PR body says such runs "failed ... or ran models in the Workflow", which doesn't hold for that case.
Fix: add these to Breaking changes and to this list, and name modelSettings: {} as the opt-out from implicit settings.
| return streamed; | ||
| } | ||
| return (await innerRunner.run(converted, preparedInput as any, sharedRunOptions)) as RunResult<any, TAgent>; | ||
| const result = (await innerRunner.run(agent, preparedInput as any, sharedRunOptions)) as RunResult<any, TAgent>; |
There was a problem hiding this comment.
P2: the same SandboxAgent can no longer serve two runs at once, and main histories that did so fail replay. Neither the PR body nor the upgrade section mentions it. Upstream refuses concurrent reuse of one SandboxAgent object (sandbox/runtime/agentKeys.mjs:51-60). main never hit that guard, because convertAgent gave each run its own clone; this line now runs the caller's object.
Probe, same Workflow on both sides, with a fake sandbox client:
Promise.allSettledof tworunner.run(sbx, ...)calls withrunConfig.sandbox:maingives["ok:plain","ok:plain"]. This head gives["ok:plain","err:Agent workflow failed: SandboxAgent 'sbx' cannot be reused concurrently across runs"].- A parent model making two parallel calls to
agentAsTool(sbx, ...):mainanswers both. Here the second tool output isAn error occurred while running the tool. Please try again. Error: Error: SandboxAgent 'sbx' cannot be reused concurrently across runs, and the run completes on that output. - Both
mainhistories replayed on this head fail with[TMPRL1100]. The sequential control replays cleanly.
The upgrade list's "use agentAsTool or agent.asTool()" covers the nested case only by accident, and nothing covers concurrent top-level runs. Fix: list it under Breaking changes, in the Replay list and in the upgrade section. Tell users to create one SandboxAgent per concurrent run, or to set modelSettings: { parallelToolCalls: false } on a parent that calls a sandbox agent's tool.
| // eslint-disable-next-line @typescript-eslint/no-deprecated -- the alias takes its own deprecated options | ||
| options: AgentAsToolOptions<TAgent> | ||
| ): FunctionTool<any, any, any> { | ||
| return agent.asTool({ |
There was a problem hiding this comment.
P2 (edge): the deprecated alias rejects an agent from another copy of @openai/agents-core, which agentAsTool on main ran. agent.asTool resolves to the other copy's unwrapped method, so the tool is never recorded and checkToolOptions (check-agent.ts:183-193) fails the run as "looks like an agent.asTool() tool that @temporalio/openai-agents did not record".
Probe, bundled with a second copy as in test-openai-agents.ts: main completes through 3 model Activities. This head fails with that AgentsWorkflowError and 0 Activities. With Agent.prototype.asTool.call(agent, { ... }) here (this copy's wrapped method), the run completes through 3 Activities, as on main.
That is the fix. Otherwise the README's "returns agent.asTool(...)" and the PR body's "a deprecated alias that does the same" need the exception.
| // resolve manifest environment references on the Workflow thread rather than in the resume Activity. | ||
| // fromString lists tools, which runs MCP tool filters; their errors fail the run, as in the run below. | ||
| const restored = (await RunState.fromString(converted, input.toString()).catch((error: unknown) => { | ||
| const restored = (await RunState.fromString(agent, input.toString()).catch((error: unknown) => { |
There was a problem hiding this comment.
P2 (edge): resuming an approval fails when a custom Handoff's callback picked an agent that isn't declared in the graph, a path #237 now reaches. This round trip rebuilds the state from agent. Upstream's buildAgentMap (runState.mjs:2471-2502) only walks declared handoff agents and asTool() sources, and resolveSerializedAgent (:2510-2522) has no fallback for the current agent.
Probe: a Handoff declared with Declared whose callback returns Picked; Picked has a tool with needsApproval. After approving on first.state, runner.run(lead, first.state) fails with AgentsWorkflowError: Agent workflow failed: Agent Picked not found, non-retryable. Native upstream completes the same in-memory resume. main never reached Picked at all.
Fix: round-trip only when the state holds a live sandbox session, or fall back to the in-memory state when fromString can't find its current agent. Otherwise document that an agent a callback returns must also be declared, for example as a handoff of some agent in the graph, to resume its approvals.
|
|
||
| A passing replay test of old histories doesn't show these runs are safe. Every model call is the same Activity type, and replay doesn't compare Activity inputs, so a run that now makes a different model call can replay without an error and continue with the wrong answer. | ||
|
|
||
| Before resuming a `RunState` saved on an earlier version, give agents unique names across handoffs and `asTool()` tools. The framework also finds `asTool()` agents by name when it restores a `RunState`, and `agentAsTool` tools weren't included, so a saved run can resume on the wrong agent. |
There was a problem hiding this comment.
P3: this advice can resume a saved run on the wrong agent, depending on which agent is renamed. Probe with a state saved from a 1.24-shaped graph (currentAgent: {"name":"X"}, handoff target X). Renaming the handoff target resumes on the asTool() agent, with no error. Renaming the asTool() agent resumes correctly. buildAgentIdentityMap includes asTool() sources (runStateIdentity.mjs:63-66).
Suggested: "rename the asTool() agents; keep the names of the starting agent and its handoff targets".
|
|
||
| Before resuming a `RunState` saved on an earlier version, give agents unique names across handoffs and `asTool()` tools. The framework also finds `asTool()` agents by name when it restores a `RunState`, and `agentAsTool` tools weren't included, so a saved run can resume on the wrong agent. | ||
|
|
||
| `agentAsTool(agent, { toolName, toolDescription, customOutputExtractor })` is deprecated and will be removed in a later release. It returns `agent.asTool({ toolName, toolDescription, customOutputExtractor })`; call that instead. As upstream, a nested approval now pauses the parent run, where `agentAsTool` failed the tool call with `NestedAgentInterruption`. |
There was a problem hiding this comment.
P3: on 1.24 a nested approval failed the whole run, and code written for that now completes with no output. "failed the tool call with NestedAgentInterruption" understates the old behaviour. Probe on main: run() threw AgentsWorkflowError: Agent workflow failed: Failed to run function tools: ApplicationFailure: Nested agent 'Nested' ... paused for 1 approval interruption(s) (upstream wraps the tool's error). Here the same Workflow completes with finalOutput undefined and one interruption.
Suggested: "where agentAsTool failed the run", plus: check result.interruptions after run() when a nested agent has tools that need approval.
| ``` | ||
|
|
||
| Nested approval interruptions are not supported. If a nested run pauses for approval, the tool invocation fails with `ApplicationFailure` type `NestedAgentInterruption`. | ||
| As in the framework, the nested run takes the run's `model` (for agents that set none) and `modelSettings`. Its model Activities use the same options as the runner's. A nested approval pauses the parent run: approve it on the parent's `result.state` and resume, as in [Run state and approvals](#run-state-and-approvals). |
There was a problem hiding this comment.
P3: nested runs share more with the parent than this paragraph says, and some settings do not carry over.
- They share the parent's
RunContext(agent.mjs:218-226), so context writes and usage accumulate in the parent.main'sagentAsToolpassed no context. traceIncludeSensitiveData, guardrails andhandoffInputFilterare not inherited (agentToolRunConfig.mjs:149-190). The nestedRunnerdefaultstraceIncludeSensitiveDatato true, so tool I/O from nested runs is traced with data even when the parent opted out. This matches upstream andmain; the Python plugin passes the wholeRunConfig.asTool({ onStream })or an.on(...)handler makes the nested run stream. That needs the runner'sstreamingTopicand a hostedWorkflowStream; otherwise the tool output is theStreamingTopicNotConfiguredmessage (activity-backed-model.ts:129-136). Its events are published to the parent's topic with no agent attribution.
One or two sentences each would cover it.
| | Sessions | Supported | Via `WorkflowSafeMemorySession`; upstream `MemorySession` is rejected | | ||
| | Run state and approvals | Supported | Serialize with `result.state.toString()` and rehydrate with `RunState.fromString` | | ||
| | Guardrails | Supported | Guardrail callbacks must be deterministic | | ||
| | Agent lifecycle hooks | Supported | `agent.on(...)` listeners run in the Workflow and must be deterministic | |
There was a problem hiding this comment.
P3: listener errors behave differently from Node, and this PR is what makes these listeners fire.
- A listener that throws synchronously is ignored: the Workflow's
EventTargetpolyfill catches and drops it (load-polyfills.ts:24-30), and the run completes. In plain Node the same listener fails the run with its error, so a listener used as a guard no longer stops anything. - An
asynclistener that rejects (for example one that awaits a failing Activity) givesWorkflowTaskFailed: Unhandled Promise rejection, a retry loop that leaves the run open.
Fix: rethrow the first listener error after dispatch so the run fails as it does in Node, or at least say both in this row or in the hooks section.
| test('agentAsTool: the deprecated alias returns the agent.asTool() tool', (t) => { | ||
| const agent = new Agent({ name: 'Nested', model: 'm' }); | ||
| // eslint-disable-next-line @typescript-eslint/no-deprecated -- tests the deprecated alias | ||
| const tool = agentAsTool(agent, { toolName: 'ask_nested', toolDescription: 'Ask the nested agent.' }); |
There was a problem hiding this comment.
P3 (tests): two mutants of new code survive the full suite (270 of 270).
- Dropping
customOutputExtractorfrom the alias (agent-tools.ts:139). This test checks only the name, description and record; head behaves correctly (a Worker probe shows the parent receiving the extracted text). Assert that the extractor reachesasTool, or run it in a Worker test. - Initialising
runningAgenttoagentinstead of the state's_currentAgent(runner.ts:256-257). After resuming aRunStatepaused on a handoff target, theModelSummaryProviderlabels the resumed calls with the starting agent: head gives["agent:lead","agent:billing","agent:billing"]and the mutant gives[..., "agent:lead"]. Upstream emitsagent_starton resume only whennoActiveAgentRunis set. Add this resume case to the summary test.
What was changed
TemporalOpenAIRunnerruns the caller's agents as they are written. It no longer clones the agent graph to swap in Activity-backed models. Its innerRunnergets a model provider,ActivityBackedModelProvider, that sends every model name to the plugin's model Activity with the runner's Activity options, and the run'smodel.agent.asTool()runs, so the framework's ownasTool()now works in a Workflow. Nested runs take the run'smodel(for agents that set none), itsmodelSettingswithouttoolChoiceandparallelToolCalls, itscontextand sandbox, and the runner's Activity options.agentAsToolandAgentAsToolOptionsbecome deprecated aliases for one release:agentAsToolreturnsagent.asTool({ toolName, toolDescription, customOutputExtractor }). The hook that passed the sandbox to it is removed.agent.on(...)listeners on the caller's agents fire, and a customHandoffhands off to the agent its callback returns.convertAgentbecomescheckAgent, which checks the graph without changing it: raw functions intools, a missing model, andModelobjects, through handoffs andasTool()tools. It also rejectsasTool()options that would run model calls or sandbox operations in the Workflow:runConfig.modelProvider(evenundefined, which upstream spreads over the runner's provider), aModelobject asrunConfig.model, a sandbox client or live session not fromtemporalSandboxClient(), and aMemorySession.Agent.prototype.getSystemPrompt, which upstream calls once per model call, after it picks the model. This covers agents the check before the run can't see: one that a customHandoff's callback returns, or one that anagent.on(...)listener changes. Its model must be a name, and itsasTool()tools must pass the checks above. In the outer run, the run fails. In a nested run, the error goes back to the parent model as the tool's output, and the model isn't called. An agent from another copy of@openai/agents-core, such as one a dependency brings, is checked the same way: the plugin wraps that copy'sAgent.prototype.getSystemPromptbefore the agent's first model call, from the check before the run, from each run's start and from the runner'sagent_startevent.asTool()tool'srunConfigandrunOptionsonly when the nested run starts, after the tool'sparametersschema, itsinputBuilderand anyagent_tool_startlistener ran. So the plugin also checks eachRunnerrun as it starts, by wrappingRunner.prototype.runin the Workflow: the run's model provider must be the runner's, itsrunConfig.modela name, its session not aMemorySession, and its sandbox config must pass the checks above. A nested run that fails this doesn't start, and the error is the tool's output. The run then uses a frozen copy of its sandbox config, which upstream reads as it goes, so a later change to the caller's object, for example in anagent_startlistener, doesn't reach it.useLocalActivitynow fails inActivityBackedModel, not only inrun(), so a nestedasTool()run withonStreamoron()handlers is rejected too.asTool()tool returns a nested run's failure as its output, including a cancelled model Activity. If the run's cancellation scope was cancelled,run()now throws theCancelledFailureinstead of returning, and a streamed run fails with it, even when the run ended on that output, for example withtoolUseBehavior: 'stop_on_first_tool'.asTool()tool, which upstream keeps private, the plugin wrapsAgent.prototype.asToolin the Workflow before the Workflow module loads, and records each tool's agent and therunConfigandrunOptionsit was made with. The returned tool is not modified. The record is keyed by the tool'sonclosure, so copies, such as the onestoolNamespace()makes, keep it. A tool that looks like anasTool()tool but wasn't recorded, such as one made by an agent from another copy of@openai/agents-core, is rejected.ModelSummaryProviderstill gets the running agent, tracked from the innerRunner'sagent_startandagent_handoffevents. A model call made during a tool call belongs to a nestedasTool()run and getsagent: undefined, which its type allows.agent.asTool(), lists the rejected nested setups, the check before each model call (including agents from another copy of@openai/agents-core) and the check when each run starts, explains cancellation when a run ends on a tool's output and which nested failures become the tool's output, adds agent lifecycle hooks to the feature table, saysrunConfig.modelapplies to agents that set none, and adds an "Upgrading open runs" section.Migration: replace
agentAsTool(agent, { toolName, toolDescription })withagent.asTool({ toolName, toolDescription }), created in Workflow code.agentAsToolstill works for one release, as a deprecated alias that does the same.Why?
Three defects had the same root, the plugin's copies of the framework's objects:
agentAsTool, the plugin's partial copy ofasTool(), dropped the run's model, model settings and runner defaults. The framework's ownasTool()didn't run the nested agent at all: the parent model gotNo default model provider setas the tool result.agent.on(...)listeners on the caller's agents never fired.Handoff's callback returns, so the run went to the wrong agent.With a provider on the runner, upstream's code does the work, including fields it passes to nested runs later.
Breaking changes
agentAsToolandAgentAsToolOptionsare deprecated.agentAsToolnow returns anagent.asTool()tool, so it behaves as below.runConfig.modelnow applies only to agents that set no model of their own, as upstream. Before, it replaced every agent's model. Runs in progress andRunStatesaved before the upgrade follow the new rule from their next model call: an agent that sets its own model switches to it, without an error. Let those runs finish before upgrading, or use Worker Versioning. Upstream's docs sayRunConfig.modeloverrides every agent's model, but its code and tests (selectModel,test/run.test.ts) let the agent's own model win, and the plugin follows the code. The Python plugin follows the Python SDK, where the run's model wins.Nested runs behave like upstream
asTool(). A nested run's failure goes back to the parent model as the tool's output instead of failing the run. This includes model Activity failures that aren't retried, such as an authentication error, and a nestedSandboxAgentin a run withoutrunConfig.sandbox, which failed the run before. The parent model may call the tool again untilmaxTurns, and that error doesn't carry the cause. A nested approval pauses the parent run instead of failing it withNestedAgentInterruption. Nested model calls use the runner's Activity options, includinguseLocalActivityandcancellationType.A
SandboxAgentreturned by a customHandoff's callback, in a run withoutrunConfig.sandbox, fails the run with upstream's error, anAgentsWorkflowErrorwithSandboxAgent execution requires RunConfig.sandbox.An agent that a custom
Handoff's callback returns, or that anagent.on(...)listener changes, now fails the run if its model is aModelobject or one of itsasTool()tools is rejected, including an agent from another copy of@openai/agents-core. AnasTool()tool whoserunConfigorrunOptionsa listener, itsinputBuilderor itsparametersschema changes to a rejected setup doesn't start its nested run; the error is the tool's output, as upstream.An upstream
Runner, orrun(), can't run agents in a Workflow: its run is rejected unless its model provider is the plugin's. It had no Activity-backed provider before, so it failed withNo default model provider setor ran models in the Workflow. UseTemporalOpenAIRunner.A
runConfig.sandbox.sessionthat isn't from a Temporal sandbox client is rejected withSandboxConfigurationError.asTool()tools must be created in Workflow code, so the plugin can record them. Copies keep the record; a tool made by an agent from another copy of@openai/agents-coreis rejected.Replay: a passing replay test of old histories doesn't show that an upgrade is safe. Every model call is the same Activity type, and replay doesn't compare Activity inputs, so a run that now makes a different model call replays without an error and continues with the wrong answer. Runs that started before the upgrade keep working, except these:
useLocalActivity: the nested model call was a regular Activity and is now local, so replay fails with[TMPRL1100];cancellationType: ABANDON: the nested call no longer requests cancellation, so replay fails with[TMPRL1100];agent.asTool()before this change: the nested agent never ran, and now it schedules a model Activity. A run held afterrun()continues with the parent's recorded response as the nested agent's answer;agentAsToolruns whose nested run failed: the failure now goes back to the parent model. If the Workflow then did other work, such as a recovery Timer, replay fails with[TMPRL1100]. If that work was a model call, such as a recovery agent, the run replays and continues without an error, but returns the wrong answer and skips itscatchbranch;agent.on(...)listeners, or that go through a customHandoffwhose callback returns a different agent from the one it was declared with: these now take effect. A different Activity or Timer fails replay with[TMPRL1100]; a different model call doesn't;runConfig.modeland agents that set their own model, above;RunStatesaved before the upgrade in which anasTool()agent has the same name as a handoff target: the framework also findsasTool()agents by name when it restores aRunState, andagentAsTooltools weren't included, so the run can resume on the wrong agent.Let these runs finish before upgrading, or use Worker Versioning. The README's "Upgrading open runs" section says the same.
Checklist
Closes [Bug] openai-agents (TypeScript): agentAsTool runs drop the run's model, model settings and runner defaults #243, closes [Bug] openai-agents (TypeScript): agent.on(...) lifecycle hooks never run through TemporalOpenAIRunner #235, closes [Bug] openai-agents (TypeScript): TemporalOpenAIRunner ignores the agent a custom Handoff's callback returns #237. Replaces fix(openai-agents): give agentAsTool runs the run model, model settings and runner defaults #244, Run agent.on(...) lifecycle hooks in TemporalOpenAIRunner #236 and fix(openai-agents): hand off to the agent a custom Handoff returns #238.
How was this tested:
New Worker tests:
agent.asTool() runs take the run model, model settings and runner defaults;agent.on listeners run and their context writes reach later turns;A custom Handoff hands off to the agent its callback returns;agentAsToolreplaced byasTool();runConfig.model leaves an agent that sets its own model on that model, as upstream.Each of these fails without the change.
A ModelSummaryProvider gets the running agent; nested agent.asTool() calls get nonepasses before and after the clone removal, with the same labels.Tests from the review rounds, which pass on this design:
Agents picked or changed during the run are checked before they call their model. It covers aModelobject from a customHandoff's callback, a callback-returned agent with anasTool()tool that setsrunConfig.modelProvider, andagent_startlisteners in the outer and a nested run. It also covers anasTool()tool whose capturedrunConfiganagent_tool_startlistener changes to amodelProvider, tomodelProvider: undefined, or to aModelobject for a model-less nested agent; aninputBuilderthat sets amodelProvider; andasTool({ runConfig: { modelProvider: undefined } }), with a default model provider registered. Each is rejected before the nested model is called, and each replays. Without the checks, the inline model runs in the Workflow. A control, a listener that changes the tool'srunConfig.modelto another name, schedules two model Activities and the nested call uses the new name. AparametersZod refinement that sets amodelProviderwas checked the same way in a local probe (the plugin has no directzoddependency): the inline model ran without the run check and was rejected with it;Agents from another copy of @openai/agents-core are checked before they call their modelbundles a second, unmodified copy of@openai/agents-core0.14.3. It covers three ways such an agent gets aModelobject: anagent_startlistener, a customHandoff's callback, and a tool between two turns. Each is rejected before the model is called (0, 1 and 1 model Activities), and each replays. Without the fix, all three ran the inline model in the Workflow. A control agent from that copy with a string model completes through one model Activity;A sandbox client that an agent_start listener replaces with a raw one never runs: the run completes through sandbox Activities; without the frozen copy, the raw client ran in the Workflow. Theagent_tool_startvariant, on anasTool()tool'srunOptions.sandbox, is rejected as the tool's output without calling the backend. Both replay. A rawrunConfig.sandbox.sessionis rejected; without the check it ran in the Workflow;stop_on_first_tool, plain and streamed. Without the fix, both completed withActivityFailure: Activity cancelledas output. Two controls, the default loop andstop_on_first_toolwithout cancelling, pass with and without the fix;continueAsNewwith an agent that is both a handoff target and anasTool()source;onStreamand toon()handlers, including a handler registered after the run starts;useLocalActivity: the nested model isn't called, and the tool's output is theStreamingLocalActivityUnsupportedmessage. Without the check, the nested model streamed through a local Activity;asTool()options object, with a tool made at module scope before@temporalio/openai-agents/workflowloads, which the Workflow interceptor module records;asTool()tool from a second copy of@openai/agents-core, rejected before any model call, and a run-startMemorySessionset from anagent_tool_startlistener, rejected as the tool's output;SandboxAgentin a run withoutrunConfig.sandbox: its error is the tool's output, as upstream.Without the interceptor module's install, the module-scope tool's run is rejected; without the run-start
MemorySessioncheck, the nested run uses it.check-agent.test.tscovers the checks, theasTool()record (also on spread andtoolNamespace()copies), the deprecatedagentAsToolalias, the check before each model call (also on another copy'sAgentclass, installed once) and the check when each run starts (each installed once, inactive outside a Workflow).Locally, in a Linux container with Node 24 and pnpm 10, at the head: lint, which builds (the 8 warnings already on
main, none in changed code), all 270 tests, and Prettier with the SDK settings pass. The Node 22, macOS and Windows cells did not run locally.Replay check, with Workers with no Workflow cache, on runs with no run config, a run model and settings, a local-Activity runner and native
asTool():main(agentAsTool) and this change (asTool()), in both directions;asTool()runs fail with[TMPRL1100], as listed above;[TMPRL1100]in both directions; the same run held in itscatchhandler onmaincompleted on this change through the parent model, without its recovery;agent_endlistener that schedules an Activity, and a customHandoffthat picks an agent with a different Activity tool, failed with[TMPRL1100]in both directions, while their self-replays passed; they are listed above.The same histories, and runs held across versions, replay between the first commit and the clone-free code before the review-round merge. The merge doesn't change which commands are scheduled, and neither do the checks added since: they schedule nothing for valid setups.
Cancelling a Workflow during a nested model call fails it on
mainand cancels it with this change.Any docs updates needed?
The README is updated here. The docs page and the samples are updated in Show agent.asTool() instead of agentAsTool on the TypeScript OpenAI Agents page documentation#5597 and Use agent.asTool() in the OpenAI Agents agent-patterns sample samples-typescript#534, drafts until this is released.
Not covered:
getSystemPrompt. AnAgentsubclass that overrides it without callingsuperskips the check.Runner.prototype.run. Code that replacesrunon the prototype or on aRunnerinstance skips it.Handoffobject from another copy of@openai/agents-corein an agent'shandoffs: upstream's run loop doesn't recognize it (it checksinstanceof Handoff), so the run fails in upstream before any model call.