Repository navigation
fix(strands): run stateful models with their server-side state (breaking) - #285
Conversation
…ing) A worker model that keeps the conversation server-side, such as OpenAIModel with stateful: true, failed every model call through TemporalAgent. The agent's modelState store reached the model activity as a plain object, so the model could not read the last response id, and TemporalModel.stateful was always false. - The model config activity also returns the model's stateful flag, and TemporalModel.stateful reports it. - Stateful calls run as their own activities, strandsInvokeStatefulModel and strandsInvokeStatefulModelStreaming. They take a copy of the agent's model state and return the updated state with the events. Strands then clears the agent's messages after each turn, as natively. - TemporalAgent refuses a conversationManager or contextManager for a stateful model at its first invocation, as Strands does at construction. - README: stateful models, their rollout, and the model state in the continue-as-new example. Breaking: a run that used a stateful model on this version does not replay on an earlier version.
DABH
left a comment
There was a problem hiding this comment.
Review at head f6573c5 (two commits over 3af00d9). Recipe: ten finder angles, 20 raw items deduplicated to 8 candidates, one adversarial verifier per candidate with end-to-end reproductions for seven of them (a legacy-config pin across a rolling upgrade, the real OpenAIModel with a counting fetch stub, the Workflow Task failure loop and its fix-and-redeploy recovery, the #279 overflow probe, the unbound-method probe, direct TemporalModel runs, and a single-builder refactor that passes lint and all 73 tests), then a gap sweep that found nothing new. Verdicts: 6 confirmed, 2 plausible. Two sub-claims were refuted along the way and are not posted: a live proactive-compression path for the default manager, and the failure converter rebuilding ContextWindowOverflowError today. In a clean clone at the head with Strands 1.19.0 locked: eslint clean, make lint exit 0 with pnpm 10.27.0 on PATH, make test 73 of 73, conventions check OK. Not verified: a live OpenAI endpoint, the macOS, Windows and Node 22 cells, and the body's replay matrix beyond the SDK's activity-type comparison.
Severity: P1 = wrong result or stuck run in realistic use; P2 = correctness or robustness in an edge case; P3 = docs, tests, cleanup.
Where the problems cluster: the design is the right one and mirrors the Python plugin, but the two things that decide whether the feature works at all are the rollout window and one stray key in the options. A run whose config read lands on a pre-upgrade Worker is pinned stateless forever, and the real stateful provider cannot complete a single call on the locked SDK because cancelSignal still crosses the Activity boundary. The rest is documentation of behaviours the code already has, one unbound method, and test diagnostics.
PR body: it ends 'Draft while this question is open', but the PR is not marked draft.
| return activities[GET_MODEL_CONFIG_ACTIVITY]({ modelName: this.modelName }).then(({ stateful, ...config }) => { | ||
| this.config = config; | ||
| // A result recorded before the flag existed has none: the model ran as stateless then. | ||
| this.isStateful = stateful === true; |
There was a problem hiding this comment.
P2 (borderline P1): a run whose strandsGetModelConfig result was served by a pre-upgrade Worker records no stateful flag, is pinned to the stateless path for its whole lifetime, and then retries invokeModel forever with a stateful OpenAIModel even after every Worker is upgraded. Rolling upgrade on one task queue without Worker Versioning: a stateful Workflow starts during the window, its config Activity is served by a main Worker whose getModelConfig (3af00d9 model-activity.ts:113-116) returns only {modelId, contextWindowLimit}, so this line sets isStateful = false and line 90 never re-reads (one TemporalModel per agent; only Continue-As-New re-reads). Every call then takes the stateless branch (138-150) with options.modelState serialized as {"_state":{...}}, and on an upgraded Worker formatResponsesRequest (Strands 1.19.0 responses-adapter.js:50) throws TypeError: options?.modelState?.get is not a function before any request, a retryable failure under the default policy (unlimited attempts, 10 min start-to-close), so the run never completes a turn and only terminate or reset recovers it. Reproduced end to end with a legacy-config Worker then a PR-head Worker: pending invokeModel at attempt 4 with that lastFailure, nonRetryable=false, stub calls empty. The README rollout paragraph (line 102) covers only the two new Activity types and says upgrading the fleet suffices; the body's 'terminate it' covers only runs started before the upgrade; the mixed-fleet test (test-strands.ts:356-394) binds the head's getModelConfig, which already returns stateful: true (line 365), so it covers only the opposite direction, and no committed history pins the 'absent flag reads as stateless' contract. Sound mitigations: fail non-retryably in the registered stateless entry points (the plugin.ts:90-91 wrappers, not inside invokeModel, which the stateful Activities delegate to) when model.stateful is true; add the README sentence the Python README has at its line 121; commit a main-recorded history fixture replayed on the head.
There was a problem hiding this comment.
Fixed in f6fac51. invokeModel and invokeModelStreaming now fail a call for a stateful Worker model at once, with a non-retryable StatefulModelOnStatelessPath error that names the cause, instead of retrying it forever. The stateful Activities run the model through private helpers, so the check doesn't affect them.
The README's rollout paragraph (a69c4d8, a follow-up I had pending) now says a run needs its Workflow and config Activity from one version, and with f6fac51 it names this error and says to reset or terminate such runs. src/__tests__/histories/stateful-before-flag.json, recorded on main, pins that a config without the flag replays on the stateless path; it fails with [TMPRL1100] if a missing flag reads as stateful. A new test covers the pinned run: one attempt, non-retryable, and the model never runs (50 attempts without the check).
One cost, now in the PR body: an open run of a custom stateful model that never reads modelState, which kept working on the stateless path, fails at its next call after the upgrade.
| if (this.isStateful) { | ||
| // The agent's model state is a live store; send a copy of its values, and copy the updated | ||
| // values back, as the model would have written them natively. | ||
| const { modelState, ...rest } = options ?? {}; |
There was a problem hiding this comment.
P2: the new stateful branch strips only modelState from the stream options and still forwards cancelSignal (an AbortSignal, serialized to {}), so on the locked Strands 1.19.0 the only real stateful provider, OpenAIModel, fails before sending any request. Strands always sets cancelSignal: self._abortSignal (agent.js:182, :1656) and OpenAIModel forwards { signal: options?.cancelSignal } (models/openai/model.js:149, :198); the payload converter turns it into {}. Reproduced with the real OpenAIModel({stateful: true}) (openai 6.49.0, counting fetch stub, options round-tripped through the converter and the StateStore wrapping at model-activity.ts:176-177): PR-head options give APIConnectionError: Connection error. caused by TypeError: signal.addEventListener is not a function, zero fetch calls, retried under the unlimited policy; with const { modelState, cancelSignal: _unused, ...rest } = options ?? {} call one returns resp_1 and call two sends {previous_response_id: "resp_1", store: true}, so chaining works on the locked SDK. Dropping it is safe (openai client.js:534 guards if (signal); Bedrock duck-types it and {} never aborts; the Anthropic model provider never reads it; nothing in the plugin reads it) and replay-safe (sdk-core compares only Activity id and type). The stateless path (142, 149) is equally broken on main, which the body scopes out and README line 100 documents as a limitation, though its remedy (Strands below 1.15.0) contradicts the plugin's own lock and the top-level error is actually Connection error.. The new tests pass only because StatefulStub ignores cancelSignal. Better placement is Activity-side at model-activity.ts:144 and 159, or substitute Context.current().cancellationSignal so Temporal cancellation aborts the provider call; a StatefulStub assertion that cancelSignal is absent would pin the contract.
There was a problem hiding this comment.
Agreed, this blocks a real stateful OpenAIModel on the locked SDK. The same bug breaks every OpenAIModel call, stateless too, so it gets its own issue and PR: the Activity replaces the serialized cancelSignal with Temporal's own cancellation signal, on both paths. I'll link it here once it's open. This PR keeps the README limitation until that lands.
| private initializeWithConfig(): Promise<void> { | ||
| // Strands' Agent refuses these managers for a stateful model at construction, before | ||
| // TemporalAgent can read the worker model's stateful flag, so refuse them here. | ||
| if (this.model.stateful && this.managerGiven) { |
There was a problem hiding this comment.
P3 (plausible; the open question in your body): a stateful TemporalAgent keeps Strands' default SlidingWindowConversationManager, picked by super(...) at line 72 while stateful was still false, instead of NullConversationManager. agent.js:236-245 reads this.model.stateful at construction and initializeWithConfig only refuses user-given managers. Inert with the locked SDK: the Activity failure reaches Strands as ModelError{cause: ActivityFailure} (the converter overrides only errorToFailure; temporal-model.ts has no rethrow), so the overflow hook's instanceof is false, proactive compression is off and cannot be enabled, and _applyManagement runs after ModelPlugin cleared the messages (probe at the head: turn two gives ModelError, no trim). But with #279's rethrow applied to the stateful branch, call four chained to resp_2 with the tool result truncated from 1000 to 426 characters and the turn 'recovered', where native Strands (NullConversationManager, reduce returning false) fails on call three. Python swaps in NullConversationManager after the config read (_temporal_agent.py:173-187); here _conversationManager is private and already queued in _pluginRegistry. Options: say in the README's stateful paragraph that the default manager stays installed and is inert, and have #279 skip the rethrow for a stateful model or swap the manager; or a stateful?: boolean hint on TemporalAgentOptions validated against the config read; or neutralize the default instance after the read (reduce = () => false, _applyManagement = () => {}).
There was a problem hiding this comment.
Documented in the README's stateful paragraph (f6fac51): the default manager stays installed and, through the plugin, has nothing to trim. I dropped the open question and the Draft line from the body.
On the overflow interaction: #279 was folded into #200, which rethrows ContextWindowOverflowError so the agent's manager trims. Whichever of #200 and this PR lands second keeps that rethrow on the stateless calls only, so a stateful model's overflow ends the turn, as it does natively, instead of being trimmed by the default manager. With #200 it ends it with the Activity failure rather than Strands' own error.
| // Strands' Agent refuses these managers for a stateful model at construction, before | ||
| // TemporalAgent can read the worker model's stateful flag, so refuse them here. | ||
| if (this.model.stateful && this.managerGiven) { | ||
| return Promise.reject(new Error(STATEFUL_MANAGER_REFUSED)); |
There was a problem hiding this comment.
P3 (docs): STATEFUL_MANAGER_REFUSED is a plain Error, so when Workflow code does not catch it the run does not fail; the Workflow Task fails and is retried about every 10 s until the manager is removed and the Worker redeployed, and README line 98 ('throws at the agent's first invocation') could say so. internals.js:1051-1072 fails the run only for a TemporalFailure or a configured type; Strands' stream() (agent.js:663-668) awaits initialize() inside its try and propagates the rejection unwrapped. Reproduced: a Workflow passing conversationManager with a stateful stub and no try/catch stayed RUNNING with WorkflowTaskFailed at attempt 4; flipping the factory back does not help because the recorded config says stateful; redeploying with the manager dropped completed the stuck runs without termination. That is the SDK's contract and matches the pre-existing SNAPSHOT_DISABLED and RETRY_STRATEGY_DISABLED guards and the Python plugin's plain ValueError. Do not switch to a non-retryable ApplicationFailure: it would permanently fail every in-flight run at first invocation where today they finish once the manager is dropped. The PR's test covers only the caught path (workflows/strands.ts:91-98; test-strands.ts:349). Suggested clause: catch it, or the uncaught error fails the workflow task until the manager is removed and the worker redeployed.
There was a problem hiding this comment.
Added to the README (f6fac51): catch the error, or the Workflow Task fails and is retried until the manager is removed and the Worker redeployed. It stays a plain Error, as you suggest.
| ...this.activityOptions, | ||
| }); | ||
|
|
||
| if (this.isStateful) { |
There was a problem hiding this comment.
P3 (docs): isStateful is set only by loadConfig(), which only TemporalAgent.initialize calls, so the exported TemporalModel used with a plain Strands Agent never reads the config and routes a stateful Worker model through invokeModel, failing every call. new Agent({ model: new TemporalModel({ modelName: 'openai' }) }) with a stateful factory schedules no strandsGetModelConfig, the Activity input carries modelState: {"_state":{}}, and formatResponsesRequest throws modelState.get is not a function on every attempt. Reproduced in a scratch copy: the stateless stub works on the direct path; the stateful stub without loadConfig fails as described; with await model.loadConfig() before new Agent(...) the run chains ids and gets NullConversationManager. The direct path already skipped the config read at the base (since #158), the README never mentions TemporalModel, and awaiting loadConfig() inside stream() would add an undocumented TMPRL1100 break for direct-path histories and land after Strands' constructor-time checks, so a sentence in the class doc (lines 38-46), and optionally 'through TemporalAgent' at README line 98, is the right outcome.
There was a problem hiding this comment.
Added to the class doc (f6fac51): with a plain Strands Agent, call await model.loadConfig() before building it. The README now says "through TemporalAgent". With the change above, such a call now fails at once with a message that names this case, instead of retrying.
| // stateful support takes it. | ||
|
|
||
| /** Run a stateful model with the agent's model state; return its events and the updated state. */ | ||
| async invokeStatefulModel(input: InvokeStatefulModelInput): Promise<InvokeStatefulModelResult> { |
There was a problem hiding this comment.
P3: invokeStatefulModel and invokeStatefulModelStreaming (182) are plain prototype methods while invokeModel and invokeModelStreaming are bound in the constructor (109-110), so registering them by reference, the pattern the PR's own test uses for the bound pair (test-strands.ts:375-376), throws on the first call because the Worker invokes Activities with no receiver. Probe through MockActivityEnvironment and the Worker's real Activity.execute: strandsInvokeStatefulModel: modelActivity.invokeStatefulModel gives TypeError: Cannot read properties of undefined (reading 'invokeModel') at lib/model-activity.js:110, retried as a plain retryable failure; getModelConfig is already unbound on main, which is why the test binds it explicitly (374). No supported path reaches this (ModelActivity is not exported; StrandsPlugin registers arrow wrappers at plugin.ts:90-95). If fixed, bind only; wrapping them in autoHeartbeat again doubles the heartbeat timers (12 versus 6 in the probe) because the inner call already goes through the wrapped invokeModel.
There was a problem hiding this comment.
Fixed in f6fac51: all four model Activities are bound and heartbeat-wrapped once in the constructor. The stateful ones no longer go through invokeModel, which now refuses stateful models, so nothing is wrapped twice.
| // values back, as the model would have written them natively. | ||
| const { modelState, ...rest } = options ?? {}; | ||
| const input = { modelName: this.modelName, messages, options: rest, modelState: modelState?.getAll() ?? {} }; | ||
| const result = this.streamingTopic |
There was a problem hiding this comment.
P3 (cleanup): stream() now dispatches through four near-identical Activity calls that write the {modelName, messages, options} input three times with two different options treatments and two yield idioms. Lines 121-129 build input once for the stateful pair (options: rest), while 139-150 repeat two full literals for the stateless pair (options whole, so the Activity receives modelState: {"_state":{...}} and cancelSignal: {}), followed by yield* result.events; return; versus for (const event of events) yield event. The next input-shape change (dropping cancelSignal, above) must be edited at 121, 142 and 149. A form that destructures once, builds input and the streaming variant once, and selects among the four Activities with one-line calls was applied to a copy: tsc, eslint and 73 of 73 tests pass. A literal single union call is not type-safe without over-supplying fields to the stateless Activities, so four one-line calls over one builder is the honest form; options: this.isStateful ? rest : options keeps the stateless input byte-identical if that is wanted.
There was a problem hiding this comment.
Done in f6fac51: stream() destructures once and builds one input, with options unchanged on the stateless path.
| }, | ||
| }); | ||
| const pending = await legacyWorker.runUntil(async () => { | ||
| for (;;) { |
There was a problem hiding this comment.
P3 (plausible, test diagnostics): the rollout test's for (;;) describe-poll exits only when a pending Activity reaches attempt > 2, so any other outcome (a different pending Activity type, an empty pendingActivities because the run ended, no NotFoundError produced) spins until AVA's 60 s timeout with no assertion diagnostic. With STATEFUL_ACTIVITY_OPTIONS retrying every 100 ms (backoff 1, 50 attempts) the observable window runs from about 200 ms to about 5 s, which the 100 ms poll cannot realistically miss; but if the legacy Worker ever registered the stateful Activity or the run ended early, legacyWorker.runUntil would never settle. Breaking when (await handle.describe()).status.name !== 'RUNNING', or capping the iterations and failing with the last describe, makes such failures immediate without weakening the test.
There was a problem hiding this comment.
Fixed in f6fac51: the loop now throws with the run's status and pending Activities when the run leaves RUNNING, or after 300 polls (30 s).
The stateful path is chosen from the flag the config activity returns, so a run needs its workflow and config activity from the same version. Say so in the README, with the rollout that keeps them together, and what changes for a custom model that reports stateful but ignores modelState. Correct the comment on the test whose activity worker lacks the stateful activities.
A run whose config read had no stateful flag, from a worker without stateful support, stays on the stateless path, and a model that reads modelState then failed every attempt with a retryable error, forever. invokeModel and invokeModelStreaming now fail such a call at once with a non-retryable StatefulModelOnStatelessPath error that names the cause. The stateful activities run the model through private helpers instead, and all four model activities are bound and heartbeat once. Also from review: - stream() builds the activity input once. - README: stateful models work through TemporalAgent; catch the manager refusal or the workflow task retries; the default manager stays installed and has nothing to trim; the rollout paragraph names the new error. - TemporalModel's doc says to call loadConfig() with a plain Agent. - A history recorded on main, replayed here, pins that a config without the flag reads as stateless. - The rollout test's poll loop stops with a message when the run ends.
|
Thanks for the review, @DABH. a69c4d8 and f6fac51 address it; replies are in the threads, and the PR body is updated.
|
DABH
left a comment
There was a problem hiding this comment.
Second round, at head f6fac51 (two commits over f6573c5: a69c4d8 rewrites the README rollout paragraph; f6fac51 adds the non-retryable stateless-path guard, binds the stateful Activities, builds one input in stream(), and adds the history fixture and test fixes). In a worktree with Strands 1.19.0 locked and pnpm 10.27.0: make sync, make lint clean, make test 75 of 75, conventions check OK. CI is green on all four cells. I verified the findings with three temporary probe tests against the dev server, since removed: a config recorded with stateful: true followed by a stateless factory, a config recorded with stateful: false followed by a stateful factory, and the pinned run under a workflow that does not catch. Checked and not raised: Strands' other construction-time stateful check (delegation tools) runs in AgentDelegation.initAgent during initialize(), after the config read, so it still fires. #202 builds its duplicate check from the plugin's activity map, so it picks up the two new names. #297 edits lines that now live in runModel and runModelStreaming, so it needs a rebase, after which it covers both paths as you say. Not verified: a live OpenAI endpoint, and the body's mixed-version matrix beyond what the committed history pins.
Severity: P1 = wrong result or stuck run in realistic use; P2 = correctness or robustness in an edge case; P3 = docs, tests, cleanup.
Previous rounds. Addressed:
- A pinned-stateless run now fails at once with a non-retryable
StatefulModelOnStatelessPath: one attempt, no model call, pinned by the new test. The README states the one-version-per-run rule and that such runs need a reset or terminate, andstateful-before-flag.jsonpins that a config without the flag replays on the stateless path. - The stateful Activities are bound and heartbeat once, through
runModelinstead of the guardedinvokeModel. stream()builds one input and keeps the stateless input unchanged.- The class doc tells plain-
Agentusers to callloadConfig(), and the README says 'throughTemporalAgent'. - The README covers the inert default manager and the uncaught manager refusal.
- The rollout test's poll loop throws with the run status, and the Draft line is gone.
Accepted decline: cancelSignal moves to #296 and #297, because it breaks stateless OpenAIModel calls on main too, and the README limitation stays until then.
Where the problems cluster: the new guard fails fast on one direction of a model and config mismatch. The other direction (a recorded stateful config, with a model that is now stateless) still completes silently, sending only each turn's new message. Neither the guard's message nor the README names the realistic trigger for either direction: switching a model between stateful and stateless while runs are open.
Approving, because no P1 or blocking item remains and CI is green. The non-blocking P2 is a short mirror check that I would still add before merge.
| /** Run a stateful model with the agent's model state; return its events and the updated state. */ | ||
| async invokeStatefulModel(input: InvokeStatefulModelInput): Promise<InvokeStatefulModelResult> { | ||
| const modelState = new StateStore(input.modelState); | ||
| const model = this.getModel(input.modelName); |
There was a problem hiding this comment.
P2 (non-blocking): the stateful Activities run whatever model the factory now returns without checking that it is still stateful, so a run whose config recorded stateful: true silently loses its context once the factory turns stateless, for example OpenAIModel redeployed with stateful: false while chats are open.
Because the recorded config still says stateful, TemporalModel.stateful stays true and Strands' ModelPlugin clears agent.messages after each turn. The next call therefore gives the now-stateless model only the new message. That model ignores modelState, so OpenAIModel sends neither previous_response_id nor the history (responses-adapter.js:37-54 with stateful false), and nothing fails.
Probe at this head: statefulAgent('models-queue'), with the models queue serving strandsGetModelConfig: async () => ({ stateful: true }) and strandsInvokeStatefulModel from a ModelActivity whose factory returns a stateless recording stub. The run completes with [0, 0], and the model saw 1 message on both calls. Natively, a stateless agent sends 1 and then 3. The opposite mismatch now fails at once with a named error, but this one runs to completion with the wrong context. The stateful tests use the same StatefulStub for the config and the call, so no test covers the mismatch.
Fix:
- Add the mirror check here and at line 216: for example, a
getStatefulModelthat throws a non-retryableApplicationFailurewhen!model.stateful, with its own type or message. - Add a test shaped like the pinned-run one.
- Add a README clause saying that switching a model between stateful and stateless while runs are open needs the same reset or terminate.
There was a problem hiding this comment.
Added in 6c7d8d6. invokeStatefulModel and invokeStatefulModelStreaming now get the model through getStatefulModel, which fails the call at once with a non-retryable StatelessModelOnStatefulPath when the model isn't stateful. The pinned-run test is now a two-case table, and its new case is your shape: a { stateful: true } config, a model that became stateless, one attempt, the typed failure, and no model call. That case fails on f6fac51, where the stateless model runs on the stateful path. The README's rollout paragraph now says that changing a model between stateful and stateless applies only to new runs, and those runs need a reset or terminate.
| nonRetryable: true, | ||
| message: | ||
| `The worker model ${JSON.stringify(name ?? this.defaultName)} is stateful, but the workflow ` + | ||
| `called it without its model state. Its config was read from a worker without stateful ` + |
There was a problem hiding this comment.
P3: the guard's message names two causes, a config read from a worker without stateful support and TemporalModel used without TemporalAgent, but a run whose config was read on this version also hits it when the factory becomes stateful afterwards, for example when stateful: true is added to a deployed OpenAIModel while chats are open.
Probe: the config activity returns { stateful: false }, and then invokeModel is served by a ModelActivity with a stateful model. Result: one attempt, StatefulModelOnStatelessPath, and this message. An operator who has just enabled stateful mode, with every worker on this version, is sent looking for an old worker that does not exist. A stateful model that uses modelState never worked through the plugin before, so enabling it on a deployed model is the likely first way users meet this error.
Fix: add the third cause here and in README line 102, for example 'or the worker model became stateful after the run read its config'. Also say that enabling stateful mode applies only to new runs, as in the rollout.
There was a problem hiding this comment.
Done in 6c7d8d6. The message and the doc comment now also name a worker model that became stateful after the run read its config, and the README says that enabling or disabling stateful mode applies only to new runs. The test checks that each message names its cause.
|
|
||
| `OpenAIModel` calls through `TemporalAgent`, stateful or not, currently work only with `@strands-agents/sdk` releases before 1.15.0. From 1.15.0, Strands passes an `AbortSignal` in the model's stream options, the signal reaches the model activity as a plain object, and the OpenAI client fails with `signal.addEventListener is not a function` before it sends a request. Temporal retries that activity until its retry policy or timeout runs out. | ||
|
|
||
| Stateful model calls run as their own activities, `strandsInvokeStatefulModel` and `strandsInvokeStatefulModelStreaming`, chosen from the stateful flag that `strandsGetModelConfig` returns, so a run that uses a stateful model needs the workflow and the config activity from the same version. An earlier workflow that reads this version's config schedules `invokeModel`, and this version then fails to replay the run with a nondeterminism error. This version's workflow that reads an earlier config, which has no flag, keeps the stateless path for the rest of the run, and an upgraded worker fails each of its model calls with a non-retryable `StatefulModelOnStatelessPath` error. Neither run recovers: reset or terminate it. Roll out with Worker Versioning, keeping each run's workflow and activity workers on one version, including any task queue set in `activityOptions`. Without it, stop the runs that use a stateful model, replace every worker on those task queues, and then start new runs. |
There was a problem hiding this comment.
P3: 'an upgraded worker fails each of its model calls with a non-retryable StatefulModelOnStatelessPath error' reads as if the run fails, but unless the workflow catches the agent's error, the run stays Running with a failing workflow task, and the new type is two causes down.
Strands' Model.streamAggregated wraps the Activity failure in a plain ModelError (models/model.js:350-355), and the workflow SDK treats that as a task failure. Probe at this head: the pinned-run setup with a workflow that does not catch. After 8 s the status is RUNNING, with two WorkflowTaskFailed events whose message is Activity task failed, whose cause is Activity task failed, and only below that StatefulModelOnStatelessPath with the guard's message. A workflow that does catch sees err.message === 'Activity task failed'; the new test's workflow returns exactly that string. The PR body's 'ends a run the same way' holds only once #200 unwraps Temporal failures.
Fix: add a clause like the one in the manager paragraph. It should say that the run stays open with a failing workflow task until you reset or terminate it, and that the error type is on the ModelError's cause.cause. Operators can then find these runs and catch the error by type.
There was a problem hiding this comment.
Done in 6c7d8d6. The rollout paragraph now says that, unless the workflow catches the agent's error, the run stays open: the error reaches the workflow as a ModelError with the type on its cause.cause, and the workflow task fails on each attempt. I also corrected the body: the errors end a run the same way only with #200; without it the run stays open, as the README says.
…ateful A run whose config read recorded a stateful model keeps the stateful path, and its agent clears its messages after each turn. If the worker model became stateless since, for example an OpenAIModel redeployed with stateful: false, the stateful activities ran it with only each turn's new message and nothing failed. - The stateful activities now fail such a call at once with a non-retryable StatelessModelOnStatefulPath error. - The StatefulModelOnStatelessPath message also names a model that became stateful after the run read its config. - README: changing a model between stateful and stateless applies only to new runs, and an uncaught error leaves the run open with a failing workflow task, with the type on the ModelError's cause.cause. - The pinned-run test covers both directions.
Conflicts: - temporal-model.ts: #200's context-overflow rethrow stays around the stateless model calls only. A stateful model's overflow is not turned into Strands' error, so the default manager the agent keeps does not trim and retry a server-side conversation. - README.md: #200 moved the model config note to "Upgrading from 1.24 and earlier". With #200, a stateful/stateless mismatch error reaches invoke() as an ActivityFailure, so an uncaught one fails the run. - test-strands.ts and workflows/strands.ts: both sides' imports and tests kept.
What was changed
A Worker model that keeps the conversation server-side, such as
new OpenAIModel({ stateful: true }),now works through
TemporalAgentas it does natively. Before, every model call with it failed inthe model Activity (
options?.modelState?.get is not a function), because the agent'smodelStatestore reached the Activity as a plain object.
This is the TypeScript side of #196, with the same design.
statefulflag, andTemporalModel.statefulreports it. A config result recorded before this change has no flag andreads as stateless.
strandsInvokeStatefulModelandstrandsInvokeStatefulModelStreaming. They take a copy of the agent's model state, give the modela
StateStorewith it, and return the events with the updated state.TemporalModelcopies thatstate into the store Strands passed, and Strands copies it into
agent.modelState. Strands' modelplugin then clears the agent's messages after each turn, as natively. Stateless calls keep
invokeModelandinvokeModelStreamingunchanged.of running the model without its state, and Temporal retries the call on another Worker. This does
not make a rolling upgrade safe (see Breaking change).
invokeModelandinvokeModelStreamingfail a call for a stateful Worker model at once, with anon-retryable
StatefulModelOnStatelessPatherror that names the cause, instead of retrying a callthat cannot get the model state. This happens when the run read its config from a Worker without
stateful support, when the Worker model became stateful after the run read its config, or when
TemporalModelruns with a plain StrandsAgentthat never read the config.StatelessModelOnStatefulPatherror when the Worker model is no longer stateful, for example anOpenAIModelredeployed withstateful: falsewhile a run that read a stateful config is open.Before, that run went on with each turn's new message only, because its agent clears its messages
after each turn.
TemporalAgentthrows Strands' own error when aconversationManagerorcontextManageris givenwith a stateful model. Natively this happens at construction;
TemporalAgentlearns the flag whenit reads the config, so it throws at the agent's first invocation.
installed), the changes for existing custom stateful models, and the one-version-per-run rollout,
and
agent.modelStatein the continue-as-new example.TemporalModel's doc says to callloadConfig()before building a plainAgent.modelStateis already anAgentConfigoption,so there is no new API.
Why?
See #284.
Breaking change
OpenAIModel({ stateful: true })failed every call before this change, so it has no workingcallers to break.
statefulbut never readsmodelStateworked before, statelessly,with the agent keeping its messages. Now an agent that gives it a
conversationManagerorcontextManagerthrows, and without one the messages are cleared after each turn, so the modelgets only the new message. To keep the earlier behavior, report
statefulas false.agent.messagesloses a stateful model's context; itmust also carry
agent.modelState.before the upgrade with a stateful model keeps its recorded config, without the flag, so it stays
on the stateless path, and its next model call now fails at once with
StatefulModelOnStatelessPath. That includes a custom stateful model that never readsmodelState, whose open runs kept working before; let those runs finish before you upgrade, orreset or terminate them.
A run that used a stateful model on this version does not replay on an earlier version: it fails
with
[TMPRL1100](strandsInvokeStatefulModelwhere the earlier version schedulesinvokeModel). Stateless runs replay in both directions.model needs the Workflow and the config Activity from one version. An earlier Workflow that reads
this version's config schedules
invokeModel; a fixed Worker then fails the run's next WorkflowTask with
[TMPRL1100], and the run staysRUNNING. A fixed Workflow that reads an earlierconfig stays on the stateless path for the rest of the run, and a fixed Worker fails its model calls
at once with
StatefulModelOnStatelessPath. Neither run recovers; reset or terminate it.new runs: an open run keeps the mode its config recorded, and its model calls fail at once with
StatefulModelOnStatelessPathorStatelessModelOnStatefulPath. Reset or terminate those runs.The title says it is breaking because release notes come from the squash commit subject.
Upgrade: roll out with Worker Versioning, keeping each run's Workflow and Activity Workers on one
version, including any task queue set in
activityOptions. Without it, stop the runs that use astateful model, replace every Worker on those task queues, and then start new runs. Do not roll back
while stateful runs are open; keep them on this version with Worker Versioning, or let them finish.
Checklist
Closes [Bug] strands-agents (TypeScript): a stateful OpenAIModel fails every model call through the plugin #284
How was this tested:
test-strands.ts. The first three fail onmainand pass with this change:turn, the second call chained to the first response id, and the history replays; a
conversationManageris refused. Onmainthe run fails ('Activity task failed').config: the call waits until a Worker with them takes it, then chains. On
mainthe pendingActivity is
invokeModel. This is not a mixed-version test; its Worker serves thisversion's config.
no flag and a stateful model, the call reaches
invokeModeland fails once with thenon-retryable
StatefulModelOnStatelessPath; with a stateful config and a model that becamestateless, the call reaches
strandsInvokeStatefulModeland fails once withStatelessModelOnStatefulPath. The model never runs, and each message names its cause.Without the second check, the stateless model runs on the stateful path.
main(src/__tests__/histories/stateful-before-flag.json, whose confighas no flag) replays on this change. With a missing flag read as stateful it fails with
[TMPRL1100].OpenAIModeland a recordingfetch, and@strands-agents/sdk1.6.0: it differs from native on 1.24.0 andmain, and matches native withthis change: the second request carries
previous_response_id: "resp_1"and only the newmessage, and the agent has 0 messages after each turn. The stateless control matches native on
all three.
streaming stateful runs:
mainreplay on this change; open ones finish on it, and thestateful ones end their next turn with
StatefulModelOnStatelessPath;turn 1's response id;
main, this change's stateless runs replay and finish; its stateful runs fail with[TMPRL1100], and an open one stops with that error on its next Workflow Task;[TMPRL1100].Workflow with an earlier config is the case the new pinned-run test covers.
with this version's config replays on the earlier code, then fails on a fixed Worker with
[TMPRL1100]and staysRUNNING; a fixed Workflow with an earlier config fails both turnsafter every Activity Worker is fixed; fixed Workers throughout chain both turns. With a custom
stateful model that never reads
modelState,mainends with[2, 4]messages with andwithout a manager; this change refuses the manager and ends the manager-free run with
[0, 0].make sync,make lint,make test(85 tests after mergingmainwith fix(strands): keep the caller's failure converter and turn Strands errors into Temporal failures (breaking) #200),make build, the npm package check, thesmoke install and
check_conventions.pypass. With a 4 GB container limit, the merged testfile was killed for memory; it passes with 8 GB. The Node 22, macOS and Windows cells did not
run locally.
main85b968a: its histories replay on thisbranch, and this branch's stateful runs fail on
mainwith[TMPRL1100], as before.Any docs updates needed?
The plugin README has two new paragraphs and an updated continue-as-new example. Release notes
must give the custom stateful model changes, that a run started before the upgrade with a stateful
model fails at its next model call, the one-version-per-run rollout, that changing a model between
stateful and stateless applies only to new runs, and that a run that used a stateful model does
not replay on an earlier version.
Not covered
@strands-agents/sdk1.19.0 (the locked version) and 1.22.0, anOpenAIModelcall throughthe plugin fails before its request whether or not the model is stateful. Strands passes an
AbortSignal(cancelSignal) in the stream options, it reaches the Activity as a plain object,and the OpenAI client fails with
signal.addEventListener is not a function. That is a separatebug, [Bug] strands-agents (TypeScript): OpenAIModel calls fail in the model Activity on Strands 1.15.0 and later #296, fixed for both paths in fix(strands): give the model the activity's cancellation signal #297. Until that lands, a stateful
OpenAIModelworks through the plugin only with Strandsreleases that do not pass
cancelSignal; with 1.19.0 and 1.22.0 this change gets past the modelstate and then hits that error.
NullConversationManager. Strands picks themanager at construction, before
TemporalAgentcan read the flag, so a statefulTemporalAgentkeeps Strands' default sliding window manager; the README says so. Through the plugin it has
nothing to trim: Strands clears the messages after each turn before the manager runs, and a context
window overflow reaches the agent as a model error. fix(strands): keep the caller's failure converter and turn Strands errors into Temporal failures (breaking) #200, merged into this branch from
main,rethrows
ContextWindowOverflowErrorso the agent's manager trims; the merge keeps that rethrow onthe stateless calls only, so a stateful model's overflow ends the turn instead of being trimmed.
TemporalAgent.stream()onmainalso rethrows the Temporal failure inside a StrandsModelError, so a model Activity that failed or was cancelled fails or cancels the run (or theUpdate), unless the workflow catches it. A stateful model's overflow ends the turn with its
Activity failure rather than Strands' own error, and this PR's
StatefulModelOnStatelessPathandStatelessModelOnStatefulPatherrors reachinvoke()as anActivityFailureand end a run thesame way, as the README says.