Skip to content

fix(strands): run stateful models with their server-side state (breaking) - #285

Merged
campoy merged 6 commits into
mainfrom
fix/strands-typescript-stateful-model-state-lost
Oct 31, 2026
Merged

campoy merged 6 commits into
mainfrom
fix/strands-typescript-stateful-model-state-lost

Conversation

@campoy

@campoy campoy commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What was changed

A Worker model that keeps the conversation server-side, such as new OpenAIModel({ stateful: true }),
now works through TemporalAgent as it does natively. Before, every model call with it failed in
the model Activity (options?.modelState?.get is not a function), because the agent's modelState
store reached the Activity as a plain object.

This is the TypeScript side of #196, with the same design.

  • The model config Activity from fix(strands): give Strands the worker model's id and context window (breaking) #158 also returns the model's stateful flag, and
    TemporalModel.stateful reports it. A config result recorded before this change has no flag and
    reads as stateless.
  • A stateful model's calls run as their own Activities, strandsInvokeStatefulModel and
    strandsInvokeStatefulModelStreaming. They take a copy of the agent's model state, give the model
    a StateStore with it, and return the events with the updated state. TemporalModel copies that
    state into the store Strands passed, and Strands copies it into agent.modelState. Strands' model
    plugin then clears the agent's messages after each turn, as natively. Stateless calls keep
    invokeModel and invokeModelStreaming unchanged.
  • An Activity Worker without those Activity types fails a stateful call as not registered instead
    of running the model without its state, and Temporal retries the call on another Worker. This does
    not make a rolling upgrade safe (see Breaking change).
  • invokeModel and invokeModelStreaming fail a call for a stateful Worker model at once, with a
    non-retryable StatefulModelOnStatelessPath error that names the cause, instead of retrying a call
    that cannot get the model state. This happens when the run read its config from a Worker without
    stateful support, when the Worker model became stateful after the run read its config, or when
    TemporalModel runs with a plain Strands Agent that never read the config.
  • In the other direction, the stateful Activities fail a call at once with a non-retryable
    StatelessModelOnStatefulPath error when the Worker model is no longer stateful, for example an
    OpenAIModel redeployed with stateful: false while a run that read a stateful config is open.
    Before, that run went on with each turn's new message only, because its agent clears its messages
    after each turn.
  • TemporalAgent throws Strands' own error when a conversationManager or contextManager is given
    with a stateful model. Natively this happens at construction; TemporalAgent learns the flag when
    it reads the config, so it throws at the agent's first invocation.
  • README: paragraphs on stateful models (catching the manager error, the default manager that stays
    installed), the changes for existing custom stateful models, and the one-version-per-run rollout,
    and agent.modelState in the continue-as-new example. TemporalModel's doc says to call
    loadConfig() before building a plain Agent. modelState is already an AgentConfig option,
    so there is no new API.

Why?

See #284.

Breaking change

  • OpenAIModel({ stateful: true }) failed every call before this change, so it has no working
    callers to break.
  • A custom model that reports stateful but never reads modelState worked before, statelessly,
    with the agent keeping its messages. Now an agent that gives it a conversationManager or
    contextManager throws, and without one the messages are cleared after each turn, so the model
    gets only the new message. To keep the earlier behavior, report stateful as false.
  • A continue-as-new handoff that carries only agent.messages loses a stateful model's context; it
    must also carry agent.modelState.
  • Replay: no version gate. Runs recorded entirely before this change replay on it. A run started
    before the upgrade with a stateful model keeps its recorded config, without the flag, so it stays
    on the stateless path, and its next model call now fails at once with
    StatefulModelOnStatelessPath. That includes a custom stateful model that never reads
    modelState, whose open runs kept working before; let those runs finish before you upgrade, or
    reset or terminate them.
    A run that used a stateful model on this version does not replay on an earlier version: it fails
    with [TMPRL1100] (strandsInvokeStatefulModel where the earlier version schedules
    invokeModel). Stateless runs replay in both directions.
  • Mixed versions: the config result picks the Workflow's commands, so a run that uses a stateful
    model needs the Workflow and the config Activity from one version. An earlier Workflow that reads
    this version's config schedules invokeModel; a fixed Worker then fails the run's next Workflow
    Task with [TMPRL1100], and the run stays RUNNING. A fixed Workflow that reads an earlier
    config stays on the stateless path for the rest of the run, and a fixed Worker fails its model calls
    at once with StatefulModelOnStatelessPath. Neither run recovers; reset or terminate it.
  • Changing a Worker model between stateful and stateless while runs are open also applies only to
    new runs: an open run keeps the mode its config recorded, and its model calls fail at once with
    StatefulModelOnStatelessPath or StatelessModelOnStatefulPath. Reset or terminate those runs.

The title says it is breaking because release notes come from the squash commit subject.

Upgrade: roll out with Worker Versioning, keeping each run's Workflow and Activity Workers on one
version, including any task queue set in activityOptions. Without it, stop the runs that use a
stateful model, replace every Worker on those task queues, and then start new runs. Do not roll back
while stateful runs are open; keep them on this version with Worker Versioning, or let them finish.

Checklist

  1. Closes [Bug] strands-agents (TypeScript): a stateful OpenAIModel fails every model call through the plugin #284

  2. How was this tested:

    • Five new tests in test-strands.ts. The first three fail on main and pass with this change:
      • two turns with a stateful stub model, with and without streaming: 0 messages after each
        turn, the second call chained to the first response id, and the history replays; a
        conversationManager is refused. On main the run fails ('Activity task failed').
      • an Activity Worker without the stateful Activities, behind this version's Workflow and
        config: the call waits until a Worker with them takes it, then chains. On main the pending
        Activity is invokeModel. This is not a mixed-version test; its Worker serves this
        version's config.
      • the README's continue-as-new handoff carries the response id to the next run.
      • a model whose stateful flag differs from the run's config, both ways: with a config that has
        no flag and a stateful model, the call reaches invokeModel and fails once with the
        non-retryable StatefulModelOnStatelessPath; with a stateful config and a model that became
        stateless, the call reaches strandsInvokeStatefulModel and fails once with
        StatelessModelOnStatefulPath. The model never runs, and each message names its cause.
        Without the second check, the stateless model runs on the stateful path.
      • a history recorded on main (src/__tests__/histories/stateful-before-flag.json, whose config
        has no flag) replays on this change. With a missing flag read as stateful it fails with
        [TMPRL1100].
    • [Bug] strands-agents (TypeScript): a stateful OpenAIModel fails every model call through the plugin #284's reproduction, with the real OpenAIModel and a recording fetch, and
      @strands-agents/sdk 1.6.0: it differs from native on 1.24.0 and main, and matches native with
      this change: the second request carries previous_response_id: "resp_1" and only the new
      message, and the agent has 0 messages after each turn. The stateless control matches native on
      all three.
    • Replay check on a dev server, Workers with a Workflow cache of 0, for stateless, stateful and
      streaming stateful runs:
      • finished and open runs from main replay on this change; open ones finish on it, and the
        stateful ones end their next turn with StatefulModelOnStatelessPath;
      • this change's runs replay on it, and a new Worker finishes its open runs, chaining turn 2 to
        turn 1's response id;
      • on main, this change's stateless runs replay and finish; its stateful runs fail with
        [TMPRL1100], and an open one stops with that error on its next Workflow Task;
      • a control with one extra Activity fails with [TMPRL1100].
    • The replay check was rerun on this head; the mixed-version check below was not. Its fixed
      Workflow with an earlier config is the case the new pinned-run test covers.
    • Mixed-version check, with the model Activities on their own task queue: an earlier Workflow
      with this version's config replays on the earlier code, then fails on a fixed Worker with
      [TMPRL1100] and stays RUNNING; a fixed Workflow with an earlier config fails both turns
      after every Activity Worker is fixed; fixed Workers throughout chain both turns. With a custom
      stateful model that never reads modelState, main ends with [2, 4] messages with and
      without a manager; this change refuses the manager and ends the manager-free run with [0, 0].
    • Local, in a Linux container with Node 24.22.0 and pnpm 10.27.0: make sync, make lint,
      make test (85 tests after merging main with fix(strands): keep the caller's failure converter and turn Strands errors into Temporal failures (breaking) #200), make build, the npm package check, the
      smoke install and check_conventions.py pass. With a 4 GB container limit, the merged test
      file was killed for memory; it passes with 8 GB. The Node 22, macOS and Windows cells did not
      run locally.
    • After the merge, the replay check passes against main 85b968a: its histories replay on this
      branch, and this branch's stateful runs fail on main with [TMPRL1100], as before.
    • Not run against a real model provider.
  3. Any docs updates needed?
    The plugin README has two new paragraphs and an updated continue-as-new example. Release notes
    must give the custom stateful model changes, that a run started before the upgrade with a stateful
    model fails at its next model call, the one-version-per-run rollout, that changing a model between
    stateful and stateless applies only to new runs, and that a run that used a stateful model does
    not replay on an earlier version.

Not covered

  • With @strands-agents/sdk 1.19.0 (the locked version) and 1.22.0, an OpenAIModel call through
    the plugin fails before its request whether or not the model is stateful. Strands passes an
    AbortSignal (cancelSignal) in the stream options, it reaches the Activity as a plain object,
    and the OpenAI client fails with signal.addEventListener is not a function. That is a separate
    bug, [Bug] strands-agents (TypeScript): OpenAIModel calls fail in the model Activity on Strands 1.15.0 and later #296, fixed for both paths in fix(strands): give the model the activity's cancellation signal #297. Until that lands, a stateful OpenAIModel works through the plugin only with Strands
    releases that do not pass cancelSignal; with 1.19.0 and 1.22.0 this change gets past the model
    state and then hits that error.
  • Natively, a stateful agent with no manager uses NullConversationManager. Strands picks the
    manager at construction, before TemporalAgent can read the flag, so a stateful TemporalAgent
    keeps Strands' default sliding window manager; the README says so. Through the plugin it has
    nothing to trim: Strands clears the messages after each turn before the manager runs, and a context
    window overflow reaches the agent as a model error. fix(strands): keep the caller's failure converter and turn Strands errors into Temporal failures (breaking) #200, merged into this branch from main,
    rethrows ContextWindowOverflowError so the agent's manager trims; the merge keeps that rethrow on
    the stateless calls only, so a stateful model's overflow ends the turn instead of being trimmed.
    TemporalAgent.stream() on main also rethrows the Temporal failure inside a Strands
    ModelError, so a model Activity that failed or was cancelled fails or cancels the run (or the
    Update), unless the workflow catches it. A stateful model's overflow ends the turn with its
    Activity failure rather than Strands' own error, and this PR's StatefulModelOnStatelessPath and
    StatelessModelOnStatefulPath errors reach invoke() as an ActivityFailure and end a run the
    same way, as the README says.

…ing)

A worker model that keeps the conversation server-side, such as
OpenAIModel with stateful: true, failed every model call through
TemporalAgent. The agent's modelState store reached the model activity
as a plain object, so the model could not read the last response id,
and TemporalModel.stateful was always false.

- The model config activity also returns the model's stateful flag, and
  TemporalModel.stateful reports it.
- Stateful calls run as their own activities, strandsInvokeStatefulModel
  and strandsInvokeStatefulModelStreaming. They take a copy of the
  agent's model state and return the updated state with the events.
  Strands then clears the agent's messages after each turn, as natively.
- TemporalAgent refuses a conversationManager or contextManager for a
  stateful model at its first invocation, as Strands does at
  construction.
- README: stateful models, their rollout, and the model state in the
  continue-as-new example.

Breaking: a run that used a stateful model on this version does not
replay on an earlier version.
@campoy
campoy marked this pull request as ready for review October 30, 2026 02:21
@campoy
campoy requested a review from a team as a code owner October 30, 2026 02:21

@DABH DABH left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at head f6573c5 (two commits over 3af00d9). Recipe: ten finder angles, 20 raw items deduplicated to 8 candidates, one adversarial verifier per candidate with end-to-end reproductions for seven of them (a legacy-config pin across a rolling upgrade, the real OpenAIModel with a counting fetch stub, the Workflow Task failure loop and its fix-and-redeploy recovery, the #279 overflow probe, the unbound-method probe, direct TemporalModel runs, and a single-builder refactor that passes lint and all 73 tests), then a gap sweep that found nothing new. Verdicts: 6 confirmed, 2 plausible. Two sub-claims were refuted along the way and are not posted: a live proactive-compression path for the default manager, and the failure converter rebuilding ContextWindowOverflowError today. In a clean clone at the head with Strands 1.19.0 locked: eslint clean, make lint exit 0 with pnpm 10.27.0 on PATH, make test 73 of 73, conventions check OK. Not verified: a live OpenAI endpoint, the macOS, Windows and Node 22 cells, and the body's replay matrix beyond the SDK's activity-type comparison.

Severity: P1 = wrong result or stuck run in realistic use; P2 = correctness or robustness in an edge case; P3 = docs, tests, cleanup.

Where the problems cluster: the design is the right one and mirrors the Python plugin, but the two things that decide whether the feature works at all are the rollout window and one stray key in the options. A run whose config read lands on a pre-upgrade Worker is pinned stateless forever, and the real stateful provider cannot complete a single call on the locked SDK because cancelSignal still crosses the Activity boundary. The rest is documentation of behaviours the code already has, one unbound method, and test diagnostics.

PR body: it ends 'Draft while this question is open', but the PR is not marked draft.

return activities[GET_MODEL_CONFIG_ACTIVITY]({ modelName: this.modelName }).then(({ stateful, ...config }) => {
this.config = config;
// A result recorded before the flag existed has none: the model ran as stateless then.
this.isStateful = stateful === true;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 (borderline P1): a run whose strandsGetModelConfig result was served by a pre-upgrade Worker records no stateful flag, is pinned to the stateless path for its whole lifetime, and then retries invokeModel forever with a stateful OpenAIModel even after every Worker is upgraded. Rolling upgrade on one task queue without Worker Versioning: a stateful Workflow starts during the window, its config Activity is served by a main Worker whose getModelConfig (3af00d9 model-activity.ts:113-116) returns only {modelId, contextWindowLimit}, so this line sets isStateful = false and line 90 never re-reads (one TemporalModel per agent; only Continue-As-New re-reads). Every call then takes the stateless branch (138-150) with options.modelState serialized as {"_state":{...}}, and on an upgraded Worker formatResponsesRequest (Strands 1.19.0 responses-adapter.js:50) throws TypeError: options?.modelState?.get is not a function before any request, a retryable failure under the default policy (unlimited attempts, 10 min start-to-close), so the run never completes a turn and only terminate or reset recovers it. Reproduced end to end with a legacy-config Worker then a PR-head Worker: pending invokeModel at attempt 4 with that lastFailure, nonRetryable=false, stub calls empty. The README rollout paragraph (line 102) covers only the two new Activity types and says upgrading the fleet suffices; the body's 'terminate it' covers only runs started before the upgrade; the mixed-fleet test (test-strands.ts:356-394) binds the head's getModelConfig, which already returns stateful: true (line 365), so it covers only the opposite direction, and no committed history pins the 'absent flag reads as stateless' contract. Sound mitigations: fail non-retryably in the registered stateless entry points (the plugin.ts:90-91 wrappers, not inside invokeModel, which the stateful Activities delegate to) when model.stateful is true; add the README sentence the Python README has at its line 121; commit a main-recorded history fixture replayed on the head.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f6fac51. invokeModel and invokeModelStreaming now fail a call for a stateful Worker model at once, with a non-retryable StatefulModelOnStatelessPath error that names the cause, instead of retrying it forever. The stateful Activities run the model through private helpers, so the check doesn't affect them.

The README's rollout paragraph (a69c4d8, a follow-up I had pending) now says a run needs its Workflow and config Activity from one version, and with f6fac51 it names this error and says to reset or terminate such runs. src/__tests__/histories/stateful-before-flag.json, recorded on main, pins that a config without the flag replays on the stateless path; it fails with [TMPRL1100] if a missing flag reads as stateful. A new test covers the pinned run: one attempt, non-retryable, and the model never runs (50 attempts without the check).

One cost, now in the PR body: an open run of a custom stateful model that never reads modelState, which kept working on the stateless path, fails at its next call after the upgrade.

if (this.isStateful) {
// The agent's model state is a live store; send a copy of its values, and copy the updated
// values back, as the model would have written them natively.
const { modelState, ...rest } = options ?? {};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: the new stateful branch strips only modelState from the stream options and still forwards cancelSignal (an AbortSignal, serialized to {}), so on the locked Strands 1.19.0 the only real stateful provider, OpenAIModel, fails before sending any request. Strands always sets cancelSignal: self._abortSignal (agent.js:182, :1656) and OpenAIModel forwards { signal: options?.cancelSignal } (models/openai/model.js:149, :198); the payload converter turns it into {}. Reproduced with the real OpenAIModel({stateful: true}) (openai 6.49.0, counting fetch stub, options round-tripped through the converter and the StateStore wrapping at model-activity.ts:176-177): PR-head options give APIConnectionError: Connection error. caused by TypeError: signal.addEventListener is not a function, zero fetch calls, retried under the unlimited policy; with const { modelState, cancelSignal: _unused, ...rest } = options ?? {} call one returns resp_1 and call two sends {previous_response_id: "resp_1", store: true}, so chaining works on the locked SDK. Dropping it is safe (openai client.js:534 guards if (signal); Bedrock duck-types it and {} never aborts; the Anthropic model provider never reads it; nothing in the plugin reads it) and replay-safe (sdk-core compares only Activity id and type). The stateless path (142, 149) is equally broken on main, which the body scopes out and README line 100 documents as a limitation, though its remedy (Strands below 1.15.0) contradicts the plugin's own lock and the top-level error is actually Connection error.. The new tests pass only because StatefulStub ignores cancelSignal. Better placement is Activity-side at model-activity.ts:144 and 159, or substitute Context.current().cancellationSignal so Temporal cancellation aborts the provider call; a StatefulStub assertion that cancelSignal is absent would pin the contract.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, this blocks a real stateful OpenAIModel on the locked SDK. The same bug breaks every OpenAIModel call, stateless too, so it gets its own issue and PR: the Activity replaces the serialized cancelSignal with Temporal's own cancellation signal, on both paths. I'll link it here once it's open. This PR keeps the README limitation until that lands.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed as #296, with the fix in #297: the model Activity passes Temporal's own cancellation signal as cancelSignal, on both paths. Once #297 lands, this PR's stateful helpers pick it up and the README limitation goes.

private initializeWithConfig(): Promise<void> {
// Strands' Agent refuses these managers for a stateful model at construction, before
// TemporalAgent can read the worker model's stateful flag, so refuse them here.
if (this.model.stateful && this.managerGiven) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 (plausible; the open question in your body): a stateful TemporalAgent keeps Strands' default SlidingWindowConversationManager, picked by super(...) at line 72 while stateful was still false, instead of NullConversationManager. agent.js:236-245 reads this.model.stateful at construction and initializeWithConfig only refuses user-given managers. Inert with the locked SDK: the Activity failure reaches Strands as ModelError{cause: ActivityFailure} (the converter overrides only errorToFailure; temporal-model.ts has no rethrow), so the overflow hook's instanceof is false, proactive compression is off and cannot be enabled, and _applyManagement runs after ModelPlugin cleared the messages (probe at the head: turn two gives ModelError, no trim). But with #279's rethrow applied to the stateful branch, call four chained to resp_2 with the tool result truncated from 1000 to 426 characters and the turn 'recovered', where native Strands (NullConversationManager, reduce returning false) fails on call three. Python swaps in NullConversationManager after the config read (_temporal_agent.py:173-187); here _conversationManager is private and already queued in _pluginRegistry. Options: say in the README's stateful paragraph that the default manager stays installed and is inert, and have #279 skip the rethrow for a stateful model or swap the manager; or a stateful?: boolean hint on TemporalAgentOptions validated against the config read; or neutralize the default instance after the read (reduce = () => false, _applyManagement = () => {}).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documented in the README's stateful paragraph (f6fac51): the default manager stays installed and, through the plugin, has nothing to trim. I dropped the open question and the Draft line from the body.

On the overflow interaction: #279 was folded into #200, which rethrows ContextWindowOverflowError so the agent's manager trims. Whichever of #200 and this PR lands second keeps that rethrow on the stateless calls only, so a stateful model's overflow ends the turn, as it does natively, instead of being trimmed by the default manager. With #200 it ends it with the Activity failure rather than Strands' own error.

// Strands' Agent refuses these managers for a stateful model at construction, before
// TemporalAgent can read the worker model's stateful flag, so refuse them here.
if (this.model.stateful && this.managerGiven) {
return Promise.reject(new Error(STATEFUL_MANAGER_REFUSED));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 (docs): STATEFUL_MANAGER_REFUSED is a plain Error, so when Workflow code does not catch it the run does not fail; the Workflow Task fails and is retried about every 10 s until the manager is removed and the Worker redeployed, and README line 98 ('throws at the agent's first invocation') could say so. internals.js:1051-1072 fails the run only for a TemporalFailure or a configured type; Strands' stream() (agent.js:663-668) awaits initialize() inside its try and propagates the rejection unwrapped. Reproduced: a Workflow passing conversationManager with a stateful stub and no try/catch stayed RUNNING with WorkflowTaskFailed at attempt 4; flipping the factory back does not help because the recorded config says stateful; redeploying with the manager dropped completed the stuck runs without termination. That is the SDK's contract and matches the pre-existing SNAPSHOT_DISABLED and RETRY_STRATEGY_DISABLED guards and the Python plugin's plain ValueError. Do not switch to a non-retryable ApplicationFailure: it would permanently fail every in-flight run at first invocation where today they finish once the manager is dropped. The PR's test covers only the caught path (workflows/strands.ts:91-98; test-strands.ts:349). Suggested clause: catch it, or the uncaught error fails the workflow task until the manager is removed and the worker redeployed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to the README (f6fac51): catch the error, or the Workflow Task fails and is retried until the manager is removed and the Worker redeployed. It stays a plain Error, as you suggest.

...this.activityOptions,
});

if (this.isStateful) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 (docs): isStateful is set only by loadConfig(), which only TemporalAgent.initialize calls, so the exported TemporalModel used with a plain Strands Agent never reads the config and routes a stateful Worker model through invokeModel, failing every call. new Agent({ model: new TemporalModel({ modelName: 'openai' }) }) with a stateful factory schedules no strandsGetModelConfig, the Activity input carries modelState: {"_state":{}}, and formatResponsesRequest throws modelState.get is not a function on every attempt. Reproduced in a scratch copy: the stateless stub works on the direct path; the stateful stub without loadConfig fails as described; with await model.loadConfig() before new Agent(...) the run chains ids and gets NullConversationManager. The direct path already skipped the config read at the base (since #158), the README never mentions TemporalModel, and awaiting loadConfig() inside stream() would add an undocumented TMPRL1100 break for direct-path histories and land after Strands' constructor-time checks, so a sentence in the class doc (lines 38-46), and optionally 'through TemporalAgent' at README line 98, is the right outcome.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to the class doc (f6fac51): with a plain Strands Agent, call await model.loadConfig() before building it. The README now says "through TemporalAgent". With the change above, such a call now fails at once with a message that names this case, instead of retrying.

// stateful support takes it.

/** Run a stateful model with the agent's model state; return its events and the updated state. */
async invokeStatefulModel(input: InvokeStatefulModelInput): Promise<InvokeStatefulModelResult> {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: invokeStatefulModel and invokeStatefulModelStreaming (182) are plain prototype methods while invokeModel and invokeModelStreaming are bound in the constructor (109-110), so registering them by reference, the pattern the PR's own test uses for the bound pair (test-strands.ts:375-376), throws on the first call because the Worker invokes Activities with no receiver. Probe through MockActivityEnvironment and the Worker's real Activity.execute: strandsInvokeStatefulModel: modelActivity.invokeStatefulModel gives TypeError: Cannot read properties of undefined (reading 'invokeModel') at lib/model-activity.js:110, retried as a plain retryable failure; getModelConfig is already unbound on main, which is why the test binds it explicitly (374). No supported path reaches this (ModelActivity is not exported; StrandsPlugin registers arrow wrappers at plugin.ts:90-95). If fixed, bind only; wrapping them in autoHeartbeat again doubles the heartbeat timers (12 versus 6 in the probe) because the inner call already goes through the wrapped invokeModel.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f6fac51: all four model Activities are bound and heartbeat-wrapped once in the constructor. The stateful ones no longer go through invokeModel, which now refuses stateful models, so nothing is wrapped twice.

// values back, as the model would have written them natively.
const { modelState, ...rest } = options ?? {};
const input = { modelName: this.modelName, messages, options: rest, modelState: modelState?.getAll() ?? {} };
const result = this.streamingTopic

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 (cleanup): stream() now dispatches through four near-identical Activity calls that write the {modelName, messages, options} input three times with two different options treatments and two yield idioms. Lines 121-129 build input once for the stateful pair (options: rest), while 139-150 repeat two full literals for the stateless pair (options whole, so the Activity receives modelState: {"_state":{...}} and cancelSignal: {}), followed by yield* result.events; return; versus for (const event of events) yield event. The next input-shape change (dropping cancelSignal, above) must be edited at 121, 142 and 149. A form that destructures once, builds input and the streaming variant once, and selects among the four Activities with one-line calls was applied to a copy: tsc, eslint and 73 of 73 tests pass. A literal single union call is not type-safe without over-supplying fields to the stateless Activities, so four one-line calls over one builder is the honest form; options: this.isStateful ? rest : options keeps the stateless input byte-identical if that is wanted.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f6fac51: stream() destructures once and builds one input, with options unchanged on the stateless path.

},
});
const pending = await legacyWorker.runUntil(async () => {
for (;;) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 (plausible, test diagnostics): the rollout test's for (;;) describe-poll exits only when a pending Activity reaches attempt > 2, so any other outcome (a different pending Activity type, an empty pendingActivities because the run ended, no NotFoundError produced) spins until AVA's 60 s timeout with no assertion diagnostic. With STATEFUL_ACTIVITY_OPTIONS retrying every 100 ms (backoff 1, 50 attempts) the observable window runs from about 200 ms to about 5 s, which the 100 ms poll cannot realistically miss; but if the legacy Worker ever registered the stateful Activity or the run ended early, legacyWorker.runUntil would never settle. Breaking when (await handle.describe()).status.name !== 'RUNNING', or capping the iterations and failing with the last describe, makes such failures immediate without weakening the test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f6fac51: the loop now throws with the run's status and pending Activities when the run leaves RUNNING, or after 300 polls (30 s).

The stateful path is chosen from the flag the config activity returns, so a
run needs its workflow and config activity from the same version. Say so in
the README, with the rollout that keeps them together, and what changes for
a custom model that reports stateful but ignores modelState. Correct the
comment on the test whose activity worker lacks the stateful activities.
A run whose config read had no stateful flag, from a worker without
stateful support, stays on the stateless path, and a model that reads
modelState then failed every attempt with a retryable error, forever.
invokeModel and invokeModelStreaming now fail such a call at once with a
non-retryable StatefulModelOnStatelessPath error that names the cause. The
stateful activities run the model through private helpers instead, and all
four model activities are bound and heartbeat once.

Also from review:
- stream() builds the activity input once.
- README: stateful models work through TemporalAgent; catch the
  manager refusal or the workflow task retries; the default manager stays
  installed and has nothing to trim; the rollout paragraph names the new
  error.
- TemporalModel's doc says to call loadConfig() with a plain Agent.
- A history recorded on main, replayed here, pins that a config without
  the flag reads as stateless.
- The rollout test's poll loop stops with a message when the run ends.
@campoy

campoy commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review, @DABH. a69c4d8 and f6fac51 address it; replies are in the threads, and the PR body is updated.

  • Rollout window: a run pinned to the stateless path now fails its stateful model's call at once with a non-retryable StatefulModelOnStatelessPath error instead of retrying forever, and the README says a run needs its Workflow and config Activity from one version. A history recorded on main pins that a config without the flag reads as stateless.
  • cancelSignal: it breaks every OpenAIModel call, stateless too, so it moves to its own issue and PR, fixed in the Activity for both paths. I'll link it in the thread.
  • Open question: answered by documenting the default manager. The body no longer has the question or the Draft line; the overflow interaction with fix(strands): keep the caller's failure converter and turn Strands errors into Temporal failures (breaking) #200 is noted in the thread.

@DABH DABH left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second round, at head f6fac51 (two commits over f6573c5: a69c4d8 rewrites the README rollout paragraph; f6fac51 adds the non-retryable stateless-path guard, binds the stateful Activities, builds one input in stream(), and adds the history fixture and test fixes). In a worktree with Strands 1.19.0 locked and pnpm 10.27.0: make sync, make lint clean, make test 75 of 75, conventions check OK. CI is green on all four cells. I verified the findings with three temporary probe tests against the dev server, since removed: a config recorded with stateful: true followed by a stateless factory, a config recorded with stateful: false followed by a stateful factory, and the pinned run under a workflow that does not catch. Checked and not raised: Strands' other construction-time stateful check (delegation tools) runs in AgentDelegation.initAgent during initialize(), after the config read, so it still fires. #202 builds its duplicate check from the plugin's activity map, so it picks up the two new names. #297 edits lines that now live in runModel and runModelStreaming, so it needs a rebase, after which it covers both paths as you say. Not verified: a live OpenAI endpoint, and the body's mixed-version matrix beyond what the committed history pins.

Severity: P1 = wrong result or stuck run in realistic use; P2 = correctness or robustness in an edge case; P3 = docs, tests, cleanup.

Previous rounds. Addressed:

  • A pinned-stateless run now fails at once with a non-retryable StatefulModelOnStatelessPath: one attempt, no model call, pinned by the new test. The README states the one-version-per-run rule and that such runs need a reset or terminate, and stateful-before-flag.json pins that a config without the flag replays on the stateless path.
  • The stateful Activities are bound and heartbeat once, through runModel instead of the guarded invokeModel.
  • stream() builds one input and keeps the stateless input unchanged.
  • The class doc tells plain-Agent users to call loadConfig(), and the README says 'through TemporalAgent'.
  • The README covers the inert default manager and the uncaught manager refusal.
  • The rollout test's poll loop throws with the run status, and the Draft line is gone.

Accepted decline: cancelSignal moves to #296 and #297, because it breaks stateless OpenAIModel calls on main too, and the README limitation stays until then.

Where the problems cluster: the new guard fails fast on one direction of a model and config mismatch. The other direction (a recorded stateful config, with a model that is now stateless) still completes silently, sending only each turn's new message. Neither the guard's message nor the README names the realistic trigger for either direction: switching a model between stateful and stateless while runs are open.

Approving, because no P1 or blocking item remains and CI is green. The non-blocking P2 is a short mirror check that I would still add before merge.

/** Run a stateful model with the agent's model state; return its events and the updated state. */
async invokeStatefulModel(input: InvokeStatefulModelInput): Promise<InvokeStatefulModelResult> {
const modelState = new StateStore(input.modelState);
const model = this.getModel(input.modelName);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 (non-blocking): the stateful Activities run whatever model the factory now returns without checking that it is still stateful, so a run whose config recorded stateful: true silently loses its context once the factory turns stateless, for example OpenAIModel redeployed with stateful: false while chats are open.

Because the recorded config still says stateful, TemporalModel.stateful stays true and Strands' ModelPlugin clears agent.messages after each turn. The next call therefore gives the now-stateless model only the new message. That model ignores modelState, so OpenAIModel sends neither previous_response_id nor the history (responses-adapter.js:37-54 with stateful false), and nothing fails.

Probe at this head: statefulAgent('models-queue'), with the models queue serving strandsGetModelConfig: async () => ({ stateful: true }) and strandsInvokeStatefulModel from a ModelActivity whose factory returns a stateless recording stub. The run completes with [0, 0], and the model saw 1 message on both calls. Natively, a stateless agent sends 1 and then 3. The opposite mismatch now fails at once with a named error, but this one runs to completion with the wrong context. The stateful tests use the same StatefulStub for the config and the call, so no test covers the mismatch.

Fix:

  • Add the mirror check here and at line 216: for example, a getStatefulModel that throws a non-retryable ApplicationFailure when !model.stateful, with its own type or message.
  • Add a test shaped like the pinned-run one.
  • Add a README clause saying that switching a model between stateful and stateless while runs are open needs the same reset or terminate.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 6c7d8d6. invokeStatefulModel and invokeStatefulModelStreaming now get the model through getStatefulModel, which fails the call at once with a non-retryable StatelessModelOnStatefulPath when the model isn't stateful. The pinned-run test is now a two-case table, and its new case is your shape: a { stateful: true } config, a model that became stateless, one attempt, the typed failure, and no model call. That case fails on f6fac51, where the stateless model runs on the stateful path. The README's rollout paragraph now says that changing a model between stateful and stateless applies only to new runs, and those runs need a reset or terminate.

nonRetryable: true,
message:
`The worker model ${JSON.stringify(name ?? this.defaultName)} is stateful, but the workflow ` +
`called it without its model state. Its config was read from a worker without stateful ` +

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: the guard's message names two causes, a config read from a worker without stateful support and TemporalModel used without TemporalAgent, but a run whose config was read on this version also hits it when the factory becomes stateful afterwards, for example when stateful: true is added to a deployed OpenAIModel while chats are open.

Probe: the config activity returns { stateful: false }, and then invokeModel is served by a ModelActivity with a stateful model. Result: one attempt, StatefulModelOnStatelessPath, and this message. An operator who has just enabled stateful mode, with every worker on this version, is sent looking for an old worker that does not exist. A stateful model that uses modelState never worked through the plugin before, so enabling it on a deployed model is the likely first way users meet this error.

Fix: add the third cause here and in README line 102, for example 'or the worker model became stateful after the run read its config'. Also say that enabling stateful mode applies only to new runs, as in the rollout.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 6c7d8d6. The message and the doc comment now also name a worker model that became stateful after the run read its config, and the README says that enabling or disabling stateful mode applies only to new runs. The test checks that each message names its cause.

Comment thread typescript/strands-agents/README.md Outdated

`OpenAIModel` calls through `TemporalAgent`, stateful or not, currently work only with `@strands-agents/sdk` releases before 1.15.0. From 1.15.0, Strands passes an `AbortSignal` in the model's stream options, the signal reaches the model activity as a plain object, and the OpenAI client fails with `signal.addEventListener is not a function` before it sends a request. Temporal retries that activity until its retry policy or timeout runs out.

Stateful model calls run as their own activities, `strandsInvokeStatefulModel` and `strandsInvokeStatefulModelStreaming`, chosen from the stateful flag that `strandsGetModelConfig` returns, so a run that uses a stateful model needs the workflow and the config activity from the same version. An earlier workflow that reads this version's config schedules `invokeModel`, and this version then fails to replay the run with a nondeterminism error. This version's workflow that reads an earlier config, which has no flag, keeps the stateless path for the rest of the run, and an upgraded worker fails each of its model calls with a non-retryable `StatefulModelOnStatelessPath` error. Neither run recovers: reset or terminate it. Roll out with Worker Versioning, keeping each run's workflow and activity workers on one version, including any task queue set in `activityOptions`. Without it, stop the runs that use a stateful model, replace every worker on those task queues, and then start new runs.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: 'an upgraded worker fails each of its model calls with a non-retryable StatefulModelOnStatelessPath error' reads as if the run fails, but unless the workflow catches the agent's error, the run stays Running with a failing workflow task, and the new type is two causes down.

Strands' Model.streamAggregated wraps the Activity failure in a plain ModelError (models/model.js:350-355), and the workflow SDK treats that as a task failure. Probe at this head: the pinned-run setup with a workflow that does not catch. After 8 s the status is RUNNING, with two WorkflowTaskFailed events whose message is Activity task failed, whose cause is Activity task failed, and only below that StatefulModelOnStatelessPath with the guard's message. A workflow that does catch sees err.message === 'Activity task failed'; the new test's workflow returns exactly that string. The PR body's 'ends a run the same way' holds only once #200 unwraps Temporal failures.

Fix: add a clause like the one in the manager paragraph. It should say that the run stays open with a failing workflow task until you reset or terminate it, and that the error type is on the ModelError's cause.cause. Operators can then find these runs and catch the error by type.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 6c7d8d6. The rollout paragraph now says that, unless the workflow catches the agent's error, the run stays open: the error reaches the workflow as a ModelError with the type on its cause.cause, and the workflow task fails on each attempt. I also corrected the body: the errors end a run the same way only with #200; without it the run stays open, as the README says.

…ateful

A run whose config read recorded a stateful model keeps the stateful
path, and its agent clears its messages after each turn. If the worker
model became stateless since, for example an OpenAIModel redeployed
with stateful: false, the stateful activities ran it with only each
turn's new message and nothing failed.

- The stateful activities now fail such a call at once with a
  non-retryable StatelessModelOnStatefulPath error.
- The StatefulModelOnStatelessPath message also names a model that
  became stateful after the run read its config.
- README: changing a model between stateful and stateless applies only
  to new runs, and an uncaught error leaves the run open with a failing
  workflow task, with the type on the ModelError's cause.cause.
- The pinned-run test covers both directions.
Conflicts:
- temporal-model.ts: #200's context-overflow rethrow stays around the
  stateless model calls only. A stateful model's overflow is not turned
  into Strands' error, so the default manager the agent keeps does not
  trim and retry a server-side conversation.
- README.md: #200 moved the model config note to "Upgrading from 1.24
  and earlier". With #200, a stateful/stateless mismatch error reaches
  invoke() as an ActivityFailure, so an uncaught one fails the run.
- test-strands.ts and workflows/strands.ts: both sides' imports and
  tests kept.
@campoy
campoy merged commit 34fb981 into main Oct 31, 2026
27 checks passed
@campoy
campoy deleted the fix/strands-typescript-stateful-model-state-lost branch October 31, 2026 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] strands-agents (TypeScript): a stateful OpenAIModel fails every model call through the plugin

2 participants