Skip to content

fix(strands): run stateful models with their server-side state (breaking) #285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
campoy merged 6 commits into from
Oct 31, 2026
Merged

fix(strands): run stateful models with their server-side state (breaking) #285

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
930fc63
fix(strands): run stateful models with their server-side state (break…
campoy Oct 29, 2026
f6573c5
Address review findings
campoy Oct 30, 2026
a69c4d8
docs(strands): require one version per run for stateful models
campoy Oct 30, 2026
f6fac51
fix(strands): fail a stateful model's call that lost its model state
campoy Oct 30, 2026
6c7d8d6
fix(strands): fail a stateful call whose worker model is no longer st…
campoy Oct 30, 2026
dd556c4
Merge main (#200) into the stateful model fix
campoy Oct 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions typescript/strands-agents/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,12 @@ Each `TemporalAgent` carries its own activity options (timeouts, retry policy, t

Before its first model call, each `TemporalAgent` reads the model's `modelId` and `contextWindowLimit` from its `getConfig()` through the `strandsGetModelConfig` activity, with the agent's activity options. Strands' traces and [proactive compression](https://strandsagents.com/docs/user-guide/sdk/agents/conversation-management/#proactive-context-compression) then use the real model's values, as they do outside Temporal. If you scaled `compressionThreshold` by the model's window divided by 200,000 to make up for the missing limit, remove that scaling.

A worker model that keeps the conversation server-side, such as `OpenAIModel` with `stateful: true`, gets the agent's `modelState`, such as the last response id, on each model call through `TemporalAgent`, and the agent clears its messages after each turn, as natively. As in Strands, such an agent cannot use a `conversationManager` or `contextManager`; because `TemporalAgent` learns that the model is stateful when it reads the config, it throws at the agent's first invocation instead of at construction. Catch that error, or the workflow task fails and is retried until you remove the manager and redeploy the worker. With no manager given, such an agent keeps Strands' default sliding window manager, which Strands picks before the agent reads the config, where natively it would use `NullConversationManager`. Through the plugin that manager has nothing to trim: the agent clears its messages after each turn, before the manager runs, and a context window overflow reaches the agent as a model error. A custom model that reports `stateful` but does not use `modelState` ran statelessly through earlier versions, with the agent keeping its messages; it now gets only each turn's new message, and an agent that gives it a manager now throws. Report `stateful` as false for such a model to keep the earlier behavior.

`OpenAIModel` calls through `TemporalAgent`, stateful or not, currently work only with `@strands-agents/sdk` releases before 1.15.0. From 1.15.0, Strands passes an `AbortSignal` in the model's stream options, the signal reaches the model activity as a plain object, and the OpenAI client fails with `signal.addEventListener is not a function` before it sends a request. Temporal retries that activity until its retry policy or timeout runs out.

Stateful model calls run as their own activities, `strandsInvokeStatefulModel` and `strandsInvokeStatefulModelStreaming`, chosen from the stateful flag that `strandsGetModelConfig` returns, so a run that uses a stateful model needs the workflow and the config activity from the same version. An earlier workflow that reads this version's config schedules `invokeModel`, and this version then fails to replay the run with a nondeterminism error. This version's workflow that reads an earlier config, which has no flag, keeps the stateless path for the rest of the run, and an upgraded worker fails each of its model calls with a non-retryable `StatefulModelOnStatelessPath` error. Changing a worker model between stateful and stateless, such as adding `stateful: true` to a deployed `OpenAIModel`, also applies only to new runs: an open run keeps the mode its config read recorded, and its model calls fail with `StatefulModelOnStatelessPath`, or with `StatelessModelOnStatefulPath` when the model became stateless. None of these runs recovers. The agent's `invoke()` throws an `ActivityFailure` whose cause has that type, so unless the workflow catches it, the run fails; reset a run you want to continue. Roll out with Worker Versioning, keeping each run's workflow and activity workers on one version, including any task queue set in `activityOptions`. Without it, stop the runs that use a stateful model, replace every worker on those task queues, and then start new runs.

## Retries

`TemporalAgent` disables Strands' built-in `ModelRetryStrategy` so retries are handled exclusively by Temporal. Configure retries via `activityOptions.retry` on `TemporalAgent`, and on the activity options accepted by `workflow.activityAsTool`, `workflow.activityAsHook`, and `TemporalMCPClient`:
Expand Down Expand Up @@ -242,24 +248,26 @@ The plugin keeps the failure converter your client and worker set with `dataConv

## Continue-as-new

A chat-style workflow accumulates history with every turn and will eventually hit Temporal's per-workflow history limit. `workflowInfo().continueAsNewSuggested` flips true once the server decides history has grown large enough; check it after each turn and hand off to a fresh run, carrying `agent.messages` as input:
A chat-style workflow accumulates history with every turn and will eventually hit Temporal's per-workflow history limit. `workflowInfo().continueAsNewSuggested` flips true once the server decides history has grown large enough; check it after each turn and hand off to a fresh run, carrying `agent.messages` and `agent.modelState` as input. A stateful model clears the messages after each turn and keeps its context in the model state, such as the last response id; a run that carries only the messages starts the model's conversation over:

```ts
import { defineSignal, setHandler, condition, continueAsNew, workflowInfo } from '@temporalio/workflow';
import { TemporalAgent } from '@temporalio/strands-agents';
import type { Message } from '@strands-agents/sdk';
import type { JSONValue, Message } from '@strands-agents/sdk';

export const userSays = defineSignal<[string]>('userSays');
export const endChat = defineSignal('endChat');

export interface ChatInput {
messages?: Message[];
modelState?: Record<string, JSONValue>;
}

export async function chatWorkflow(input: ChatInput = {}): Promise<void> {
const agent = new TemporalAgent({
activityOptions: { startToCloseTimeout: '60 seconds' },
messages: input.messages,
modelState: input.modelState,
});

const pending: string[] = [];
Expand All @@ -272,7 +280,7 @@ export async function chatWorkflow(input: ChatInput = {}): Promise<void> {
if (done) return;
await agent.invoke(pending.shift()!);
if (workflowInfo().continueAsNewSuggested) {
await continueAsNew<typeof chatWorkflow>({ messages: agent.messages });
await continueAsNew<typeof chatWorkflow>({ messages: agent.messages, modelState: agent.modelState.getAll() });
}
}
}
Expand Down
250 changes: 250 additions & 0 deletions typescript/strands-agents/src/__tests__/histories/stateful-before-flag.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,250 @@
{
"events": [
{
"eventId": "1",
"eventTime": "2026-10-30T07:43:24.163131175Z",
"eventType": "EVENT_TYPE_WORKFLOW_EXECUTION_STARTED",
"taskId": "1048587",
"workflowExecutionStartedEventAttributes": {
"workflowType": {
"name": "statefulAgent"
},
"taskQueue": {
"name": "record-a-stateful-run-on-main",
"kind": "TASK_QUEUE_KIND_NORMAL"
},
"input": {
"payloads": [
{
"metadata": {
"encoding": "anNvbi9wbGFpbg=="
},
"data": "InBsYWluIg=="
}
]
},
"workflowTaskTimeout": "10s",
"originalExecutionRunId": "01a18bc3-7203-71fa-8179-be473da5e3cb",
"identity": "119@a529f6a35480",
"firstExecutionRunId": "01a18bc3-7203-71fa-8179-be473da5e3cb",
"attempt": 1,
"firstWorkflowTaskBackoff": "0s",
"header": {},
"workflowId": "ead9880d-031a-4bb5-a558-fe6da5afedeb"
}
},
{
"eventId": "2",
"eventTime": "2026-10-30T07:43:24.165304134Z",
"eventType": "EVENT_TYPE_WORKFLOW_TASK_SCHEDULED",
"taskId": "1048588",
"workflowTaskScheduledEventAttributes": {
"taskQueue": {
"name": "record-a-stateful-run-on-main",
"kind": "TASK_QUEUE_KIND_NORMAL"
},
"startToCloseTimeout": "10s",
"attempt": 1
}
},
{
"eventId": "3",
"eventTime": "2026-10-30T07:43:24.244065967Z",
"eventType": "EVENT_TYPE_WORKFLOW_TASK_STARTED",
"taskId": "1048593",
"workflowTaskStartedEventAttributes": {
"scheduledEventId": "2",
"identity": "119@a529f6a35480",
"requestId": "aa9e4dbb-6a2b-47d5-b117-2a477445069b",
"historySizeBytes": "329",
"workerVersion": {
"buildId": "@temporalio/worker@1.24.0+935bcefedc3349fb2c2daaa8be302fc89c585e57f7f410c833f142f5b7ed6955"
}
}
},
{
"eventId": "4",
"eventTime": "2026-10-30T07:43:25.214269551Z",
"eventType": "EVENT_TYPE_WORKFLOW_TASK_COMPLETED",
"taskId": "1048598",
"workflowTaskCompletedEventAttributes": {
"scheduledEventId": "2",
"startedEventId": "3",
"identity": "119@a529f6a35480",
"workerVersion": {
"buildId": "@temporalio/worker@1.24.0+935bcefedc3349fb2c2daaa8be302fc89c585e57f7f410c833f142f5b7ed6955"
},
"sdkMetadata": {
"coreUsedFlags": [
2,
3,
1
],
"sdkName": "temporal-typescript",
"sdkVersion": "1.24.0"
},
"meteringMetadata": {}
}
},
{
"eventId": "5",
"eventTime": "2026-10-30T07:43:25.215057134Z",
"eventType": "EVENT_TYPE_ACTIVITY_TASK_SCHEDULED",
"taskId": "1048599",
"activityTaskScheduledEventAttributes": {
"activityId": "1",
"activityType": {
"name": "strandsGetModelConfig"
},
"taskQueue": {
"name": "record-a-stateful-run-on-main",
"kind": "TASK_QUEUE_KIND_NORMAL"
},
"header": {},
"input": {
"payloads": [
{
"metadata": {
"encoding": "anNvbi9wbGFpbg=="
},
"data": "eyJtb2RlbE5hbWUiOiJzdGF0ZWZ1bCJ9"
}
]
},
"scheduleToCloseTimeout": "0s",
"scheduleToStartTimeout": "0s",
"startToCloseTimeout": "15s",
"heartbeatTimeout": "0s",
"workflowTaskCompletedEventId": "4",
"retryPolicy": {
"initialInterval": "0.100s",
"backoffCoefficient": 1,
"maximumInterval": "10s",
"maximumAttempts": 50
},
"useWorkflowBuildId": true
}
},
{
"eventId": "6",
"eventTime": "2026-10-30T07:43:25.215569968Z",
"eventType": "EVENT_TYPE_ACTIVITY_TASK_STARTED",
"taskId": "1048603",
"activityTaskStartedEventAttributes": {
"scheduledEventId": "5",
"identity": "119@a529f6a35480",
"requestId": "66ea76a3-d045-44e5-86a8-ca7b0eb02891",
"attempt": 1,
"workerVersion": {
"buildId": "@temporalio/worker@1.24.0+935bcefedc3349fb2c2daaa8be302fc89c585e57f7f410c833f142f5b7ed6955"
}
}
},
{
"eventId": "7",
"eventTime": "2026-10-30T07:43:25.246124134Z",
"eventType": "EVENT_TYPE_ACTIVITY_TASK_COMPLETED",
"taskId": "1048604",
"activityTaskCompletedEventAttributes": {
"result": {
"payloads": [
{
"metadata": {
"encoding": "anNvbi9wbGFpbg=="
},
"data": "e30="
}
]
},
"scheduledEventId": "5",
"startedEventId": "6",
"identity": "119@a529f6a35480"
}
},
{
"eventId": "8",
"eventTime": "2026-10-30T07:43:25.246205759Z",
"eventType": "EVENT_TYPE_WORKFLOW_TASK_SCHEDULED",
"taskId": "1048605",
"workflowTaskScheduledEventAttributes": {
"taskQueue": {
"name": "119@a529f6a35480-d1063acad8154749a5ad79f09925f0d4",
"kind": "TASK_QUEUE_KIND_STICKY",
"normalName": "record-a-stateful-run-on-main"
},
"startToCloseTimeout": "10s",
"attempt": 1
}
},
{
"eventId": "9",
"eventTime": "2026-10-30T07:43:25.249141176Z",
"eventType": "EVENT_TYPE_WORKFLOW_TASK_STARTED",
"taskId": "1048609",
"workflowTaskStartedEventAttributes": {
"scheduledEventId": "8",
"identity": "119@a529f6a35480",
"requestId": "855b5a9d-8e00-4ace-8dd1-acd30f9b6a3d",
"historySizeBytes": "1255",
"workerVersion": {
"buildId": "@temporalio/worker@1.24.0+935bcefedc3349fb2c2daaa8be302fc89c585e57f7f410c833f142f5b7ed6955"
}
}
},
{
"eventId": "10",
"eventTime": "2026-10-30T07:43:25.285266968Z",
"eventType": "EVENT_TYPE_WORKFLOW_TASK_COMPLETED",
"taskId": "1048614",
"workflowTaskCompletedEventAttributes": {
"scheduledEventId": "8",
"startedEventId": "9",
"identity": "119@a529f6a35480",
"workerVersion": {
"buildId": "@temporalio/worker@1.24.0+935bcefedc3349fb2c2daaa8be302fc89c585e57f7f410c833f142f5b7ed6955"
},
"sdkMetadata": {},
"meteringMetadata": {}
}
},
{
"eventId": "11",
"eventTime": "2026-10-30T07:43:25.285404093Z",
"eventType": "EVENT_TYPE_ACTIVITY_TASK_SCHEDULED",
"taskId": "1048615",
"activityTaskScheduledEventAttributes": {
"activityId": "2",
"activityType": {
"name": "invokeModel"
},
"taskQueue": {
"name": "record-a-stateful-run-on-main",
"kind": "TASK_QUEUE_KIND_NORMAL"
},
"header": {},
"input": {
"payloads": [
{
"metadata": {
"encoding": "anNvbi9wbGFpbg=="
},
"data": "eyJtb2RlbE5hbWUiOiJzdGF0ZWZ1bCIsIm1lc3NhZ2VzIjpbeyJyb2xlIjoidXNlciIsImNvbnRlbnQiOlt7InRleHQiOiJ0dXJuIG9uZSJ9XSwidHJhY2tpbmdJZCI6IjY5NWY3Y2MxLTk1NzctNDJhNC1iNTNhLWIwNDY1ZWU0ZjJiZiJ9XSwib3B0aW9ucyI6eyJjYW5jZWxTaWduYWwiOnt9LCJ0b29sU3BlY3MiOltdLCJtb2RlbFN0YXRlIjp7Il9zdGF0ZSI6e319fX0="
}
]
},
"scheduleToCloseTimeout": "0s",
"scheduleToStartTimeout": "0s",
"startToCloseTimeout": "15s",
"heartbeatTimeout": "0s",
"workflowTaskCompletedEventId": "10",
"retryPolicy": {
"initialInterval": "0.100s",
"backoffCoefficient": 1,
"maximumInterval": "10s",
"maximumAttempts": 50
},
"useWorkflowBuildId": true
}
}
]
}
Loading
Loading