Skip to content

perf(opencode): load session history once per turn, append the id-tail per step - #26

Closed
PierrotAWB wants to merge 1 commit into
turn-phase-timingsfrom
turn-history-cache
Closed

PierrotAWB wants to merge 1 commit into
turn-phase-timingsfrom
turn-history-cache

Conversation

@PierrotAWB

@PierrotAWB PierrotAWB commented Aug 24, 2026 •

Copy link
Copy Markdown
Collaborator

Issue for this PR

Issues are disabled on this fork; motivation below. Stacked on #25 (phase telemetry); re-target after it merges.

Type of change

  • Refactor / code improvement

What does this PR do?

runLoop reloaded and re-hydrated the entire session history from SQLite on every step (filterCompactedEffect inside the while loop) — O(history × steps) I/O per turn, the largest per-step prep cost on long sessions. The runner lock makes a turn the session's single writer, so the loop can reuse what it already loaded.

Now the first iteration loads history as before; each later iteration fetches only rows with id greater than the last-seen id (new MessageV2.after, one indexed query — ids are monotonic per session) and appends. Two correctness guards, both traced against real behaviors:

  • A queued follow-up prompt writes its user message mid-turn (createUserMessage runs before the runner parks); today's per-step reload picks it up, and the id-tail query preserves exactly that.
  • If the fresh tail contains any compaction part or summary assistant, the retained-window computation (filterCompacted) is no longer append-stable, so the loop recomputes from scratch.

The cache is captured after SessionReminders.apply so reminder transforms persist into it, and it lives only for one runLoop invocation — cross-turn behavior is untouched.

How did you verify your code works?

The existing multi-step, subtask, question, and compaction suites all depend on correct cross-step history threading and are the golden tests here: prompt.test.ts + compaction.test.ts + message-v2.test.ts 148/148, full packages/opencode suite 3129 pass with the only failures being the pre-existing Truncate date bug (fixed in #24, not in this branch's ancestry) and a pty test that passes 9/9 in isolation (load-contention flake). bun run typecheck clean, bun run lint 0 errors. Fleet-wide before/after comes free from #25's history_ms field.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code


Summary by cubic

Loads session history once per turn and appends newly created messages by id on subsequent steps, replacing per-step full reloads. This removes O(history × steps) I/O in long turns while preserving correctness and cross-turn behavior.

  • Adds MessageV2.after(sessionID, id) to fetch rows with id > last seen (ids are monotonic per session).
  • Caches msgs across steps in the turn runLoop; if fresh rows include any compaction part or an assistant summary, recomputes the retained window from scratch.
  • Captures the cache after SessionReminders.apply, so reminder transforms persist; cache lives for a single runLoop.
  • Preserves mid-turn queued user messages: the id-tail fetch picks them up.
  • No schema or API changes; no migration required. Monitor history_ms to validate the perf win.

Written for commit 2137c4c. Summary will update on new commits.

Review in cubic

…l per step

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Hey! Your PR title perf(opencode): load session history once per turn, append the id-tail per step doesn't follow conventional commit format.

Please update it to start with one of:

  • feat: or feat(scope): new feature
  • fix: or fix(scope): bug fix
  • docs: or docs(scope): documentation changes
  • chore: or chore(scope): maintenance tasks
  • refactor: or refactor(scope): code refactoring
  • test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

@github-actions

Copy link
Copy Markdown

The following comment was made by an LLM, it may be inaccurate:

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/opencode/src/session/prompt.ts">

<violation number="1" location="packages/opencode/src/session/prompt.ts:1218">
P2: On multi-step plan/build turns, `cachedMsgs` retains each iteration's in-memory reminder and plugin mutations, so synthetic prompts or transformed parts accumulate across model requests. Keep the cached history immutable, or run per-step transforms on a clone and merge only persisted changes.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Effect.provideService(FSUtil.Service, fsys),
Effect.provideService(Session.Service, sessions),
)
cachedMsgs = msgs

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: On multi-step plan/build turns, cachedMsgs retains each iteration's in-memory reminder and plugin mutations, so synthetic prompts or transformed parts accumulate across model requests. Keep the cached history immutable, or run per-step transforms on a clone and merge only persisted changes.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/session/prompt.ts, line 1218:

<comment>On multi-step plan/build turns, `cachedMsgs` retains each iteration's in-memory reminder and plugin mutations, so synthetic prompts or transformed parts accumulate across model requests. Keep the cached history immutable, or run per-step transforms on a clone and merge only persisted changes.</comment>

<file context>
@@ -1197,6 +1215,7 @@ const layer = Layer.effect(
             Effect.provideService(FSUtil.Service, fsys),
             Effect.provideService(Session.Service, sessions),
           )
+          cachedMsgs = msgs
 
           const msg: SessionV1.Assistant = {
</file context>

@PierrotAWB

Copy link
Copy Markdown
Collaborator Author

Consolidated into #29 (one commit per concern there; review history for the prep_ms retry semantics is on this PR's threads).

@PierrotAWB PierrotAWB closed this Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant