Skip to content

perf(opencode): overlap the pre-turn snapshot with the provider request - #27

Closed
PierrotAWB wants to merge 1 commit into
turn-history-cachefrom
turn-snapshot-overlap
Closed

PierrotAWB wants to merge 1 commit into
turn-history-cachefrom
turn-snapshot-overlap

Conversation

@PierrotAWB

@PierrotAWB PierrotAWB commented Aug 24, 2026 •

Copy link
Copy Markdown
Collaborator

Issue for this PR

Issues are disabled on this fork; motivation below. Stacked on #26; re-target after it merges.

Type of change

  • Refactor / code improvement

What does this PR do?

SessionProcessor.create awaited snapshot.track() — a chain of git subprocesses (init/add/write-tree) over the worktree — before the provider stream could start, putting the whole git cost inside time-to-first-token on every step. The original eager await existed because the AI SDK can execute tools before emitting step events, so the snapshot must exist before any tool mutates files — a data-ordering constraint, not a "before the provider request" constraint.

Now the track forks (Effect.forkIn(scope)) and the stream starts immediately; an idempotent awaitSnapshot on the processor handle is the barrier, taken at exactly the consumption points: the shared tool execute wrapper (before any tool runs — the case the original comment warned about) and step-start. The provider's TTFT absorbs the git work. snapshot.track is internally mutex-locked, so the forked fiber serializes safely against the step-finish re-track; orDie keeps a git failure failing the turn as before, and forkIn(scope) ties the fiber to the turn's scope so interruption still cancels it. snapshot_ms (#25) now reports the true concurrent git cost rather than serial prep.

How did you verify your code works?

snapshot-tool-race.test.ts (the purpose-built test for this exact barrier) passes, plus test/snapshot 55/55, and prompt.test.ts + compaction.test.ts + retry.test.ts 145/145. bun run typecheck clean, bun run lint 0 errors. Fleet-wide effect visible post-release as prep_ms dropping while snapshot_ms holds steady.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code


Summary by cubic

Overlaps the pre-turn snapshot with the provider request to reduce time-to-first-token while preserving tool ordering. Previously we awaited the snapshot before starting the provider stream; now we track the snapshot concurrently and gate tool execution and step-start behind an idempotent awaitSnapshot.

  • Adds awaitSnapshot to SessionProcessor.Handle; tracks the snapshot in a forked fiber bound to the turn scope, preserving failure and cancellation semantics.
  • Gates tool execution via SessionTools.resolve and step-start on awaitSnapshot, ensuring tools cannot mutate files before the snapshot lands.
  • Maintains snapshot serialization via the internal mutex; snapshot_ms measures the concurrent git cost, and prep_ms should drop accordingly.

Review notes

  • Focus on concurrency boundaries: forkIn(scope), Fiber.join, and the cached result on the handle.
  • If tests construct fake processors, provide awaitSnapshot (e.g., Effect.succeed(undefined)).

Written for commit 2f010a4. Summary will update on new commits.

Review in cubic

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Hey! Your PR title perf(opencode): overlap the pre-turn snapshot with the provider request doesn't follow conventional commit format.

Please update it to start with one of:

  • feat: or feat(scope): new feature
  • fix: or fix(scope): bug fix
  • docs: or docs(scope): documentation changes
  • chore: or chore(scope): maintenance tasks
  • refactor: or refactor(scope): code refactoring
  • test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

@github-actions

Copy link
Copy Markdown

The following comment was made by an LLM, it may be inaccurate:

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/opencode/src/session/tools.ts">

<violation number="1" location="packages/opencode/src/session/tools.ts:108">
P1: When a generic MCP tool runs before `step-start`, this barrier does not protect it because only registry tools await the snapshot. A local MCP filesystem tool can therefore mutate the worktree before `snapshot.track()` finishes; await `input.processor.awaitSnapshot` in the MCP execute wrapper before the plugin hook and remote call.</violation>
</file>

<file name="packages/opencode/src/session/processor.ts">

<violation number="1" location="packages/opencode/src/session/processor.ts:132">
P2: `Effect.forkIn(scope)` forks the snapshot fiber into the shared SessionProcessor layer scope, not a per-turn scope, so aborting a turn does not cancel that turn's snapshot git work as the PR description claims. The fiber keeps running git subprocesses after the turn ends, and since `snapshot.track()` is globally mutex-locked it can delay the next turn's snapshot. To get the claimed per-turn interruption, capture the current turn's scope (e.g. `yield* Scope.Scope` inside `create`) and fork into that.</violation>

<violation number="2" location="packages/opencode/src/session/processor.ts:135">
P1: After the first provider step finishes, `awaitSnapshot` rejoins the already-completed initial snapshot fiber, so later `step-start` events record a stale pre-step snapshot. Rotate or re-track the barrier when `ctx.snapshot` is cleared so each provider step starts from the previous step's final tree.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Effect.gen(function* () {
const ctx = context(args, options)
// Tools can run before any step event; the pre-turn snapshot must land before a tool mutates files.
yield* input.processor.awaitSnapshot

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: When a generic MCP tool runs before step-start, this barrier does not protect it because only registry tools await the snapshot. A local MCP filesystem tool can therefore mutate the worktree before snapshot.track() finishes; await input.processor.awaitSnapshot in the MCP execute wrapper before the plugin hook and remote call.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/session/tools.ts, line 108:

<comment>When a generic MCP tool runs before `step-start`, this barrier does not protect it because only registry tools await the snapshot. A local MCP filesystem tool can therefore mutate the worktree before `snapshot.track()` finishes; await `input.processor.awaitSnapshot` in the MCP execute wrapper before the plugin hook and remote call.</comment>

<file context>
@@ -104,6 +104,8 @@ export const resolve = Effect.fn("SessionTools.resolve")(function* (input: {
           Effect.gen(function* () {
             const ctx = context(args, options)
+            // Tools can run before any step event; the pre-turn snapshot must land before a tool mutates files.
+            yield* input.processor.awaitSnapshot
             yield* plugin.trigger(
               "tool.execute.before",
</file context>

Effect.forkIn(scope),
)
const awaitSnapshot = Effect.gen(function* () {
if (ctx.snapshot === undefined) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: After the first provider step finishes, awaitSnapshot rejoins the already-completed initial snapshot fiber, so later step-start events record a stale pre-step snapshot. Rotate or re-track the barrier when ctx.snapshot is cleared so each provider step starts from the previous step's final tree.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/session/processor.ts, line 135:

<comment>After the first provider step finishes, `awaitSnapshot` rejoins the already-completed initial snapshot fiber, so later `step-start` events record a stale pre-step snapshot. Rotate or re-track the barrier when `ctx.snapshot` is cleared so each provider step starts from the previous step's final tree.</comment>

<file context>
@@ -127,6 +123,21 @@ const layer = Layer.effect(
+        Effect.forkIn(scope),
+      )
+      const awaitSnapshot = Effect.gen(function* () {
+        if (ctx.snapshot === undefined) {
+          ctx.snapshot = yield* Fiber.join(snapshotFiber)
+        }
</file context>

const snapshotFiber = yield* snapshot.track().pipe(
Effect.tap(() => Effect.sync(() => (ctx.snapshotMs = Date.now() - snapshotStart))),
Effect.orDie,
Effect.forkIn(scope),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Effect.forkIn(scope) forks the snapshot fiber into the shared SessionProcessor layer scope, not a per-turn scope, so aborting a turn does not cancel that turn's snapshot git work as the PR description claims. The fiber keeps running git subprocesses after the turn ends, and since snapshot.track() is globally mutex-locked it can delay the next turn's snapshot. To get the claimed per-turn interruption, capture the current turn's scope (e.g. yield* Scope.Scope inside create) and fork into that.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/session/processor.ts, line 132:

<comment>`Effect.forkIn(scope)` forks the snapshot fiber into the shared SessionProcessor layer scope, not a per-turn scope, so aborting a turn does not cancel that turn's snapshot git work as the PR description claims. The fiber keeps running git subprocesses after the turn ends, and since `snapshot.track()` is globally mutex-locked it can delay the next turn's snapshot. To get the claimed per-turn interruption, capture the current turn's scope (e.g. `yield* Scope.Scope` inside `create`) and fork into that.</comment>

<file context>
@@ -127,6 +123,21 @@ const layer = Layer.effect(
+      const snapshotFiber = yield* snapshot.track().pipe(
+        Effect.tap(() => Effect.sync(() => (ctx.snapshotMs = Date.now() - snapshotStart))),
+        Effect.orDie,
+        Effect.forkIn(scope),
+      )
+      const awaitSnapshot = Effect.gen(function* () {
</file context>

@PierrotAWB PierrotAWB mentioned this pull request Aug 24, 2026
4 tasks done
@PierrotAWB

Copy link
Copy Markdown
Collaborator Author

Consolidated into #29 (one commit per concern there; review history for the prep_ms retry semantics is on this PR's threads).

@PierrotAWB PierrotAWB closed this Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant