Repository navigation
fix(opencode): persist question requests in SQLite - #22
Conversation
Question requests (que_*) lived only in process memory, so any opencode restart under a parked question orphaned the coordinator's durable pointer: every answer submit then 404ed (~44% of answerQuestion attempts in prod over 30 days). Requests now persist in a question_request table; replies to requests whose waiter died with the process complete the interrupted tool part with the real answers and re-enter the assistant loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Seed real project/session rows in question tests (the new question_request table enforces its session FK), assert reply outcomes, and add coverage for rejectAllForSession plus the reload-survival + orphaned-reply heal path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
|
The following comment was made by an LLM, it may be inaccurate: |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
7 issues found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/test/question/question.test.ts">
<violation number="1" location="packages/opencode/test/question/question.test.ts:490">
P3: The test titled "request survives instance reload and reply heals it" never exercises the healing path. It calls Question.reply directly, whose orphaned branch only deletes the row and publishes Question.Event.Replied; the actual healing (resumeOrphanedReply, which replays the tool output and resumes the session) is invoked only by the HTTP reply handler and never runs in this test. The test therefore verifies persistence and the orphaned outcome but leaves the PR's central healing behavior untested. Either invoke the handler/healing path (e.g. via the HTTP API or by calling resumeOrphanedReply) and assert the message part is completed and the session loop resumes, or rename the test to reflect that it only checks orphaned-reply survival, not healing.</violation>
<violation number="2" location="packages/opencode/test/question/question.test.ts:523">
P3: The test named "request survives instance reload and reply heals it" never exercises the healing itself. `replyEffect` only calls `Question.Service.reply`, which for an orphaned request returns `{ outcome: "orphaned", request }` and deletes the row; the actual heal (`resumeOrphanedReply` in question/resume.ts, which updates the tool part, replays the output, and resumes `SessionPrompt.loop`) is invoked only in the HTTP handler `src/server/routes/instance/httpapi/handlers/question.ts` via `Effect.forkIn(scope)`. As written, the test asserts durability (row survives reload) plus the orphaned reply data path, but a regression in the resume/replay logic would pass this test undetected. Either rename the test to describe what it actually covers or wire in the healing path / handler to make the claimed coverage real.</violation>
</file>
<file name="packages/opencode/src/question/format.ts">
<violation number="1" location="packages/opencode/src/question/format.ts:5">
P3: This file is an exact duplicate of `toModelOutput` in `packages/core/src/tool/question.ts` (lines 40-48): identical format string, identical join separator, identical `Unanswered` fallback and identical output sentence. Maintain both copies means future format changes (or the quote-escaping bug) must be applied twice and can drift. Reuse one implementation instead of forking the same logic into `packages/opencode/src/question/format.ts`; e.g. export `toModelOutput` from `@opencode-ai/core` and call it from both `tool/question.ts` and `question/resume.ts`.</violation>
</file>
<file name="packages/opencode/src/question/resume.ts">
<violation number="1" location="packages/opencode/src/question/resume.ts:35">
P2: When answers exceed the configured tool-output limits, orphan healing stores a different, untruncated tool result than the live `QuestionTool` path. Run the resumed result through the same truncation pipeline, including its metadata and output path, before updating the part.</violation>
<violation number="2" location="packages/opencode/src/question/resume.ts:45">
P1: When an orphaned reply arrives while the session is busy, this branch consumes the reply but drops the healing attempt. Queue the resume for after the current runner becomes idle, or retain/retry the request until a resume is scheduled.</violation>
</file>
<file name="packages/opencode/src/server/routes/instance/httpapi/handlers/question.ts">
<violation number="1" location="packages/opencode/src/server/routes/instance/httpapi/handlers/question.ts:39">
P1: When a request ID is replied to through another directory or project, this branch resumes it using the reply handler’s instance context. Validate that the persisted request belongs to the routed instance before accepting and replaying it.</violation>
<violation number="2" location="packages/opencode/src/server/routes/instance/httpapi/handlers/question.ts:41">
P1: When `resumeOrphanedReply` fails, this catch only logs the error after the durable row has been deleted. Requeue failed orphan replies or retain the row until healing succeeds, otherwise the answer is lost and the session remains incomplete.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| } | ||
|
|
||
| const current = yield* status.get(input.request.sessionID) | ||
| if (current.type !== "idle") { |
There was a problem hiding this comment.
P1: When an orphaned reply arrives while the session is busy, this branch consumes the reply but drops the healing attempt. Queue the resume for after the current runner becomes idle, or retain/retry the request until a resume is scheduled.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/question/resume.ts, line 45:
<comment>When an orphaned reply arrives while the session is busy, this branch consumes the reply but drops the healing attempt. Queue the resume for after the current runner becomes idle, or retain/retry the request until a resume is scheduled.</comment>
<file context>
@@ -0,0 +1,50 @@
+ }
+
+ const current = yield* status.get(input.request.sessionID)
+ if (current.type !== "idle") {
+ yield* Effect.logInfo("skipping question resume, session busy", { sessionID: input.request.sessionID })
+ return
</file context>
| ), | ||
| ), | ||
| ) | ||
| if (result.outcome === "orphaned") { |
There was a problem hiding this comment.
P1: When a request ID is replied to through another directory or project, this branch resumes it using the reply handler’s instance context. Validate that the persisted request belongs to the routed instance before accepting and replaying it.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/server/routes/instance/httpapi/handlers/question.ts, line 39:
<comment>When a request ID is replied to through another directory or project, this branch resumes it using the reply handler’s instance context. Validate that the persisted request belongs to the routed instance before accepting and replaying it.</comment>
<file context>
@@ -32,6 +36,14 @@ export const questionHandlers = HttpApiBuilder.group(InstanceHttpApi, "question"
),
),
)
+ if (result.outcome === "orphaned") {
+ yield* resumeOrphanedReply({ request: result.request, answers: ctx.payload.answers }).pipe(
+ Effect.catchCause((cause) =>
</file context>
| ) | ||
| if (result.outcome === "orphaned") { | ||
| yield* resumeOrphanedReply({ request: result.request, answers: ctx.payload.answers }).pipe( | ||
| Effect.catchCause((cause) => |
There was a problem hiding this comment.
P1: When resumeOrphanedReply fails, this catch only logs the error after the durable row has been deleted. Requeue failed orphan replies or retain the row until healing succeeds, otherwise the answer is lost and the session remains incomplete.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/server/routes/instance/httpapi/handlers/question.ts, line 41:
<comment>When `resumeOrphanedReply` fails, this catch only logs the error after the durable row has been deleted. Requeue failed orphan replies or retain the row until healing succeeds, otherwise the answer is lost and the session remains incomplete.</comment>
<file context>
@@ -32,6 +36,14 @@ export const questionHandlers = HttpApiBuilder.group(InstanceHttpApi, "question"
)
+ if (result.outcome === "orphaned") {
+ yield* resumeOrphanedReply({ request: result.request, answers: ctx.payload.answers }).pipe(
+ Effect.catchCause((cause) =>
+ Effect.logError("question resume failed", { requestID: ctx.params.requestID, cause }),
+ ),
</file context>
| state: { | ||
| status: "completed", | ||
| input: part.state.input, | ||
| output: formatAnswerOutput({ questions: input.request.questions, answers: input.answers }), |
There was a problem hiding this comment.
P2: When answers exceed the configured tool-output limits, orphan healing stores a different, untruncated tool result than the live QuestionTool path. Run the resumed result through the same truncation pipeline, including its metadata and output path, before updating the part.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/question/resume.ts, line 35:
<comment>When answers exceed the configured tool-output limits, orphan healing stores a different, untruncated tool result than the live `QuestionTool` path. Run the resumed result through the same truncation pipeline, including its metadata and output path, before updating the part.</comment>
<file context>
@@ -0,0 +1,50 @@
+ state: {
+ status: "completed",
+ input: part.state.input,
+ output: formatAnswerOutput({ questions: input.request.questions, answers: input.answers }),
+ title: `Asked ${input.request.questions.length} question${input.request.questions.length > 1 ? "s" : ""}`,
+ metadata: { answers: input.answers.map((answer) => [...answer]) },
</file context>
| { git: true }, | ||
| ) | ||
|
|
||
| lifecycle.live("request survives instance reload and reply heals it", () => |
There was a problem hiding this comment.
P3: The test titled "request survives instance reload and reply heals it" never exercises the healing path. It calls Question.reply directly, whose orphaned branch only deletes the row and publishes Question.Event.Replied; the actual healing (resumeOrphanedReply, which replays the tool output and resumes the session) is invoked only by the HTTP reply handler and never runs in this test. The test therefore verifies persistence and the orphaned outcome but leaves the PR's central healing behavior untested. Either invoke the handler/healing path (e.g. via the HTTP API or by calling resumeOrphanedReply) and assert the message part is completed and the session loop resumes, or rename the test to reflect that it only checks orphaned-reply survival, not healing.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/test/question/question.test.ts, line 490:
<comment>The test titled "request survives instance reload and reply heals it" never exercises the healing path. It calls Question.reply directly, whose orphaned branch only deletes the row and publishes Question.Event.Replied; the actual healing (resumeOrphanedReply, which replays the tool output and resumes the session) is invoked only by the HTTP reply handler and never runs in this test. The test therefore verifies persistence and the orphaned outcome but leaves the PR's central healing behavior untested. Either invoke the handler/healing path (e.g. via the HTTP API or by calling resumeOrphanedReply) and assert the message part is completed and the session loop resumes, or rename the test to reflect that it only checks orphaned-reply survival, not healing.</comment>
<file context>
@@ -457,3 +462,71 @@ lifecycle.live("pending question rejects on instance reload", () =>
+ { git: true },
+)
+
+lifecycle.live("request survives instance reload and reply heals it", () =>
+ Effect.gen(function* () {
+ const dir = yield* tmpdirScoped({ git: true })
</file context>
| return Effect.void | ||
| }) | ||
| yield* Effect.addFinalizer(() => off) | ||
| const outcome = yield* replyEffect({ requestID: pending[0].id, answers: [["Yes"]] }) |
There was a problem hiding this comment.
P3: The test named "request survives instance reload and reply heals it" never exercises the healing itself. replyEffect only calls Question.Service.reply, which for an orphaned request returns { outcome: "orphaned", request } and deletes the row; the actual heal (resumeOrphanedReply in question/resume.ts, which updates the tool part, replays the output, and resumes SessionPrompt.loop) is invoked only in the HTTP handler src/server/routes/instance/httpapi/handlers/question.ts via Effect.forkIn(scope). As written, the test asserts durability (row survives reload) plus the orphaned reply data path, but a regression in the resume/replay logic would pass this test undetected. Either rename the test to describe what it actually covers or wire in the healing path / handler to make the claimed coverage real.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/test/question/question.test.ts, line 523:
<comment>The test named "request survives instance reload and reply heals it" never exercises the healing itself. `replyEffect` only calls `Question.Service.reply`, which for an orphaned request returns `{ outcome: "orphaned", request }` and deletes the row; the actual heal (`resumeOrphanedReply` in question/resume.ts, which updates the tool part, replays the output, and resumes `SessionPrompt.loop`) is invoked only in the HTTP handler `src/server/routes/instance/httpapi/handlers/question.ts` via `Effect.forkIn(scope)`. As written, the test asserts durability (row survives reload) plus the orphaned reply data path, but a regression in the resume/replay logic would pass this test undetected. Either rename the test to describe what it actually covers or wire in the healing path / handler to make the claimed coverage real.</comment>
<file context>
@@ -457,3 +462,71 @@ lifecycle.live("pending question rejects on instance reload", () =>
+ return Effect.void
+ })
+ yield* Effect.addFinalizer(() => off)
+ const outcome = yield* replyEffect({ requestID: pending[0].id, answers: [["Yes"]] })
+ expect(outcome.outcome).toBe("orphaned")
+ return yield* Queue.take(seen).pipe(Effect.timeout("2 seconds"))
</file context>
| questions: ReadonlyArray<{ question: string }> | ||
| answers: ReadonlyArray<ReadonlyArray<string>> | ||
| }) { | ||
| const formatted = input.questions |
There was a problem hiding this comment.
P3: This file is an exact duplicate of toModelOutput in packages/core/src/tool/question.ts (lines 40-48): identical format string, identical join separator, identical Unanswered fallback and identical output sentence. Maintain both copies means future format changes (or the quote-escaping bug) must be applied twice and can drift. Reuse one implementation instead of forking the same logic into packages/opencode/src/question/format.ts; e.g. export toModelOutput from @opencode-ai/core and call it from both tool/question.ts and question/resume.ts.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/question/format.ts, line 5:
<comment>This file is an exact duplicate of `toModelOutput` in `packages/core/src/tool/question.ts` (lines 40-48): identical format string, identical join separator, identical `Unanswered` fallback and identical output sentence. Maintain both copies means future format changes (or the quote-escaping bug) must be applied twice and can drift. Reuse one implementation instead of forking the same logic into `packages/opencode/src/question/format.ts`; e.g. export `toModelOutput` from `@opencode-ai/core` and call it from both `tool/question.ts` and `question/resume.ts`.</comment>
<file context>
@@ -0,0 +1,9 @@
+ questions: ReadonlyArray<{ question: string }>
+ answers: ReadonlyArray<ReadonlyArray<string>>
+}) {
+ const formatted = input.questions
+ .map((q, i) => `"${q.question}"="${input.answers[i]?.length ? input.answers[i].join(", ") : "Unanswered"}"`)
+ .join(", ")
</file context>
Concurrent replies (or reply racing reject) could both read the same row and double-publish before either delete landed; deleting with returning() makes the row delete the single claim, restoring the pre-persistence double-submit semantics. Also trims review comments to one line. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| @@ -111,51 +136,92 @@ const layer = Layer.effect( | |||
| ) | |||
There was a problem hiding this comment.
🔍 Persisted rows intentionally linger until reply/reject/cancel/cascade
The ask cleanup in packages/opencode/src/question/index.ts:131-136 only removes the in-memory pending entry, never the DB row. Combined with the dispose finalizer at packages/opencode/src/question/index.ts:90-97 (which fails deferreds but keeps rows), a question_request row is only ever deleted by claim (reply/reject), rejectAllForSession (invoked from SessionPrompt.cancel at packages/opencode/src/session/prompt.ts:154-158), or FK cascade when the session is deleted. This means any assistant-turn interruption that does NOT route through prompt.cancel (e.g. an internal stream error interrupting the question tool fiber mid-wait) leaves a durable row behind that will surface in list() and can later be answered, triggering resumeOrphanedReply. Per the PR description this durability is the intended survival mechanism, so it is not a leak in the shutdown/reload case, but reviewers may want to confirm that non-cancel interruption paths producing a lingering answerable question are acceptable UX.
(Refers to lines 131-136)
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
1 issue found across 5 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/src/question/index.ts">
<violation number="1" location="packages/opencode/src/question/index.ts:142">
P2: When rejecting a session with multiple pending questions, an interruption or failed `Event.Rejected` publication after this bulk delete removes every request before their waiters are failed. Process each row through `claim` before `rejectClaimed`, so unprocessed requests remain durable and recoverable.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
| // Deleting the row is the claim: whichever concurrent reply/reject wins the delete owns the request. | ||
| const claim = Effect.fn("Question.claim")(function* (requestID: QuestionID) { | ||
| return yield* db | ||
| .delete(QuestionRequestTable) |
There was a problem hiding this comment.
P2: When rejecting a session with multiple pending questions, an interruption or failed Event.Rejected publication after this bulk delete removes every request before their waiters are failed. Process each row through claim before rejectClaimed, so unprocessed requests remain durable and recoverable.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/question/index.ts, line 142:
<comment>When rejecting a session with multiple pending questions, an interruption or failed `Event.Rejected` publication after this bulk delete removes every request before their waiters are failed. Process each row through `claim` before `rejectClaimed`, so unprocessed requests remain durable and recoverable.</comment>
<file context>
@@ -138,21 +136,25 @@ const layer = Layer.effect(
+ // Deleting the row is the claim: whichever concurrent reply/reject wins the delete owns the request.
+ const claim = Effect.fn("Question.claim")(function* (requestID: QuestionID) {
+ return yield* db
+ .delete(QuestionRequestTable)
+ .where(eq(QuestionRequestTable.id, requestID))
+ .returning()
</file context>
|
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
Claiming a question row now requires the row's session to belong to the instance's project (same scoping as list), so co-located instances sharing the machine-global DB cannot consume each other's live requests. A reply whose Deferred.succeed loses to the instance-disposal finalizer now falls through to the orphaned outcome instead of reporting resolved for a waiter that was already failed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three prompt-harness tests for resumeOrphanedReply: the heal completes the dangling tool part with the shared answer format and re-enters the loop, busy sessions complete the part but skip the loop, and a missing tool part is tolerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
packages/core's database-migration tests (snapshot parity --check plus the migration corpus) now run in the fork-ci test job; nothing gated the schema artifacts before this. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
What:
que_*question requests move from process memory into SQLite (newquestion_requesttable). A restart no longer kills a parked question — answering it now returns 200, fills the interrupted tool call with the real answers, and the turn resumes.Why: 44% of prod
answerQuestionattempts (210/480, 30d) failed because sandbox restarts wiped the in-memory registry while the coordinator's durablependingQuestionIdsurvived. Slack thread.How:
replywith a live waiter resolves exactly as before. With no waiter (restart happened): delete the row, publishquestion.replied, heal. A reply that loses the live-waiter race to instance disposal falls through to the same orphaned path instead of claiming success.questiontool part with the normal answer output, then re-enterSessionPrompt.loop(skipped if the session is busy — a busy session's live loop re-reads the transcript each step and absorbs the answers itself). Forked into the server scope so it outlives the HTTP request.rejectAllForSessionin cancel). Process shutdown intentionally does not — that's the survival mechanism.sessionwith cascade delete;list()and the delete-claim are both project-scoped likeSession.list, so co-located instances sharing the machine-global DB can't consume each other's live requests.Wire contract: unchanged — no coordinator or SDK changes. The coordinator already clears its slot off the
question.repliedevent.Accepted loss windows (adversarially reviewed): a hard kill inside the few-ms gap between the row claim and the tool-part write, or SQLite failing mid-heal, still loses that one answer — strictly narrower than the pre-PR baseline where every restart-parked answer died with a dead-end 404. Closing it would take Question→Session layering (a LayerNode cycle) or a two-phase claim that reintroduces the double-publish race the delete-returning claim prevents. Also inherent: rows outliving processes is the survival mechanism, so in upstream-style multi-server local setups a second process on the same project can claim and heal a question the first process asked; single-process sandboxes are unaffected.
Verified:
v1.17.14-pr22.1: parked question → opencode hard-killed → answer succeeded and the turn resumed (on the fleet build this exact click dead-ends with a 404).Migration: additive table, created automatically by the migration runner on each machine's next boot. An older binary against the newer DB is safe (unknown applied migrations are ignored). Rollback = revert.
Not in scope: QuestionV2 (core/server rewrite) parity; coordinator-side stale-slot reconcile; TUI dock rehydration from the now-durable
question.list.Next: merge → cut
v1.17.14-3per RELEASING.md → bump andytown's two pins (DockerfileOPENCODE_VERSION+sandbox-upgrader/src/opencode-version.ts).🤖 Generated with Claude Code