diff --git a/CHANGELOG.md b/CHANGELOG.md index f6be131..8115dda 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,27 @@ # Changelog +## [0.15.0] — 2026-08-31 + +A turn cut short is no longer thrown away. Whatever it produced is persisted and +marked incomplete, and it is no longer logged as a clean success. + +### Breaking Changes +- `ai_interaction_logs.status` has a new value, `aborted` (`AiInteractionStatus::Aborted`), used for a turn whose caller hung up mid-stream. It was previously recorded as `success`. Code matching exhaustively on the enum must handle the new case, and dashboards that count everything other than `success` as a failure will now count cancellations among them. The tokens an aborted turn burned still count towards conversation usage totals. +- The `message_delta` event's `delta.stop_reason` has a new value, `incomplete`, for a turn that never finished — the caller hung up, or the max-stream-duration guard cut the generation off. Such turns previously reported `end_turn`, which was indistinguishable from a clean finish. The same value is stored on `ai_llm_messages.response_data.stop_reason`. +- An interrupted turn now leaves an assistant message in the transcript even when it produced no text, so `ChatBotPresenter::transcript()` can return assistant rows whose `content` is empty. Each row carries a new `incomplete` boolean; render a flagged row as an interrupted reply rather than as a blank answer. +- Two new migrations (`ai_turn_runs`, `ai_turn_events`) ship with this release. Re-publish migrations and run them, even if you do not use the dispatched-turn API — `ai:prune-turn-events` is scheduled by default and expects the tables. + +### New Features +- An interrupted assistant message records why in its `metadata`: `incomplete` plus an `incomplete_reason` of `client_aborted` or `max_stream_duration`. +- Turns now emit a heartbeat while the provider is silent (`conversations.heartbeat_seconds`, default 5, `0` to disable). It is encoded as an SSE comment, so existing clients ignore it; it keeps intermediaries from timing out mid-answer and lets an abandoned turn be noticed in seconds rather than minutes. Currently emitted for `openai-compatible` and `lm-studio` systems; a turn dispatched as a job heartbeats for every provider. +- Added `AiPersonaConversationService::dispatchTurn()`, `resumeTurn()` and `cancelTurn()`: a turn can run as a queued job that records its events, so a browser reload resumes it instead of killing it. Events are framed with an SSE `id:` carrying their sequence, and the published client reports it through a new `onSequence` callback. Requires the two new migrations and a queue worker. +- Added `ai:prune-turn-events` (scheduled daily at 03:15) to clear finished turn runs past `turns.retention_days`. + +### Bug Fixes +- A turn interrupted before the model emitted any text is no longer discarded entirely. It previously left the user's message in the transcript with no reply beneath it and no record that a turn had ever run — the failure mode was most visible with a large context, where a model can spend minutes processing the prompt before its first token. +- Tool calls an interrupted turn already made are now persisted with it. They were dropped along with the rest of the turn, leaving the next turn's history with no record that a tool had run and changed state. +- The maximum-stream-duration guard now applies during provider silence. It was only evaluated when an event arrived, so a stalled stream could run well past `conversations.max_stream_seconds` before anything noticed. + ## [0.14.1] — 2026-08-30 ### New Features diff --git a/README.md b/README.md index 0e64828..6e7d3b8 100644 --- a/README.md +++ b/README.md @@ -149,6 +149,43 @@ Every laravel/ai HTTP request/response can be captured verbatim into the Request bodies and response bytes are stored, but request **headers are never recorded**, so provider API keys are not persisted. +### Detached Turns + +A turn dispatched with `dispatchTurn()` runs as a queued job and writes its +events to the `ai_turn_events` table, so a browser reload resumes it instead of +killing it (see [Running a turn as a job](#running-a-turn-as-a-job)). + +```php +'turns' => [ + 'queue' => env('CODE_TALKER_TURN_QUEUE'), + 'abandon_after_seconds' => (int) env('CODE_TALKER_TURN_ABANDON_SECONDS', 30), + 'poll_interval_ms' => (int) env('CODE_TALKER_TURN_POLL_MS', 250), + 'max_stream_seconds' => (int) env('CODE_TALKER_TURN_MAX_STREAM_SECONDS', 900), + 'retention_days' => (int) env('CODE_TALKER_TURN_RETENTION_DAYS', 7), +], +``` + +- `queue` — the queue `RunConversationTurnJob` is dispatched on; `null` uses + the default queue. +- `abandon_after_seconds` — a running turn stops when nobody has read its + events for this long. `connection_aborted()` reports 0 in a worker, so this + is what stops a turn nobody is waiting for. +- `poll_interval_ms` — how often a reader polls the store for new events. +- `max_stream_seconds` — ceiling for a single `resumeTurn()` read before it + ends with a `max_stream_duration` error; reconnecting starts a fresh window. +- `retention_days` — finished runs older than this are removed by + `php artisan ai:prune-turn-events`, scheduled daily at 03:15 (respects the + `schedule` flag). + +Note that `turns.max_stream_seconds` bounds only the read side. Generation +inside the worker is governed by `conversations.max_stream_seconds` (default +300), which caps each individual provider request — the same guard the +synchronous path applies, enforced promptly during provider silence by the +heartbeat rather than only when the next provider event arrives. A host running +a large-context local model, where prompt processing alone can occupy minutes +of a single request, should raise `conversations.max_stream_seconds` +accordingly. + ### Troubleshooting **`Provider is unavailable: HTTP request returned status code 404`** — returned @@ -345,8 +382,9 @@ Every event carries a `type`. These are typed in the published declarations. | `message_start` | — | | `content_block_delta` | `delta.text` | | `reasoning_block_delta` | `delta.reasoning` | -| `message_delta` | `delta.stop_reason`, `usage` | +| `message_delta` | `delta.stop_reason` (`end_turn`/`max_tokens`/`tool_use`/`incomplete`), `usage` | | `message_stop` | — | +| `heartbeat` | — (encoded as an SSE comment, not a data frame) | | `tool_use_progress` | `text` (always `""`), `tools` (one tool name per event), plus `input`/`output`/`successful` when tool payloads are enabled | | `page_reload` | — | | `error` | `message`, `reason` (`max_stream_duration`/`provider_error`) | @@ -357,6 +395,20 @@ aren't display text), so without this a turn calling a tool, especially one retrying after an error, streams nothing but silence between text/reasoning deltas. +`stop_reason` is `incomplete` when the turn never finished — the connection +dropped, or the server's duration guard cut the generation off. Whatever +content arrived stops mid-answer, and the turn is stored that way (see +[Interrupted turns](#interrupted-turns)). + +`heartbeat` fires while the provider is silent. `SseFrameEncoder` renders it as +`: ping` — an SSE comment — so browsers and the published client ignore it +without any handling. It is there so something reaches the socket during a long +gap: intermediaries stop timing out mid-answer, and PHP only flips +`connection_aborted()` after a write to a dead connection, so without it an +abandoned turn keeps generating until the model's next event. Set +`conversations.heartbeat_seconds` to `0` to disable. Detection costs two beats: +the first write marks the socket dead, the second observes it. + `page_reload` fires when a tool's structured result carries `_page_reload: true` — see [Tool Registration](#tool-registration) for how a tool sets it. Deciding what "reload" means (call `location.reload()` immediately, wait for @@ -386,6 +438,57 @@ $chat->usingCancellationCheck(fn (): bool => $job->isReleased()) ->continueConversation($conversation, $message); ``` +### Interrupted turns + +A turn that stops before the model finishes — the browser hung up, or the +duration guard tripped — is still recorded, whatever it had produced: + +- The assistant message is persisted even when it holds no text at all, so a + user's question is never left with nothing beneath it. Its `metadata` carries + `incomplete: true` and an `incomplete_reason` of `client_aborted` or + `max_stream_duration`, and `ChatBotPresenter::transcript()` surfaces the flag + as `incomplete` on the row. Render it as an interrupted reply rather than as + an answer. +- Tool calls the model made before the stop are persisted with it. A tool that + ran changed state on your side; dropping the turn would leave the next turn's + history with no record it ever happened. +- `AiInteractionLog::status` is `aborted` (not `success`) for a turn the caller + hung up on, with `provider_metadata.error_reason` set to `client_aborted`. + The tokens it burned still count towards the conversation's usage totals — + hanging up does not refund what the provider already generated. + +### Running a turn as a job + +`continueConversation()` ties the turn to the caller's connection: close the +tab and the turn stops, reload and it is gone. For turns long enough that this +matters, dispatch the turn instead and stream it from its store. + +```php +// Start it. Returns an AiTurnRun; `public_id` is the handle to put in a URL. +$run = $chat->dispatchTurn($conversation, $request->string('message')->toString()); + +// Stream it — from the start, or from wherever the browser left off. +foreach ($encoder->encode($chat->resumeTurn($run, $after)) as $frame) { + echo $frame; + ob_get_level() > 0 && ob_flush(); + flush(); +} + +// Stop it early. +$chat->cancelTurn($run); +``` + +Each event is framed with an SSE `id:` carrying its sequence. A browser that +reconnects passes the last sequence it saw back as `after`, and the turn +resumes rather than replaying. The published client reports it via +`onSequence`. + +A dispatched turn needs a queue worker. Because `connection_aborted()` reports +0 in a worker, a run stops when nobody has read it for +`turns.abandon_after_seconds` (default 30) — so closing the tab still stops +generation, and a reload inside that window reattaches to the same run. +`ai:prune-turn-events` clears finished runs past `turns.retention_days`. + ### Resolving conversations across requests The package used to keep this in the session and a cookie. It no longer does — @@ -399,6 +502,7 @@ and `$conversation->chat_hash` is a stable shareable handle that ```php $presenter->transcript($conversation); // visible messages, system prompt excluded + // each row carries `incomplete` — see Interrupted turns $presenter->totalCostUsd($persona); // lifetime cost for a persona $presenter->conversationsFor($user, $personas); // an authenticated user's conversations ``` @@ -966,13 +1070,14 @@ route file, and defaults to `['web', 'auth', 'can:manage-ai-tools']`. ## Scheduled Jobs -The package registers four jobs automatically (requires Laravel's scheduler to be running): +The package registers five jobs automatically (requires Laravel's scheduler to be running): | Job | Schedule | Description | | -------------------------------- | -------------------------- | ------------------------------------------------------- | | `ai:sync-conversation-usage` | Twice daily (00:00, 12:00) | Syncs token counts and cost to `AiConversation` | | `BackfillConversationUsageJob` | Daily at 02:30 | Backfills usage for conversations missing cost data | | `ai:prune-provider-exchanges` | Daily at 03:00 | Removes `ai_provider_exchanges` rows past retention | +| `ai:prune-turn-events` | Daily at 03:15 | Removes finished turn runs past `turns.retention_days` | | `ai:complete-idle-conversations` | Every 15 minutes | Completes idle conversations, triggering memory extract | Disable automatic scheduling in config and register manually if needed: @@ -1000,6 +1105,9 @@ php artisan ai:sync-conversation-usage # Delete ai_provider_exchanges rows older than raw_exchanges.retention_days php artisan ai:prune-provider-exchanges +# Delete finished turn runs (and their events) older than turns.retention_days +php artisan ai:prune-turn-events + # Mark idle conversations Completed, triggering memory extraction php artisan ai:complete-idle-conversations php artisan ai:complete-idle-conversations --minutes=60 --dry-run diff --git a/config/code-talker.php b/config/code-talker.php index adf2173..9babae9 100644 --- a/config/code-talker.php +++ b/config/code-talker.php @@ -103,6 +103,14 @@ 'conversations' => [ 'idle_timeout_minutes' => (int) env('CODE_TALKER_CONVERSATION_IDLE_MINUTES', 30), + // Seconds of provider silence before the turn emits a heartbeat. + // Two things depend on it: intermediaries stop timing out during a + // long gap, and PHP only flips connection_aborted() after a write to + // a dead socket — so without a heartbeat an abandoned turn keeps + // generating until the model's next event, which on a large context + // can be minutes. Set to 0 to disable. + 'heartbeat_seconds' => (int) env('CODE_TALKER_HEARTBEAT_SECONDS', 5), + // Wall-clock ceiling (seconds) for a single streamed chat turn, across // all tool steps and continuation attempts. Guards against a runaway // generation — e.g. a reasoning model that loops until it overflows the @@ -331,4 +339,25 @@ 'retention_days' => (int) env('CODE_TALKER_RAW_EXCHANGES_RETENTION_DAYS', 14), ], + /* + |-------------------------------------------------------------------------- + | Detached Turns + |-------------------------------------------------------------------------- + | + | A turn dispatched with AiPersonaConversationService::dispatchTurn() runs + | as a queued job and writes its events to ai_turn_events, so a browser + | reload resumes the turn instead of killing it. connection_aborted() is + | meaningless in a worker, so "nobody has polled for abandon_after_seconds" + | is what stops a turn nobody is waiting for. + | + */ + + 'turns' => [ + 'queue' => env('CODE_TALKER_TURN_QUEUE'), + 'abandon_after_seconds' => (int) env('CODE_TALKER_TURN_ABANDON_SECONDS', 30), + 'poll_interval_ms' => (int) env('CODE_TALKER_TURN_POLL_MS', 250), + 'max_stream_seconds' => (int) env('CODE_TALKER_TURN_MAX_STREAM_SECONDS', 900), + 'retention_days' => (int) env('CODE_TALKER_TURN_RETENTION_DAYS', 7), + ], + ]; diff --git a/database/migrations/2026_08_31_000001_create_ai_turn_runs_table.php b/database/migrations/2026_08_31_000001_create_ai_turn_runs_table.php new file mode 100644 index 0000000..3f15a55 --- /dev/null +++ b/database/migrations/2026_08_31_000001_create_ai_turn_runs_table.php @@ -0,0 +1,33 @@ +id(); + $table->string('public_id', 40)->unique(); + $table->foreignId('ai_conversation_id')->index(); + $table->string('status', 20)->index(); + $table->text('prompt'); + // The abandonment signal: connection_aborted() reports 0 in a + // worker, so "nobody is reading this" is the only usable stand-in + // for the browser having gone away. + $table->timestamp('last_polled_at')->nullable(); + $table->timestamp('cancel_requested_at')->nullable(); + $table->timestamp('started_at')->nullable(); + $table->timestamp('finished_at')->nullable(); + $table->text('error_message')->nullable(); + $table->timestamps(); + }); + } + + public function down(): void + { + Schema::dropIfExists('ai_turn_runs'); + } +}; diff --git a/database/migrations/2026_08_31_000002_create_ai_turn_events_table.php b/database/migrations/2026_08_31_000002_create_ai_turn_events_table.php new file mode 100644 index 0000000..fccd951 --- /dev/null +++ b/database/migrations/2026_08_31_000002_create_ai_turn_events_table.php @@ -0,0 +1,28 @@ +id(); + $table->foreignId('ai_turn_run_id')->index(); + $table->unsignedInteger('sequence'); + $table->json('payload'); + $table->timestamp('created_at')->nullable(); + + // The reader asks for everything after a sequence it already + // holds, so a duplicate would silently replay or skip output. + $table->unique(['ai_turn_run_id', 'sequence']); + }); + } + + public function down(): void + { + Schema::dropIfExists('ai_turn_events'); + } +}; diff --git a/docs/superpowers/plans/2026-08-31-durable-turns-and-heartbeats.md b/docs/superpowers/plans/2026-08-31-durable-turns-and-heartbeats.md new file mode 100644 index 0000000..94aab51 --- /dev/null +++ b/docs/superpowers/plans/2026-08-31-durable-turns-and-heartbeats.md @@ -0,0 +1,2552 @@ +# Stream Heartbeats and Durable Turns Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Stop a streamed chat turn from silently burning GPU after the browser leaves, and let a turn survive a page reload by running it as a queued job whose events the browser reads from a store. + +**Architecture:** Part A bounds laravel/ai's blocking SSE read with a stream timeout, emitting a `Heartbeat` stream event on each idle window; the runner forwards it as a `heartbeat` turn event and `SseFrameEncoder` renders it as an SSE comment frame, so a dead socket is detected in seconds. Part B adds `ai_turn_runs`/`ai_turn_events`, a `RunConversationTurnJob` that drives the *existing* `continueConversation()` and appends each yielded event, and a `TurnEventStream` reader a host's endpoint streams from at any sequence. The synchronous path is untouched. + +**Tech Stack:** PHP 8.3, Laravel package (Orchestra Testbench), PHPUnit 11, laravel/ai ^0.9, TypeScript (declarations + published client). + +**Spec:** `docs/superpowers/specs/2026-08-31-durable-turns-and-heartbeats-design.md` + +## Global Constraints + +- PHP `^8.3`; Laravel `^12.62 || ^13.15`; laravel/ai `^0.9` (vendor copies in `ReasoningOpenAiCompatibleGateway` are pinned to 0.9.0 — re-check on upgrade). +- Namespace: `Jvjvjv\CodeTalker`. +- Tests use direct `Model::create(...)` (no in-package factories) and extend `Jvjvjv\CodeTalker\Tests\TestCase`. +- `ai_conversations.uuid` is not created by any package migration; any test that creates a conversation must add the column in `setUp()`: + ```php + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + ``` +- `ai_llm_messages.turn_number` is a **string** column; use string values like `'1'`. +- `AiPersonaConversationService`'s constructor takes its **five** dependencies positionally (`AgentFactory`, `AiMemoryService`, `ConversationUsageService`, `RawExchangeContext`, `AiSystemProviderConfigurator`). Tests build anonymous subclasses with exactly that signature. New collaborators are constructed from those five **inside** the constructor — never added as parameters. +- `streamElapsedSeconds()` and `clientAborted()` stay `protected` on the service and reach `ConversationTurnRunner` as closures bound to `$this` (via `TurnGuards`). +- `continueConversation()` yields **structured event arrays**, never wire-encoded strings. `SseFrameEncoder` owns all framing. +- Every nested config read takes an inline default (`config('code-talker.conversations.heartbeat_seconds', 5)`) — Laravel skips `mergeConfigFrom` entirely when a host has cached config. +- Run tests with `vendor/bin/phpunit`. Run `npm run typecheck` after touching anything under `resources/js`. +- Commit messages end with: + `Co-Authored-By: Claude Opus 5 (1M context) ` +- Do **not** log security analysis in `CHANGELOG.md` or `README.md`. + +--- + +## File Structure + +**Part A — heartbeats** +- Create: `src/Services/LaravelAi/Streaming/Heartbeat.php` — the stream event; carries no payload beyond its id/timestamp. +- Create: `src/Services/LaravelAi/Concerns/HeartbeatsIdleSseReads.php` — the timeout-bounded SSE read with a partial-line buffer. +- Modify: `src/Services/LaravelAi/ReasoningOpenAiCompatibleGateway.php` — use the trait; pass a `Heartbeat` through `processTextStream()`. +- Modify: `src/Services/ChatBot/Conversation/ConversationTurnRunner.php` — treat a heartbeat as a tick, not an event. +- Modify: `src/Services/ChatBot/SseFrameEncoder.php` — render `heartbeat` as `": ping\n\n"`. +- Modify: `config/code-talker.php` — `conversations.heartbeat_seconds`. + +**Part B — durable turns** +- Create: `database/migrations/2026_08_31_000001_create_ai_turn_runs_table.php` +- Create: `database/migrations/2026_08_31_000002_create_ai_turn_events_table.php` +- Create: `src/Enums/AiTurnRunStatus.php` +- Create: `src/Models/AiTurnRun.php`, `src/Models/AiTurnEvent.php` +- Create: `src/Services/Conversation/TurnRunStore.php` — the only writer; owns sequencing and the stop signal. +- Create: `src/Jobs/RunConversationTurnJob.php` — drives `continueConversation()`, appends events. +- Create: `src/Services/Conversation/TurnEventStream.php` — the reader generator. +- Create: `src/Console/Commands/PruneTurnEventsCommand.php` +- Modify: `src/Services/AiPersonaConversationService.php` — `dispatchTurn()`, `resumeTurn()`, `cancelTurn()`. +- Modify: `src/Services/ChatBot/SseFrameEncoder.php` — `id:` lines from `_seq`. +- Modify: `src/CodeTalkerServiceProvider.php` — register + schedule the prune command. +- Modify: `config/code-talker.php` — the `turns` block. + +**Contract** +- Modify: `resources/js/types/code-talker.d.ts`, `resources/js/code-talker-stream.ts`, `README.md`, `CHANGELOG.md`. + +--- + +## Task 1: Heartbeat stream event and the idle-read override + +**Files:** +- Create: `src/Services/LaravelAi/Streaming/Heartbeat.php` +- Create: `src/Services/LaravelAi/Concerns/HeartbeatsIdleSseReads.php` +- Modify: `src/Services/LaravelAi/ReasoningOpenAiCompatibleGateway.php` +- Modify: `config/code-talker.php` +- Test: `tests/Feature/ReasoningOpenAiCompatibleGatewayTest.php` + +**Interfaces:** +- Consumes: `Laravel\Ai\Streaming\Events\StreamEvent` (abstract `toArray(): array`); `Laravel\Ai\Gateway\Concerns\ParsesServerSentEvents::parseServerSentEvents($streamBody): Generator`, reachable as `parent::parseServerSentEvents()` from a trait method. +- Produces: + - `Heartbeat::__construct(string $id, int $timestamp)`, `toArray(): array` returning `type => 'heartbeat'`. + - `HeartbeatsIdleSseReads::parseServerSentEvents($streamBody): Generator` yielding `array|Heartbeat`. + +**Background the implementer needs:** + +`ParsesServerSentEvents::readLine()` returns its buffer whenever a read yields `''`. If you simply add a timeout, a read that times out mid-frame hands the parser a partial line like `data: {"cho` — it starts with `data:`, so it is not skipped; `json_decode` fails silently; and the remainder arrives as a line that does *not* start with `data:` and is dropped. **The frame is lost.** The buffer must survive the idle window. That is the single most important property of this task. + +Guzzle routes `stream => true` to `StreamHandler`, so the body wraps a real socket resource and `stream_set_timeout()` applies. When it does not (`Http::fake()` bodies are resource-backed too, but a `PumpStream` or a host's custom handler is not), fall back to the parent parser rather than degrading. + +- [ ] **Step 1: Write the failing test** + +Add to `tests/Feature/ReasoningOpenAiCompatibleGatewayTest.php`. Add these imports at the top of the file: + +```php +use Jvjvjv\CodeTalker\Services\LaravelAi\Streaming\Heartbeat; +``` + +Then add the helper and tests: + +```php + /** + * A gateway exposing the protected SSE parser, so a test can drive the + * generator one step at a time. Stepping matters: the generator suspends on + * each yield, which is what lets a single-threaded test write the second + * half of a frame *after* observing the heartbeat for the gap. + */ + private function parsingGateway(): object + { + return new class($this->app->make(Dispatcher::class)) extends ReasoningOpenAiCompatibleGateway + { + public function parse($body): \Generator + { + return $this->parseServerSentEvents($body); + } + }; + } + + public function test_an_idle_gap_yields_a_heartbeat_without_losing_the_frame_that_spans_it(): void + { + config()->set('code-talker.conversations.heartbeat_seconds', 1); + + [$readEnd, $writeEnd] = stream_socket_pair(STREAM_PF_UNIX, STREAM_SOCK_STREAM, 0); + + // Half a frame, then silence — exactly the shape that used to be lost. + fwrite($writeEnd, 'data: {"choices":[{"delta":{"con'); + + $events = $this->parsingGateway()->parse(Utils::streamFor($readEnd)); + + // Runs until the first yield: the read times out and reports a beat. + $events->rewind(); + $this->assertInstanceOf(Heartbeat::class, $events->current()); + $this->assertSame('heartbeat', $events->current()->toArray()['type']); + + // The rest of the frame arrives after the gap and must parse intact. + fwrite($writeEnd, 'tent":"Hello"}}]}' . "\n\n"); + + $events->next(); + $this->assertSame( + [['delta' => ['content' => 'Hello']]], + $events->current()['choices'], + ); + + fclose($writeEnd); + $events->next(); + $this->assertFalse($events->valid()); + } + + public function test_heartbeats_are_disabled_by_a_zero_interval(): void + { + config()->set('code-talker.conversations.heartbeat_seconds', 0); + + $sse = 'data: {"choices":[{"delta":{"content":"Hi"}}]}' . "\n\n" + . 'data: [DONE]' . "\n\n"; + + $parsed = iterator_to_array($this->parsingGateway()->parse(Utils::streamFor($sse)), false); + + $this->assertCount(1, $parsed); + $this->assertSame('Hi', $parsed[0]['choices'][0]['delta']['content']); + } + + public function test_a_body_without_a_stream_resource_falls_back_to_the_parent_parser(): void + { + config()->set('code-talker.conversations.heartbeat_seconds', 1); + + // A PumpStream has no underlying resource, so detaching it would leave + // nothing to read; the parser must delegate instead. + $body = new \GuzzleHttp\Psr7\PumpStream(function (): string { + static $sent = false; + + if ($sent) { + return ''; + } + + $sent = true; + + return 'data: {"choices":[{"delta":{"content":"Hi"}}]}' . "\n\n"; + }); + + $parsed = iterator_to_array($this->parsingGateway()->parse($body), false); + + $this->assertCount(1, $parsed); + $this->assertSame('Hi', $parsed[0]['choices'][0]['delta']['content']); + } +``` + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: `vendor/bin/phpunit tests/Feature/ReasoningOpenAiCompatibleGatewayTest.php` +Expected: FAIL — `Heartbeat` class not found. + +- [ ] **Step 3: Create the Heartbeat event** + +Create `src/Services/LaravelAi/Streaming/Heartbeat.php`: + +```php + + */ + public function toArray(): array + { + return [ + 'id' => $this->id, + 'invocation_id' => $this->invocationId, + 'type' => 'heartbeat', + 'timestamp' => $this->timestamp, + ]; + } +} +``` + +- [ ] **Step 4: Create the trait** + +Create `src/Services/LaravelAi/Concerns/HeartbeatsIdleSseReads.php`: + +```php +|Heartbeat> + */ + protected function parseServerSentEvents($streamBody): Generator + { + $seconds = (int) config('code-talker.conversations.heartbeat_seconds', 5); + + // Checked before detaching, because detach() cannot be undone: a body + // with no resource behind it (a PumpStream, a host's custom handler) + // must reach the parent parser with its body intact. + if ($seconds <= 0 || ! is_string($streamBody->getMetadata('stream_type'))) { + yield from parent::parseServerSentEvents($streamBody); + + return; + } + + $resource = $streamBody->detach(); + + if (! is_resource($resource)) { + return; + } + + try { + yield from $this->readSseWithHeartbeats($resource, $seconds); + } finally { + // Nothing else holds it once detached. + fclose($resource); + } + } + + /** + * @param resource $resource + * @return Generator|Heartbeat> + */ + private function readSseWithHeartbeats($resource, int $seconds): Generator + { + stream_set_timeout($resource, $seconds); + + $buffer = ''; + $emptyReads = 0; + + while (true) { + $byte = fread($resource, 1); + + if ($byte === false) { + return; + } + + if ($byte === '') { + // Checked before feof(), because a socket can report EOF after + // a read timeout — treating that as the end would turn every + // silent gap into a truncated turn. + if (stream_get_meta_data($resource)['timed_out'] ?? false) { + $emptyReads = 0; + + yield new Heartbeat(strtolower((string) Str::uuid7()), time()); + + continue; + } + + if (feof($resource)) { + return; + } + + if (++$emptyReads >= self::MAX_EMPTY_READS) { + return; + } + + continue; + } + + $emptyReads = 0; + $buffer .= $byte; + + if ($byte !== "\n") { + continue; + } + + $line = trim($buffer); + $buffer = ''; + + if ($line === '' || ! str_starts_with($line, 'data:')) { + continue; + } + + $data = trim(substr($line, 5)); + + if ($data === '[DONE]') { + return; + } + + $decoded = json_decode($data, true); + + if (json_last_error() === JSON_ERROR_NONE && $decoded !== null) { + yield $decoded; + } + } + } +} +``` + +- [ ] **Step 5: Wire the trait into the gateway** + +In `src/Services/LaravelAi/ReasoningOpenAiCompatibleGateway.php`, add the imports: + +```php +use Jvjvjv\CodeTalker\Services\LaravelAi\Concerns\HeartbeatsIdleSseReads; +use Jvjvjv\CodeTalker\Services\LaravelAi\Streaming\Heartbeat; +``` + +Add the trait immediately inside the class declaration: + +```php +class ReasoningOpenAiCompatibleGateway extends OpenAiCompatibleGateway +{ + use HeartbeatsIdleSseReads; +``` + +And in `processTextStream()`, make the first statement inside `foreach ($this->parseServerSentEvents($streamBody) as $data)`: + +```php + // A tick, not model output: forward it and read on. Everything + // below this line assumes $data is a decoded provider frame. + if ($data instanceof Heartbeat) { + yield $data->withInvocationId($invocationId); + + continue; + } +``` + +- [ ] **Step 6: Add the config key** + +In `config/code-talker.php`, inside the `'conversations' => [` block, after `idle_timeout_minutes`: + +```php + // Seconds of provider silence before the turn emits a heartbeat. + // Two things depend on it: intermediaries stop timing out during a + // long gap, and PHP only flips connection_aborted() after a write to + // a dead socket — so without a heartbeat an abandoned turn keeps + // generating until the model's next event, which on a large context + // can be minutes. Set to 0 to disable. + 'heartbeat_seconds' => (int) env('CODE_TALKER_HEARTBEAT_SECONDS', 5), +``` + +- [ ] **Step 7: Run the tests to verify they pass** + +Run: `vendor/bin/phpunit tests/Feature/ReasoningOpenAiCompatibleGatewayTest.php` +Expected: PASS (the idle-gap test takes ~1s). + +- [ ] **Step 8: Run the full suite** + +Run: `vendor/bin/phpunit` +Expected: PASS — no existing test regresses. `Http::fake()` bodies are resource-backed but never idle, so they never produce a heartbeat. + +- [ ] **Step 9: Commit** + +```bash +git add src/Services/LaravelAi/Streaming/Heartbeat.php \ + src/Services/LaravelAi/Concerns/HeartbeatsIdleSseReads.php \ + src/Services/LaravelAi/ReasoningOpenAiCompatibleGateway.php \ + config/code-talker.php \ + tests/Feature/ReasoningOpenAiCompatibleGatewayTest.php +git commit -m "feat: emit heartbeats during idle provider reads" +``` + +--- + +## Task 2: The turn forwards heartbeats to the browser + +**Files:** +- Modify: `src/Services/ChatBot/Conversation/ConversationTurnRunner.php` +- Modify: `src/Services/ChatBot/SseFrameEncoder.php` +- Modify: `resources/js/types/code-talker.d.ts` +- Modify: `README.md` +- Test: `tests/Feature/AiPersonaConversationServiceTest.php`, `tests/Feature/ChatTurnLibraryTest.php` + +**Interfaces:** +- Consumes: `Heartbeat` from Task 1. +- Produces: the `['type' => 'heartbeat']` turn event, encoded as `": ping\n\n"`. + +**Background the implementer needs:** + +Three properties must hold together, and each has a test below: + +1. A heartbeat is **not** appended to `$events`. That array is serialized into `ai_llm_messages.response_data.events`; a turn with minute-long gaps would otherwise log hundreds of ticks. +2. A heartbeat **does not** reset `$stepStartedAt`. Resetting it on a tick would make the max-duration guard unreachable forever. +3. A heartbeat **does** reach the max-duration guard. Today the guard only runs when a provider event arrives, so a stalled stream can sit well past `max_stream_seconds` unnoticed. This is a second bug the tick fixes, and the test below pins it. + +- [ ] **Step 1: Write the failing tests** + +Add to `tests/Feature/AiPersonaConversationServiceTest.php`. Add imports: + +```php +use Jvjvjv\CodeTalker\Services\LaravelAi\Streaming\Heartbeat; +use Laravel\Ai\Gateway\StepResponse; +use Laravel\Ai\Streaming\Events\StreamEnd; +use Laravel\Ai\Streaming\Events\TextDelta; +``` + +(Some of these may already be imported; do not duplicate them.) + +```php + /** + * Install a gateway that emits heartbeats between its text deltas — a + * model that is slow to produce tokens rather than one that has stopped. + */ + private function fakeHeartbeatingGateway(int $beats): void + { + CodeTalkerAgent::fake([]); + + $gateway = new class([], $beats) extends FakeTextGateway { + public function __construct(array $responses, private int $beats) + { + parent::__construct($responses); + } + + public function generateStreamStep( + string $invocationId, + TextProvider $provider, + string $model, + ?string $instructions, + array $messages, + array $tools, + ?array $schema, + ?TextGenerationOptions $options, + ?int $timeout, + StepContext $stepContext, + ): Generator { + yield (new StreamStart(uniqid('', true), $provider->name(), $model, time())) + ->withInvocationId($invocationId); + + for ($i = 0; $i < $this->beats; $i++) { + yield (new Heartbeat(uniqid('', true), time()))->withInvocationId($invocationId); + } + + yield (new TextDelta(uniqid('', true), 'm1', 'Done', time())) + ->withInvocationId($invocationId); + + yield (new StreamEnd(uniqid('', true), 'stop', new Usage(), time())) + ->withInvocationId($invocationId); + + return new StepResponse( + 'Done', [], FinishReason::Stop, new Usage(), new Meta($provider->name(), $model), + ); + } + }; + + $manager = $this->app->make(AiManager::class); + (Closure::bind(function () use ($gateway): void { + $this->fakeAgentGateways[CodeTalkerAgent::class] = $gateway; + }, $manager, $manager::class))(); + } + + public function test_a_heartbeat_reaches_the_browser_but_never_the_stored_events(): void + { + Queue::fake(); + $this->fakeHeartbeatingGateway(beats: 3); + + $persona = $this->makePersona(); + $service = $this->app->make(AiPersonaConversationService::class); + + $conversation = $service->startConversation($persona); + $events = $this->drainAndDecode($service->continueConversation($conversation, 'Hi')); + + $this->assertSame(3, count(array_filter($events, fn ($e) => ($e['type'] ?? null) === 'heartbeat'))); + + // The stored event log is a record of what the model did, not of how + // long it took to do it. + $logged = AiLlmMessage::where('direction', 'response')->first()->response_data['events']; + $this->assertNotContains('heartbeat', array_column($logged, 'type')); + + // The answer itself is unaffected. + $this->assertSame('Done', AiConversationMessage::where('role', 'assistant')->first()->content); + } + + public function test_the_max_duration_guard_trips_on_a_heartbeat_with_no_provider_event(): void + { + Queue::fake(); + config()->set('code-talker.conversations.max_stream_seconds', 60); + $this->fakeHeartbeatingGateway(beats: 3); + + $persona = $this->makePersona(); + + // Elapsed time only goes over budget after the StreamStart, so the + // guard has nothing but heartbeats to trip on. + $service = new class( + $this->app->make(AgentFactory::class), + $this->app->make(AiMemoryService::class), + $this->app->make(ConversationUsageService::class), + $this->app->make(RawExchangeContext::class), + $this->app->make(AiSystemProviderConfigurator::class), + ) extends AiPersonaConversationService { + private int $calls = 0; + + protected function streamElapsedSeconds(float $startedAt): float + { + return ++$this->calls > 1 ? 9999.0 : 0.0; + } + }; + + $conversation = $service->startConversation($persona); + $events = $this->drainAndDecode($service->continueConversation($conversation, 'Hi')); + + $error = collect($events)->firstWhere('type', 'error'); + $this->assertNotNull($error); + $this->assertSame('max_stream_duration', $error['reason']); + } +``` + +Add to `tests/Feature/ChatTurnLibraryTest.php`: + +```php + public function test_a_heartbeat_is_encoded_as_a_comment_frame(): void + { + $frames = iterator_to_array((new SseFrameEncoder())->encode([ + ['type' => 'heartbeat'], + ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi']], + ]), false); + + // A comment frame: every SSE consumer ignores it, including the + // published client, which only reads lines beginning with "data:". + $this->assertSame(": ping\n\n", $frames[0]); + $this->assertStringStartsWith('data: {', $frames[1]); + + // A heartbeat is not an error, so the turn still terminates normally. + $this->assertSame("data: [DONE]\n\n", $frames[2]); + } +``` + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: `vendor/bin/phpunit tests/Feature/AiPersonaConversationServiceTest.php tests/Feature/ChatTurnLibraryTest.php` +Expected: FAIL — no `heartbeat` events are produced and the encoder emits a `data:` frame. + +- [ ] **Step 3: Handle the heartbeat in the runner** + +In `src/Services/ChatBot/Conversation/ConversationTurnRunner.php`, add the import: + +```php +use Jvjvjv\CodeTalker\Services\LaravelAi\Streaming\Heartbeat; +``` + +Inside `foreach ($agent->stream($prompt) as $event) {`, directly after the `$guards->clientAborted()` block, insert: + +```php + // A tick, not model output. It is deliberately not logged + // and not appended to $events — a turn with minute-long + // gaps would otherwise fill response_data with hundreds of + // them — and it deliberately does not reset the step + // clock, which would put the duration guard out of reach. + $isHeartbeat = $event instanceof Heartbeat; +``` + +Then guard the logging and accumulation that follow: + +```php + if (! $isHeartbeat) { + Log::debug('Chat bot API stream event', [ + // ...unchanged... + ]); + + $events[] = $event; + } +``` + +Leave the `ToolResultEvent` / `StreamStart` / `ErrorEvent` / `ToolCallEvent` branches as they are — a `Heartbeat` matches none of them. After the max-duration guard block (so a tick can trip it), insert: + +```php + if ($isHeartbeat) { + yield ['type' => 'heartbeat']; + + continue; + } +``` + +- [ ] **Step 4: Handle the heartbeat in the encoder** + +In `src/Services/ChatBot/SseFrameEncoder.php`, as the first statement of the `foreach` body: + +```php + // A comment frame, not a data frame: it exists to put a byte on + // the wire during a silent gap, and every SSE consumer ignores it + // without being taught to. + if (($event['type'] ?? null) === 'heartbeat') { + yield ": ping\n\n"; + + continue; + } +``` + +- [ ] **Step 5: Run the tests to verify they pass** + +Run: `vendor/bin/phpunit tests/Feature/AiPersonaConversationServiceTest.php tests/Feature/ChatTurnLibraryTest.php` +Expected: PASS + +- [ ] **Step 6: Document the event** + +In `README.md`, add a row to the turn-events table, after `message_stop`: + +```markdown +| `heartbeat` | — (encoded as an SSE comment, not a data frame) | +``` + +And after the `stop_reason` paragraph added in 0.15.0, add: + +```markdown +`heartbeat` fires while the provider is silent. `SseFrameEncoder` renders it as +`: ping` — an SSE comment — so browsers and the published client ignore it +without any handling. It is there so something reaches the socket during a long +gap: intermediaries stop timing out mid-answer, and PHP only flips +`connection_aborted()` after a write to a dead connection, so without it an +abandoned turn keeps generating until the model's next event. Set +`conversations.heartbeat_seconds` to `0` to disable. Detection costs two beats: +the first write marks the socket dead, the second observes it. +``` + +In `resources/js/types/code-talker.d.ts`, above the `ChatStreamEvent` union, add: + +```typescript +/** + * `heartbeat` is deliberately absent from this union. The server yields it as + * a turn event, but `SseFrameEncoder` writes it as an SSE comment (`: ping`), + * which never arrives as a message — so a wire consumer cannot receive one and + * should not be made to handle it. A host consuming the events directly, + * without the SSE encoding, will see `{ type: 'heartbeat' }`. + */ +``` + +- [ ] **Step 7: Verify the whole suite and the types** + +Run: `vendor/bin/phpunit && npm run typecheck` +Expected: PASS + +- [ ] **Step 8: Commit** + +```bash +git add src/Services/ChatBot/Conversation/ConversationTurnRunner.php \ + src/Services/ChatBot/SseFrameEncoder.php \ + resources/js/types/code-talker.d.ts README.md \ + tests/Feature/AiPersonaConversationServiceTest.php tests/Feature/ChatTurnLibraryTest.php +git commit -m "feat: forward stream heartbeats to the browser as comment frames" +``` + +--- + +## Task 3: Turn run schema, status enum, and models + +**Files:** +- Create: `database/migrations/2026_08_31_000001_create_ai_turn_runs_table.php` +- Create: `database/migrations/2026_08_31_000002_create_ai_turn_events_table.php` +- Create: `src/Enums/AiTurnRunStatus.php` +- Create: `src/Models/AiTurnRun.php` +- Create: `src/Models/AiTurnEvent.php` +- Test: `tests/Feature/AiTurnRunModelTest.php` + +**Interfaces:** +- Produces: + - `AiTurnRunStatus` cases `Queued`, `Running`, `Completed`, `Failed`, `Cancelled`, `Abandoned`; method `isTerminal(): bool`. + - `AiTurnRun` with `$fillable` = `ai_conversation_id, public_id, status, prompt, last_polled_at, cancel_requested_at, started_at, finished_at, error_message`; casts `status` to the enum and the four timestamps to `datetime`; `conversation(): BelongsTo`, `events(): HasMany`; a `booted()` hook assigning `public_id` as a ULID. + - `AiTurnEvent` with `$fillable` = `ai_turn_run_id, sequence, payload, created_at`; `payload` cast to `array`; `$timestamps = false`. + +- [ ] **Step 1: Write the failing test** + +Create `tests/Feature/AiTurnRunModelTest.php`: + +```php +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + } + + private function conversation(): AiConversation + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + return AiConversation::create([ + 'ai_system_id' => $system->id, + 'feature' => 'persona:test', + ]); + } + + public function test_a_run_gets_a_public_id_and_casts_its_status(): void + { + $run = AiTurnRun::create([ + 'ai_conversation_id' => $this->conversation()->id, + 'status' => AiTurnRunStatus::Queued, + 'prompt' => 'Hello', + ]); + + $this->assertNotEmpty($run->public_id); + $this->assertSame(AiTurnRunStatus::Queued, $run->fresh()->status); + $this->assertFalse($run->status->isTerminal()); + } + + public function test_terminal_statuses_are_the_ones_a_reader_stops_on(): void + { + $this->assertTrue(AiTurnRunStatus::Completed->isTerminal()); + $this->assertTrue(AiTurnRunStatus::Failed->isTerminal()); + $this->assertTrue(AiTurnRunStatus::Cancelled->isTerminal()); + $this->assertTrue(AiTurnRunStatus::Abandoned->isTerminal()); + $this->assertFalse(AiTurnRunStatus::Queued->isTerminal()); + $this->assertFalse(AiTurnRunStatus::Running->isTerminal()); + } + + public function test_events_belong_to_a_run_and_keep_their_payload_shape(): void + { + $run = AiTurnRun::create([ + 'ai_conversation_id' => $this->conversation()->id, + 'status' => AiTurnRunStatus::Running, + 'prompt' => 'Hello', + ]); + + AiTurnEvent::create([ + 'ai_turn_run_id' => $run->id, + 'sequence' => 1, + 'payload' => ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi']], + ]); + + $event = $run->events()->first(); + + $this->assertSame(1, $event->sequence); + $this->assertSame('Hi', $event->payload['delta']['text']); + } + + public function test_a_run_cannot_reuse_a_sequence(): void + { + $run = AiTurnRun::create([ + 'ai_conversation_id' => $this->conversation()->id, + 'status' => AiTurnRunStatus::Running, + 'prompt' => 'Hello', + ]); + + AiTurnEvent::create(['ai_turn_run_id' => $run->id, 'sequence' => 1, 'payload' => ['type' => 'a']]); + + $this->expectException(\Illuminate\Database\QueryException::class); + + AiTurnEvent::create(['ai_turn_run_id' => $run->id, 'sequence' => 1, 'payload' => ['type' => 'b']]); + } +} +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `vendor/bin/phpunit tests/Feature/AiTurnRunModelTest.php` +Expected: FAIL — `AiTurnRunStatus` not found. + +- [ ] **Step 3: Create the status enum** + +Create `src/Enums/AiTurnRunStatus.php`: + +```php + false, + default => true, + }; + } +} +``` + +- [ ] **Step 4: Create the migrations** + +Create `database/migrations/2026_08_31_000001_create_ai_turn_runs_table.php`: + +```php +id(); + $table->string('public_id', 40)->unique(); + $table->foreignId('ai_conversation_id')->index(); + $table->string('status', 20)->index(); + $table->text('prompt'); + // The abandonment signal: connection_aborted() reports 0 in a + // worker, so "nobody is reading this" is the only usable stand-in + // for the browser having gone away. + $table->timestamp('last_polled_at')->nullable(); + $table->timestamp('cancel_requested_at')->nullable(); + $table->timestamp('started_at')->nullable(); + $table->timestamp('finished_at')->nullable(); + $table->text('error_message')->nullable(); + $table->timestamps(); + }); + } + + public function down(): void + { + Schema::dropIfExists('ai_turn_runs'); + } +}; +``` + +Create `database/migrations/2026_08_31_000002_create_ai_turn_events_table.php`: + +```php +id(); + $table->foreignId('ai_turn_run_id')->index(); + $table->unsignedInteger('sequence'); + $table->json('payload'); + $table->timestamp('created_at')->nullable(); + + // The reader asks for everything after a sequence it already + // holds, so a duplicate would silently replay or skip output. + $table->unique(['ai_turn_run_id', 'sequence']); + }); + } + + public function down(): void + { + Schema::dropIfExists('ai_turn_events'); + } +}; +``` + +- [ ] **Step 5: Create the models** + +Create `src/Models/AiTurnRun.php`: + +```php +public_id)) { + $run->public_id = (string) Str::ulid(); + } + }); + } + + protected function casts(): array + { + return [ + 'status' => AiTurnRunStatus::class, + 'last_polled_at' => 'datetime', + 'cancel_requested_at' => 'datetime', + 'started_at' => 'datetime', + 'finished_at' => 'datetime', + ]; + } + + public function conversation(): BelongsTo + { + return $this->belongsTo(AiConversation::class, 'ai_conversation_id'); + } + + public function events(): HasMany + { + return $this->hasMany(AiTurnEvent::class, 'ai_turn_run_id')->orderBy('sequence'); + } +} +``` + +Create `src/Models/AiTurnEvent.php`: + +```php +created_at === null) { + $event->created_at = Carbon::now(); + } + }); + } + + protected $fillable = [ + 'ai_turn_run_id', + 'sequence', + 'payload', + 'created_at', + ]; + + protected function casts(): array + { + return [ + 'payload' => 'array', + 'created_at' => 'datetime', + ]; + } + + public function run(): BelongsTo + { + return $this->belongsTo(AiTurnRun::class, 'ai_turn_run_id'); + } +} +``` + +- [ ] **Step 6: Run the test to verify it passes** + +Run: `vendor/bin/phpunit tests/Feature/AiTurnRunModelTest.php` +Expected: PASS + +- [ ] **Step 7: Commit** + +```bash +git add database/migrations/2026_08_31_000001_create_ai_turn_runs_table.php \ + database/migrations/2026_08_31_000002_create_ai_turn_events_table.php \ + src/Enums/AiTurnRunStatus.php src/Models/AiTurnRun.php src/Models/AiTurnEvent.php \ + tests/Feature/AiTurnRunModelTest.php +git commit -m "feat: add turn run and turn event models" +``` + +--- + +## Task 4: TurnRunStore + +**Files:** +- Create: `src/Services/Conversation/TurnRunStore.php` +- Modify: `config/code-talker.php` +- Test: `tests/Feature/TurnRunStoreTest.php` + +**Interfaces:** +- Consumes: `AiTurnRun`, `AiTurnEvent`, `AiTurnRunStatus` from Task 3. +- Produces: + ```php + open(AiConversation $conversation, string $message): AiTurnRun + markRunning(AiTurnRun $run): void + append(AiTurnRun $run, array $event): int // the assigned sequence + finish(AiTurnRun $run, AiTurnRunStatus $status, ?string $error = null): void + eventsAfter(AiTurnRun $run, int $sequence, int $limit = 200): Collection + touchPoll(AiTurnRun $run): void + requestCancel(AiTurnRun $run): void + shouldStop(AiTurnRun $run): bool + stopStatusFor(AiTurnRun $run): AiTurnRunStatus + ``` + +**Background the implementer needs:** + +`shouldStop()` is consulted on **every stream event**, so it must not put a query in the token loop. It re-reads at most every two seconds and returns a cached answer in between. The store instance is per-job, so the cache is per-run and needs no keying. + +Abandonment is measured from `last_polled_at`, or from `created_at` while that is still null — a run dispatched a moment ago has no poller yet and must not be killed before its reader connects. + +- [ ] **Step 1: Write the failing test** + +Create `tests/Feature/TurnRunStoreTest.php`: + +```php +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + } + + private function conversation(): AiConversation + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + return AiConversation::create([ + 'ai_system_id' => $system->id, + 'feature' => 'persona:test', + ]); + } + + private function store(): TurnRunStore + { + return $this->app->make(TurnRunStore::class); + } + + public function test_appended_events_are_sequenced_from_one(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $this->assertSame(1, $store->append($run, ['type' => 'message_start'])); + $this->assertSame(2, $store->append($run, ['type' => 'content_block_delta'])); + + $this->assertSame( + ['message_start', 'content_block_delta'], + $store->eventsAfter($run, 0)->pluck('payload.type')->all(), + ); + } + + public function test_events_after_a_sequence_returns_only_the_tail(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $store->append($run, ['type' => 'a']); + $store->append($run, ['type' => 'b']); + $store->append($run, ['type' => 'c']); + + $this->assertSame(['b', 'c'], $store->eventsAfter($run, 1)->pluck('payload.type')->all()); + $this->assertTrue($store->eventsAfter($run, 3)->isEmpty()); + } + + public function test_a_run_nobody_polls_is_stopped_once_the_grace_period_lapses(): void + { + config()->set('code-talker.turns.abandon_after_seconds', 30); + + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + // Freshly opened and never polled: the reader has not connected yet. + $this->assertFalse($store->shouldStop($run)); + + // Still never polled, but now well past the grace period. + Carbon::setTestNow(now()->addSeconds(31)); + $this->assertTrue($store->shouldStop($run)); + $this->assertSame(AiTurnRunStatus::Abandoned, $store->stopStatusFor($run)); + + Carbon::setTestNow(); + } + + public function test_polling_keeps_a_run_alive(): void + { + config()->set('code-talker.turns.abandon_after_seconds', 30); + + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + Carbon::setTestNow(now()->addSeconds(29)); + $store->touchPoll($run); + + Carbon::setTestNow(now()->addSeconds(20)); + $this->assertFalse($store->shouldStop($run)); + + Carbon::setTestNow(); + } + + public function test_an_explicit_cancel_stops_the_run(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $store->requestCancel($run); + + $this->assertTrue($store->shouldStop($run)); + $this->assertSame(AiTurnRunStatus::Cancelled, $store->stopStatusFor($run)); + } + + public function test_should_stop_is_throttled_so_it_never_queries_per_token(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $store->shouldStop($run); + + $queries = 0; + \Illuminate\Support\Facades\DB::listen(function () use (&$queries): void { + $queries++; + }); + + for ($i = 0; $i < 50; $i++) { + $store->shouldStop($run); + } + + $this->assertSame(0, $queries); + } + + public function test_finishing_records_the_status_and_the_error(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $store->finish($run, AiTurnRunStatus::Failed, 'provider exploded'); + + $run->refresh(); + $this->assertSame(AiTurnRunStatus::Failed, $run->status); + $this->assertSame('provider exploded', $run->error_message); + $this->assertNotNull($run->finished_at); + } +} +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `vendor/bin/phpunit tests/Feature/TurnRunStoreTest.php` +Expected: FAIL — `TurnRunStore` not found. + +- [ ] **Step 3: Create the store** + +Create `src/Services/Conversation/TurnRunStore.php`: + +```php + $conversation->id, + 'status' => AiTurnRunStatus::Queued, + 'prompt' => $message, + ]); + } + + public function markRunning(AiTurnRun $run): void + { + $run->forceFill([ + 'status' => AiTurnRunStatus::Running, + 'started_at' => now(), + ])->save(); + + $this->sequence = (int) $run->events()->max('sequence'); + } + + /** + * @param array $event + */ + public function append(AiTurnRun $run, array $event): int + { + $sequence = ++$this->sequence; + + AiTurnEvent::create([ + 'ai_turn_run_id' => $run->id, + 'sequence' => $sequence, + 'payload' => $event, + ]); + + return $sequence; + } + + public function finish(AiTurnRun $run, AiTurnRunStatus $status, ?string $error = null): void + { + $run->forceFill([ + 'status' => $status, + 'finished_at' => now(), + 'error_message' => $error, + ])->save(); + } + + /** + * @return Collection + */ + public function eventsAfter(AiTurnRun $run, int $sequence, int $limit = 200): Collection + { + return AiTurnEvent::query() + ->where('ai_turn_run_id', $run->id) + ->where('sequence', '>', $sequence) + ->orderBy('sequence') + ->limit($limit) + ->get(); + } + + public function touchPoll(AiTurnRun $run): void + { + AiTurnRun::query()->whereKey($run->id)->update(['last_polled_at' => now()]); + } + + public function requestCancel(AiTurnRun $run): void + { + AiTurnRun::query()->whereKey($run->id)->update(['cancel_requested_at' => now()]); + + $this->cachedShouldStop = null; + } + + /** + * Whether the turn should stop generating: someone cancelled it, or nobody + * is reading it any more. + */ + public function shouldStop(AiTurnRun $run): bool + { + $now = microtime(true); + + if ($this->cachedShouldStop !== null && $now - $this->shouldStopCheckedAt < self::STOP_CHECK_INTERVAL) { + return $this->cachedShouldStop; + } + + $this->shouldStopCheckedAt = $now; + + return $this->cachedShouldStop = $this->readShouldStop($run); + } + + /** + * Which terminal status a stopped run earned. + */ + public function stopStatusFor(AiTurnRun $run): AiTurnRunStatus + { + return $run->fresh()?->cancel_requested_at !== null + ? AiTurnRunStatus::Cancelled + : AiTurnRunStatus::Abandoned; + } + + private function readShouldStop(AiTurnRun $run): bool + { + $fresh = $run->fresh(); + + if ($fresh === null || $fresh->cancel_requested_at !== null) { + return true; + } + + $seconds = (int) config('code-talker.turns.abandon_after_seconds', 30); + + if ($seconds <= 0) { + return false; + } + + // Measured from created_at while nothing has polled yet: a run + // dispatched a moment ago has no reader by definition, and killing it + // before its reader connects would make the feature unusable. + $since = $fresh->last_polled_at ?? $fresh->created_at; + + return $since !== null && $since->diffInSeconds(now()) > $seconds; + } +} +``` + +- [ ] **Step 4: Add the config block** + +In `config/code-talker.php`, before the closing `];`, add: + +```php + /* + |-------------------------------------------------------------------------- + | Detached Turns + |-------------------------------------------------------------------------- + | + | A turn dispatched with AiPersonaConversationService::dispatchTurn() runs + | as a queued job and writes its events to ai_turn_events, so a browser + | reload resumes the turn instead of killing it. connection_aborted() is + | meaningless in a worker, so "nobody has polled for abandon_after_seconds" + | is what stops a turn nobody is waiting for. + | + */ + + 'turns' => [ + 'queue' => env('CODE_TALKER_TURN_QUEUE'), + 'abandon_after_seconds' => (int) env('CODE_TALKER_TURN_ABANDON_SECONDS', 30), + 'poll_interval_ms' => (int) env('CODE_TALKER_TURN_POLL_MS', 250), + 'max_stream_seconds' => (int) env('CODE_TALKER_TURN_MAX_STREAM_SECONDS', 900), + 'retention_days' => (int) env('CODE_TALKER_TURN_RETENTION_DAYS', 7), + ], +``` + +- [ ] **Step 5: Run the test to verify it passes** + +Run: `vendor/bin/phpunit tests/Feature/TurnRunStoreTest.php` +Expected: PASS + +- [ ] **Step 6: Commit** + +```bash +git add src/Services/Conversation/TurnRunStore.php config/code-talker.php tests/Feature/TurnRunStoreTest.php +git commit -m "feat: add TurnRunStore for detached turn events" +``` + +--- + +## Task 5: RunConversationTurnJob + +**Files:** +- Create: `src/Jobs/RunConversationTurnJob.php` +- Test: `tests/Feature/RunConversationTurnJobTest.php` + +**Interfaces:** +- Consumes: `TurnRunStore` (Task 4); `AiPersonaConversationService::usingCancellationCheck(callable): static` and `continueConversation(AiConversation, string): Generator` (both already exist). +- Produces: `RunConversationTurnJob::__construct(int $turnRunId)`; `handle(AiPersonaConversationService $chat, TurnRunStore $store): void`; `failed(?Throwable $e): void`. + +**Background the implementer needs:** + +The job constructor takes an **id**, not a model, so a serialized payload stays small and never carries a stale copy of the run. + +The job calls the existing `continueConversation()`. There is deliberately no second turn implementation — everything the synchronous path does (system prompt, history, tool loop, `TurnRecorder`, memory job) happens here unchanged, and the 0.15.0 fixes mean a stopped run still persists its partial answer flagged incomplete. + +- [ ] **Step 1: Write the failing test** + +Create `tests/Feature/RunConversationTurnJobTest.php`: + +```php +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + } + + private function persona(): AiPersona + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + return AiPersona::create([ + 'ai_system_id' => $system->id, + 'name' => 'Test Bot', + 'slug' => 'test-bot', + 'prompt_template' => 'You are {{persona_name}}.', + 'is_active' => true, + ]); + } + + public function test_the_job_records_every_event_and_completes_the_run(): void + { + Queue::fake(); + CodeTalkerAgent::fake(['Hello there']); + + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($this->persona()); + + $run = $this->app->make(TurnRunStore::class)->open($conversation, 'Hi'); + + $this->app->make(RunConversationTurnJob::class, ['turnRunId' => $run->id]) + ->handle($service, $this->app->make(TurnRunStore::class)); + + $run->refresh(); + $this->assertSame(AiTurnRunStatus::Completed, $run->status); + $this->assertNotNull($run->finished_at); + + $types = $run->events()->pluck('payload.type')->all(); + $this->assertContains('content_block_delta', $types); + $this->assertContains('message_stop', $types); + + // Sequences are contiguous from 1, which is what a resuming reader + // relies on to know it missed nothing. + $this->assertSame(range(1, $run->events()->count()), $run->events()->pluck('sequence')->all()); + + // The turn itself behaved exactly as the synchronous path does. + $this->assertSame('Hello there', AiConversationMessage::where('role', 'assistant')->first()->content); + } + + public function test_a_cancelled_run_stops_and_is_marked_cancelled(): void + { + Queue::fake(); + CodeTalkerAgent::fake(['This answer is cancelled part way through']); + + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($this->persona()); + + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($conversation, 'Hi'); + $store->requestCancel($run); + + $this->app->make(RunConversationTurnJob::class, ['turnRunId' => $run->id]) + ->handle($service, $this->app->make(TurnRunStore::class)); + + $this->assertSame(AiTurnRunStatus::Cancelled, $run->fresh()->status); + + // 0.15.0's recorder keeps whatever the turn produced, flagged. + $message = AiConversationMessage::where('role', 'assistant')->first(); + $this->assertNotNull($message); + $this->assertTrue($message->metadata['incomplete']); + } + + public function test_a_failed_job_marks_the_run_failed_so_a_reader_stops_waiting(): void + { + Queue::fake(); + + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($this->persona()); + $run = $this->app->make(TurnRunStore::class)->open($conversation, 'Hi'); + + $this->app->make(RunConversationTurnJob::class, ['turnRunId' => $run->id]) + ->failed(new \RuntimeException('worker died')); + + $run->refresh(); + $this->assertSame(AiTurnRunStatus::Failed, $run->status); + $this->assertSame('worker died', $run->error_message); + } +} +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `vendor/bin/phpunit tests/Feature/RunConversationTurnJobTest.php` +Expected: FAIL — `RunConversationTurnJob` not found. + +- [ ] **Step 3: Create the job** + +Create `src/Jobs/RunConversationTurnJob.php`: + +```php +onQueue(config('code-talker.turns.queue') ?: null); + } + + public function handle(AiPersonaConversationService $chat, TurnRunStore $store): void + { + $run = AiTurnRun::find($this->turnRunId); + + if ($run === null || $run->status->isTerminal()) { + return; + } + + $store->markRunning($run); + + try { + $events = $chat + ->usingCancellationCheck(fn (): bool => $store->shouldStop($run)) + ->continueConversation($run->conversation, $run->prompt); + + foreach ($events as $event) { + $store->append($run, $event); + } + } catch (Throwable $exception) { + $store->finish($run, AiTurnRunStatus::Failed, $exception->getMessage()); + + throw $exception; + } + + $store->finish( + $run, + $store->shouldStop($run) ? $store->stopStatusFor($run) : AiTurnRunStatus::Completed, + ); + } + + /** + * A worker that dies leaves a reader polling forever unless the run is + * closed out here. + */ + public function failed(?Throwable $exception): void + { + $run = AiTurnRun::find($this->turnRunId); + + if ($run === null || $run->status->isTerminal()) { + return; + } + + app(TurnRunStore::class)->finish( + $run, + AiTurnRunStatus::Failed, + $exception?->getMessage(), + ); + } +} +``` + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `vendor/bin/phpunit tests/Feature/RunConversationTurnJobTest.php` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/Jobs/RunConversationTurnJob.php tests/Feature/RunConversationTurnJobTest.php +git commit -m "feat: run a conversation turn as a queued job" +``` + +--- + +## Task 6: TurnEventStream reader and resumable SSE framing + +**Files:** +- Create: `src/Services/Conversation/TurnEventStream.php` +- Modify: `src/Services/ChatBot/SseFrameEncoder.php` +- Test: `tests/Feature/TurnEventStreamTest.php` + +**Interfaces:** +- Consumes: `TurnRunStore` (Task 4), `AiTurnRun`/`AiTurnRunStatus` (Task 3). +- Produces: `TurnEventStream::__construct(TurnRunStore $store)`; `stream(AiTurnRun $run, int $after = 0): Generator` yielding event arrays each carrying a `_seq` int. +- `SseFrameEncoder::encode()` emits `id: <_seq>\n` before the data line and strips `_seq` from the payload. + +**Background the implementer needs:** + +The **drain-after-terminal** ordering is the subtle part and has its own test. The job appends its last event and *then* marks the run finished. A reader that checks status before reading events can therefore see "finished" while the final event is still unread, and drop it. The order must be: read events → if empty, re-read status → if terminal, read events **once more** → only then stop. + +`_seq` is encoder-only metadata. It is not part of the documented event vocabulary and must never reach the browser inside the JSON payload. + +- [ ] **Step 1: Write the failing test** + +Create `tests/Feature/TurnEventStreamTest.php`: + +```php +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + + config()->set('code-talker.turns.poll_interval_ms', 1); + } + + private function conversation(): AiConversation + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + return AiConversation::create([ + 'ai_system_id' => $system->id, + 'feature' => 'persona:test', + ]); + } + + public function test_a_finished_run_replays_from_the_beginning(): void + { + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($this->conversation(), 'Hi'); + $store->markRunning($run); + $store->append($run, ['type' => 'message_start']); + $store->append($run, ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi']]); + $store->finish($run, AiTurnRunStatus::Completed); + + $events = iterator_to_array($this->app->make(TurnEventStream::class)->stream($run, 0), false); + + $this->assertSame(['message_start', 'content_block_delta'], array_column($events, 'type')); + $this->assertSame([1, 2], array_column($events, '_seq')); + } + + public function test_a_reload_resumes_from_the_last_sequence_it_saw(): void + { + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($this->conversation(), 'Hi'); + $store->markRunning($run); + $store->append($run, ['type' => 'a']); + $store->append($run, ['type' => 'b']); + $store->append($run, ['type' => 'c']); + $store->finish($run, AiTurnRunStatus::Completed); + + $events = iterator_to_array($this->app->make(TurnEventStream::class)->stream($run, 1), false); + + $this->assertSame(['b', 'c'], array_column($events, 'type')); + } + + public function test_the_final_event_survives_a_run_finishing_mid_poll(): void + { + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($this->conversation(), 'Hi'); + $store->markRunning($run); + $store->append($run, ['type' => 'first']); + + $events = $this->app->make(TurnEventStream::class)->stream($run, 0); + + $events->rewind(); + $this->assertSame('first', $events->current()['type']); + + // The job's last act: append, then mark finished. A reader that read + // status before events would drop 'last' entirely. + $store->append($run, ['type' => 'last']); + $store->finish($run, AiTurnRunStatus::Completed); + + $events->next(); + $this->assertSame('last', $events->current()['type']); + + $events->next(); + $this->assertFalse($events->valid()); + } + + public function test_reading_marks_the_run_as_polled_so_it_is_not_abandoned(): void + { + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($this->conversation(), 'Hi'); + $store->markRunning($run); + $store->append($run, ['type' => 'a']); + $store->finish($run, AiTurnRunStatus::Completed); + + iterator_to_array($this->app->make(TurnEventStream::class)->stream($run, 0), false); + + $this->assertNotNull($run->fresh()->last_polled_at); + } + + public function test_sequences_become_sse_ids_and_never_leak_into_the_payload(): void + { + $frames = iterator_to_array((new SseFrameEncoder())->encode([ + ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi'], '_seq' => 7], + ]), false); + + $this->assertSame("id: 7\ndata: " . json_encode([ + 'type' => 'content_block_delta', + 'delta' => ['text' => 'Hi'], + ]) . "\n\n", $frames[0]); + } +} +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `vendor/bin/phpunit tests/Feature/TurnEventStreamTest.php` +Expected: FAIL — `TurnEventStream` not found. + +- [ ] **Step 3: Create the reader** + +Create `src/Services/Conversation/TurnEventStream.php`: + +```php +> + */ + public function stream(AiTurnRun $run, int $after = 0): Generator + { + $pollMicroseconds = max(1, (int) config('code-talker.turns.poll_interval_ms', 250)) * 1000; + $heartbeatSeconds = (int) config('code-talker.conversations.heartbeat_seconds', 5); + $maxSeconds = (int) config('code-talker.turns.max_stream_seconds', 900); + + $startedAt = microtime(true); + $lastEmittedAt = $startedAt; + + while (true) { + $this->store->touchPoll($run); + + $events = $this->store->eventsAfter($run, $after); + + if ($events->isNotEmpty()) { + foreach ($events as $event) { + /** @var AiTurnEvent $event */ + $after = $event->sequence; + $lastEmittedAt = microtime(true); + + yield $event->payload + ['_seq' => $event->sequence]; + } + + continue; + } + + // Nothing new. Read the status only now, and drain once more before + // stopping: the job appends its last event and *then* marks the run + // finished, so checking status first would drop that event. + if ($run->fresh()?->status->isTerminal() ?? true) { + foreach ($this->store->eventsAfter($run, $after) as $event) { + /** @var AiTurnEvent $event */ + $after = $event->sequence; + + yield $event->payload + ['_seq' => $event->sequence]; + } + + return; + } + + if ($maxSeconds > 0 && microtime(true) - $startedAt > $maxSeconds) { + yield [ + 'type' => 'error', + 'message' => "The turn exceeded the maximum stream duration of {$maxSeconds}s.", + 'reason' => 'max_stream_duration', + ]; + + return; + } + + if ($heartbeatSeconds > 0 && microtime(true) - $lastEmittedAt >= $heartbeatSeconds) { + $lastEmittedAt = microtime(true); + + // Provider-agnostic, unlike the gateway's own heartbeat: this + // one fires for every provider, because it is measured against + // the store rather than a socket. + yield ['type' => 'heartbeat']; + } + + usleep($pollMicroseconds); + } + } +} +``` + +- [ ] **Step 4: Emit sequences as SSE ids** + +In `src/Services/ChatBot/SseFrameEncoder.php`, replace the `yield 'data: ' . json_encode($event) . "\n\n";` line with: + +```php + // `_seq` is framing metadata, not part of the event vocabulary: it + // becomes the SSE id a reconnecting consumer resumes from, and + // never reaches the browser inside the payload. + $sequence = $event['_seq'] ?? null; + unset($event['_seq']); + + yield ($sequence === null ? '' : 'id: ' . $sequence . "\n") + . 'data: ' . json_encode($event) . "\n\n"; +``` + +- [ ] **Step 5: Run the test to verify it passes** + +Run: `vendor/bin/phpunit tests/Feature/TurnEventStreamTest.php` +Expected: PASS + +- [ ] **Step 6: Run the full suite** + +Run: `vendor/bin/phpunit` +Expected: PASS — the encoder change is inert for events without `_seq`. + +- [ ] **Step 7: Commit** + +```bash +git add src/Services/Conversation/TurnEventStream.php src/Services/ChatBot/SseFrameEncoder.php \ + tests/Feature/TurnEventStreamTest.php +git commit -m "feat: add resumable turn event reader" +``` + +--- + +## Task 7: Service entry points + +**Files:** +- Modify: `src/Services/AiPersonaConversationService.php` +- Test: `tests/Feature/AiPersonaConversationServiceTest.php` + +**Interfaces:** +- Consumes: `TurnRunStore` (Task 4), `TurnEventStream` (Task 6), `RunConversationTurnJob` (Task 5). +- Produces: + ```php + dispatchTurn(AiConversation $conversation, string $message): AiTurnRun + resumeTurn(AiTurnRun $run, int $after = 0): Generator + cancelTurn(AiTurnRun $run): void + ``` + +**Background the implementer needs:** + +The constructor signature is load-bearing — `AiPersonaConversationServiceTest` builds anonymous subclasses with exactly five positional arguments. Resolve `TurnRunStore` and `TurnEventStream` **inside the method bodies** via `app()`, the way the constructor already builds its other collaborators from the five it is given. Do not add constructor parameters and do not add properties that need constructing. + +- [ ] **Step 1: Write the failing test** + +Add to `tests/Feature/AiPersonaConversationServiceTest.php`. Add imports: + +```php +use Jvjvjv\CodeTalker\Enums\AiTurnRunStatus; +use Jvjvjv\CodeTalker\Jobs\RunConversationTurnJob; +use Jvjvjv\CodeTalker\Models\AiTurnRun; +use Jvjvjv\CodeTalker\Services\Conversation\TurnRunStore; +``` + +```php + public function test_dispatching_a_turn_queues_a_job_against_a_new_run(): void + { + Queue::fake(); + + $persona = $this->makePersona(); + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($persona); + + $run = $service->dispatchTurn($conversation, 'Hi there'); + + $this->assertSame(AiTurnRunStatus::Queued, $run->status); + $this->assertSame('Hi there', $run->prompt); + $this->assertNotEmpty($run->public_id); + + Queue::assertPushed( + RunConversationTurnJob::class, + fn (RunConversationTurnJob $job): bool => $job->turnRunId === $run->id, + ); + } + + public function test_resuming_a_turn_streams_its_stored_events(): void + { + Queue::fake(); + config()->set('code-talker.turns.poll_interval_ms', 1); + + $persona = $this->makePersona(); + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($persona); + + $run = $service->dispatchTurn($conversation, 'Hi'); + + $store = $this->app->make(TurnRunStore::class); + $store->markRunning($run); + $store->append($run, ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi']]); + $store->finish($run, AiTurnRunStatus::Completed); + + $events = iterator_to_array($service->resumeTurn($run), false); + + $this->assertSame(['content_block_delta'], array_column($events, 'type')); + } + + public function test_cancelling_a_turn_marks_it_for_the_worker(): void + { + Queue::fake(); + + $persona = $this->makePersona(); + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($persona); + + $run = $service->dispatchTurn($conversation, 'Hi'); + $service->cancelTurn($run); + + $this->assertNotNull($run->fresh()->cancel_requested_at); + } +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `vendor/bin/phpunit tests/Feature/AiPersonaConversationServiceTest.php` +Expected: FAIL — `dispatchTurn()` not defined. + +- [ ] **Step 3: Add the methods** + +In `src/Services/AiPersonaConversationService.php`, add the imports: + +```php +use Jvjvjv\CodeTalker\Jobs\RunConversationTurnJob; +use Jvjvjv\CodeTalker\Models\AiTurnRun; +use Jvjvjv\CodeTalker\Services\Conversation\TurnEventStream; +use Jvjvjv\CodeTalker\Services\Conversation\TurnRunStore; +``` + +Add these methods after `continueConversation()`: + +```php + /** + * Run a turn detached from the caller's connection. + * + * The turn becomes a queued job that writes its events to a store; the + * browser reads them with resumeTurn() and can reconnect at any point. Use + * this instead of continueConversation() when a turn is long enough that a + * reload or a flaky connection should not destroy it. + * + * The store and reader are resolved here rather than injected: this + * service's five-argument constructor is depended on by host apps and by + * tests that subclass it, so collaborators are built from what it has. + */ + public function dispatchTurn(AiConversation $conversation, string $message): AiTurnRun + { + $run = app(TurnRunStore::class)->open($conversation, $message); + + RunConversationTurnJob::dispatch($run->id); + + return $run; + } + + /** + * Stream a dispatched turn's events, starting after the given sequence. + * + * Yields the same structured events continueConversation() does, each with + * a `_seq` the encoder turns into an SSE id. A browser that reconnects + * passes back the last sequence it saw and misses nothing in between. + * + * @return Generator> + */ + public function resumeTurn(AiTurnRun $run, int $after = 0): Generator + { + yield from app(TurnEventStream::class)->stream($run, $after); + } + + /** + * Ask a running turn to stop. + * + * The worker notices within a couple of seconds and stops generating; + * whatever the turn produced by then is persisted and flagged incomplete. + */ + public function cancelTurn(AiTurnRun $run): void + { + app(TurnRunStore::class)->requestCancel($run); + } +``` + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `vendor/bin/phpunit tests/Feature/AiPersonaConversationServiceTest.php` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/Services/AiPersonaConversationService.php tests/Feature/AiPersonaConversationServiceTest.php +git commit -m "feat: add dispatchTurn, resumeTurn and cancelTurn" +``` + +--- + +## Task 8: Prune command and schedule + +**Files:** +- Create: `src/Console/Commands/PruneTurnEventsCommand.php` +- Modify: `src/CodeTalkerServiceProvider.php` +- Test: `tests/Feature/PruneTurnEventsCommandTest.php` + +**Interfaces:** +- Consumes: `AiTurnRun`, `AiTurnEvent`, `AiTurnRunStatus` (Task 3). +- Produces: the `ai:prune-turn-events` console command. + +- [ ] **Step 1: Write the failing test** + +Create `tests/Feature/PruneTurnEventsCommandTest.php`: + +```php +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + } + + private function run(AiTurnRunStatus $status, int $daysOld): AiTurnRun + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + $conversation = AiConversation::create([ + 'ai_system_id' => $system->id, + 'feature' => 'persona:test', + ]); + + $run = AiTurnRun::create([ + 'ai_conversation_id' => $conversation->id, + 'status' => $status, + 'prompt' => 'Hi', + ]); + + $run->forceFill(['created_at' => now()->subDays($daysOld)])->save(); + + AiTurnEvent::create(['ai_turn_run_id' => $run->id, 'sequence' => 1, 'payload' => ['type' => 'a']]); + + return $run; + } + + public function test_it_removes_old_terminal_runs_and_their_events(): void + { + config()->set('code-talker.turns.retention_days', 7); + + $old = $this->run(AiTurnRunStatus::Completed, daysOld: 10); + $recent = $this->run(AiTurnRunStatus::Completed, daysOld: 1); + $live = $this->run(AiTurnRunStatus::Running, daysOld: 10); + + $this->artisan('ai:prune-turn-events')->assertExitCode(0); + + $this->assertNull(AiTurnRun::find($old->id)); + $this->assertSame(0, AiTurnEvent::where('ai_turn_run_id', $old->id)->count()); + + $this->assertNotNull(AiTurnRun::find($recent->id)); + + // A long-running turn is not garbage, however old the row is. + $this->assertNotNull(AiTurnRun::find($live->id)); + } +} +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `vendor/bin/phpunit tests/Feature/PruneTurnEventsCommandTest.php` +Expected: FAIL — the command is not registered. + +- [ ] **Step 3: Create the command** + +Create `src/Console/Commands/PruneTurnEventsCommand.php`: + +```php +info('Turn event retention is disabled; nothing pruned.'); + + return self::SUCCESS; + } + + $terminal = array_values(array_map( + static fn (AiTurnRunStatus $status): string => $status->value, + array_filter(AiTurnRunStatus::cases(), static fn (AiTurnRunStatus $s): bool => $s->isTerminal()), + )); + + // Only finished runs: a turn still generating is not garbage, however + // long it has been going. + $runIds = AiTurnRun::query() + ->whereIn('status', $terminal) + ->where('created_at', '<', now()->subDays($days)) + ->pluck('id'); + + if ($runIds->isEmpty()) { + $this->info('No turn runs past retention.'); + + return self::SUCCESS; + } + + AiTurnEvent::query()->whereIn('ai_turn_run_id', $runIds)->delete(); + AiTurnRun::query()->whereIn('id', $runIds)->delete(); + + $this->info("Pruned {$runIds->count()} turn run(s)."); + + return self::SUCCESS; + } +} +``` + +- [ ] **Step 4: Register and schedule it** + +In `src/CodeTalkerServiceProvider.php`, add the import: + +```php +use Jvjvjv\CodeTalker\Console\Commands\PruneTurnEventsCommand; +``` + +Add `PruneTurnEventsCommand::class` to the `$this->commands([...])` array alongside `PruneProviderExchangesCommand::class`, and inside the `if (config('code-talker.schedule', true))` block: + +```php + Schedule::command('ai:prune-turn-events') + ->dailyAt('03:15') + ->withoutOverlapping(); +``` + +- [ ] **Step 5: Run the test to verify it passes** + +Run: `vendor/bin/phpunit tests/Feature/PruneTurnEventsCommandTest.php` +Expected: PASS + +- [ ] **Step 6: Commit** + +```bash +git add src/Console/Commands/PruneTurnEventsCommand.php src/CodeTalkerServiceProvider.php \ + tests/Feature/PruneTurnEventsCommandTest.php +git commit -m "feat: add ai:prune-turn-events command" +``` + +--- + +## Task 9: Client resumption and release documentation + +**Files:** +- Modify: `resources/js/code-talker-stream.ts` +- Modify: `resources/js/types/code-talker.d.ts` +- Modify: `README.md` +- Modify: `CHANGELOG.md` +- Test: `tests/Feature/FrontendAssetPublishingTest.php` (must keep passing unchanged) + +**Interfaces:** +- Consumes: the `id:` framing from Task 6. +- Produces: a `lastEventId` the caller can pass back when reconnecting. + +**Background the implementer needs:** + +`FrontendAssetPublishingTest` enforces that `code-talker-stream.ts` imports nothing but browser APIs and relative paths. Do not add a dependency. + +The client parses frames by splitting on `\n\n` and reading lines that begin with `data:`. Comment frames (`: ping`) and `id:` lines are already ignored — the change is to *record* the id, not to start handling new frame types. + +- [ ] **Step 1: Track the sequence in the client** + +In `resources/js/code-talker-stream.ts`, in the frame dispatch function, before the existing `data:` filtering, read the id: + +```typescript + const idLine = frame.split('\n').find((line) => line.startsWith('id:')); + + if (idLine !== undefined) { + const parsed = Number.parseInt(idLine.slice(3).trim(), 10); + + if (!Number.isNaN(parsed)) { + // Recorded so a caller reconnecting after a dropped connection can + // resume from here instead of replaying the whole turn. + callbacks.onSequence?.(parsed); + } + } +``` + +Add `onSequence?: (sequence: number) => void;` to the callbacks interface in the same file and to the declarations in `resources/js/types/code-talker.d.ts`, documented as: + +```typescript + /** + * The sequence of the last event received, present only for a turn + * dispatched with `dispatchTurn()`. Pass it back as `after` when + * reconnecting so the turn resumes rather than replays. + */ + onSequence?: (sequence: number) => void; +``` + +- [ ] **Step 2: Verify the types and the publishing constraints** + +Run: `npm run typecheck && vendor/bin/phpunit tests/Feature/FrontendAssetPublishingTest.php` +Expected: PASS + +- [ ] **Step 3: Document the durable path in the README** + +In `README.md`, after the "Interrupted turns" section added in 0.15.0, add: + +````markdown +### Running a turn as a job + +`continueConversation()` ties the turn to the caller's connection: close the +tab and the turn stops, reload and it is gone. For turns long enough that this +matters, dispatch the turn instead and stream it from its store. + +```php +// Start it. Returns an AiTurnRun; `public_id` is the handle to put in a URL. +$run = $chat->dispatchTurn($conversation, $request->string('message')); + +// Stream it — from the start, or from wherever the browser left off. +foreach ($encoder->encode($chat->resumeTurn($run, $after)) as $frame) { + echo $frame; + ob_get_level() > 0 && ob_flush(); + flush(); +} + +// Stop it early. +$chat->cancelTurn($run); +``` + +Each event is framed with an SSE `id:` carrying its sequence. A browser that +reconnects passes the last sequence it saw back as `after`, and the turn +resumes rather than replaying. The published client reports it via +`onSequence`. + +A dispatched turn needs a queue worker. Because `connection_aborted()` reports +0 in a worker, a run stops when nobody has read it for +`turns.abandon_after_seconds` (default 30) — so closing the tab still stops +generation, and a reload inside that window reattaches to the same run. +`ai:prune-turn-events` clears finished runs past `turns.retention_days`. +```` + +Add the `turns.*` keys to the configuration section alongside `raw_exchanges.*`. + +- [ ] **Step 4: Extend the 0.15.0 changelog entry** + +In `CHANGELOG.md`, add to the existing `## [0.15.0]` entry's **New Features** list: + +```markdown +- Turns now emit a heartbeat while the provider is silent (`conversations.heartbeat_seconds`, default 5, `0` to disable). It is encoded as an SSE comment, so existing clients ignore it; it keeps intermediaries from timing out mid-answer and lets an abandoned turn be noticed in seconds rather than minutes. Currently emitted for `openai-compatible` and `lm-studio` systems; a turn dispatched as a job heartbeats for every provider. +- Added `AiPersonaConversationService::dispatchTurn()`, `resumeTurn()` and `cancelTurn()`: a turn can run as a queued job that records its events, so a browser reload resumes it instead of killing it. Events are framed with an SSE `id:` carrying their sequence, and the published client reports it through a new `onSequence` callback. Requires the two new migrations and a queue worker. +- Added `ai:prune-turn-events` (scheduled daily at 03:15) to clear finished turn runs past `turns.retention_days`. +``` + +And to **Bug Fixes**: + +```markdown +- The maximum-stream-duration guard now applies during provider silence. It was only evaluated when an event arrived, so a stalled stream could run well past `conversations.max_stream_seconds` before anything noticed. +``` + +Add to **Breaking Changes**: + +```markdown +- Two new migrations (`ai_turn_runs`, `ai_turn_events`) ship with this release. Re-publish migrations and run them, even if you do not use the dispatched-turn API — `ai:prune-turn-events` is scheduled by default and expects the tables. +``` + +Remove the **Known Issues** bullet about abandoned connections only being noticed on the next write; it no longer describes the release. + +- [ ] **Step 5: Verify everything** + +Run: `vendor/bin/phpunit && npm run typecheck` +Expected: PASS + +- [ ] **Step 6: Commit** + +```bash +git add resources/js/code-talker-stream.ts resources/js/types/code-talker.d.ts README.md CHANGELOG.md +git commit -m "docs: document heartbeats and dispatched turns for 0.15.0" +``` + +--- + +## Self-Review Notes + +Checked against the spec: + +- **Part A components** — `Heartbeat` (Task 1), `HeartbeatsIdleSseReads` with the partial-line buffer and the fallback (Task 1), gateway wiring (Task 1), runner tick handling (Task 2), encoder comment frame (Task 2), config key (Task 1). ✓ +- **Part A consequences** — the two-beat detection cost and the openai-compatible-only coverage are documented in the README (Task 2) and the CHANGELOG (Task 9); the duration-guard improvement has a test (Task 2). ✓ +- **Part B data model** — both migrations, the enum, both models, and the no-`last_sequence` decision (Task 3). ✓ +- **Part B components** — `TurnRunStore` with throttled `shouldStop()` (Task 4), the job with `failed()` (Task 5), `TurnEventStream` with the drain-after-terminal ordering (Task 6), the three service methods (Task 7), the prune command (Task 8). ✓ +- **Contract** — `id:` framing (Task 6), the `heartbeat`-absent-from-the-union note (Task 2), client `onSequence` (Task 9), README and CHANGELOG (Tasks 2 and 9). ✓ +- **Error handling table** — non-resource body (Task 1 test), split frame (Task 1 test), dead worker (Task 5 test), unpolled run (Task 4 test), reader ceiling (Task 6 implementation). The "pruned run" row needs no code: `stream()` treats a missing run as terminal via `$run->fresh()?->status->isTerminal() ?? true`. +- **Type consistency** — `shouldStop()`/`stopStatusFor()` are named identically in Tasks 4, 5 and 6; `_seq` is produced in Task 6 and consumed by the encoder in the same task; `turnRunId` is the job's property name in Tasks 5 and 7. diff --git a/docs/superpowers/specs/2026-08-31-durable-turns-and-heartbeats-design.md b/docs/superpowers/specs/2026-08-31-durable-turns-and-heartbeats-design.md new file mode 100644 index 0000000..dea3123 --- /dev/null +++ b/docs/superpowers/specs/2026-08-31-durable-turns-and-heartbeats-design.md @@ -0,0 +1,352 @@ +# Design: stream heartbeats and durable turns + +**Date:** 2026-08-31 +**Status:** Approved + +## Purpose + +A streamed chat turn currently lives and dies with the browser's HTTP +connection, and the server only notices the browser is gone when it next writes +to the socket. With a large-context local model, gaps between provider events +run 100–500 seconds, so a turn abandoned at t=14s keeps generating until t=104s +and is then discarded. + +Two fixes already landed on this branch (release 0.15.0) and are assumed here: + +- A turn cut short is persisted with whatever it produced, flagged + `metadata.incomplete` with an `incomplete_reason`. +- A cut-short turn reports `stop_reason: incomplete` and logs + `AiInteractionStatus::Aborted`, never `success`. + +Those stop the data loss. They do not stop the waste, and they do not make a +turn survive a page reload. This design covers the two remaining fixes: + +- **Part A — heartbeats.** Write to the socket during silent gaps, so a dead + connection is noticed in seconds rather than minutes and intermediaries stop + timing out mid-gap. +- **Part B — durable turns.** Run a turn as a queued job that appends its + events to a store the browser reads from, so a reload resumes the turn + instead of killing it. + +## Part A — heartbeats during silent gaps + +### The constraint + +The blocking read is `Laravel\Ai\Gateway\Concerns\ParsesServerSentEvents::readLine()`, +a byte-at-a-time blocking read on the PSR-7 response body. While it blocks, +`ConversationTurnRunner`'s `foreach` over `$agent->stream()` is suspended, so a +heartbeat cannot be yielded from the runner, the service, or the host's +controller. The override point has to be inside the read. + +Three facts establish feasibility, each verified against the vendored code: + +1. Guzzle routes a request with `stream => true` to `StreamHandler`, not the + cURL handler (`Proxy::wrapStreaming`), so the response body wraps a **real + PHP socket resource** and `stream_set_timeout()` applies. +2. `TextGenerationLoop::stream()` forwards every yielded event verbatim + (`foreach ($stream as $event) { yield $event; }`), so a package-defined + `StreamEvent` subclass reaches the runner untouched. +3. `readLine()` returns its buffer when a read yields `''`. A naive timeout + therefore hands `parseServerSentEvents()` a **partial** line — `data: {"cho` — + which fails `json_decode` silently, and the remainder arrives as a line not + starting with `data:` and is dropped. **The frame is lost.** Any timeout + implementation must carry a partial-line buffer across idle windows. + +### Components + +**`Services/LaravelAi/Streaming/Heartbeat`** — a `StreamEvent` subclass whose +`toArray()` reports `type: 'heartbeat'`. It exists only to travel through +laravel/ai's loop; it never reaches the transcript or the logs. + +**`Services/LaravelAi/Concerns/HeartbeatsIdleSseReads`** — overrides +`parseServerSentEvents()`: + +- Reads `conversations.heartbeat_seconds`. When `<= 0`, delegates to the parent + implementation and returns — the feature is off and behaviour is unchanged. +- Confirms the body is resource-backed by checking `getMetadata('stream_type')` + is a non-empty string **before** detaching. When it is not (`Http::fake()`, + a `PumpStream`, a host's custom handler), delegates to the parent. Checking + first matters: `detach()` is not reversible, so a failed detach would leave + no body to fall back to. +- Otherwise `detach()`es the resource, applies `stream_set_timeout($resource, + $seconds)`, and runs its own byte loop with a `$buffer` that survives + timeouts. On `fread() === ''` it consults + `stream_get_meta_data($resource)['timed_out']`: true yields a `Heartbeat` + and continues reading into the same buffer; false plus `feof()` ends the + stream. A bounded consecutive-empty-read counter guards against a wrapper + that reports neither. +- `fclose()`s the detached resource in a `finally`. Nothing else holds it once + detached, so the trait owns closing it. + +Cost is unchanged: one syscall per byte, exactly as the parent already does. +`stream_set_timeout` is chosen over `stream_select` for that reason — a select +per byte would double the syscall count. + +The generator's yielded type widens from `array` to `array|Heartbeat`, which is +type-safe and cannot collide with a provider payload the way a sentinel array +key could. + +**`ReasoningOpenAiCompatibleGateway`** uses the trait, and `processTextStream()` +gains one branch: a `Heartbeat` is re-yielded with the invocation id and the +loop continues. + +**`ConversationTurnRunner`** treats a heartbeat as a tick, not an event: + +- It is **not** appended to `$events` and not logged — it would flood + `ai_llm_messages.response_data.events`. +- It does **not** reset `$stepStartedAt`. +- The max-duration guard **is** evaluated on a heartbeat iteration, then the + runner yields `['type' => 'heartbeat']` and continues. + +That last point fixes a second latent bug: today the duration guard only runs +when a provider event arrives, so a stalled stream can sit well past +`max_stream_seconds` unnoticed. Heartbeats make the guard real. + +**`SseFrameEncoder`** encodes a `heartbeat` event as `": ping\n\n"` — an SSE +comment frame — and continues without touching its terminal-state tracking. +`EventSource` ignores comment frames, and the published client filters on lines +starting with `data:` (`code-talker-stream.ts`), so both ignore it for free. + +### Consequences to expect + +- **Abort detection takes two heartbeats**, roughly 10s at the default. PHP + only flips `connection_aborted()` once a write to the dead socket has been + attempted: the first heartbeat is that write, the second observes the result. + This is still two orders of magnitude better than the observed 100–500s. +- **Coverage is `openai-compatible` and `lm-studio` only.** That is the one + gateway this package overrides; Anthropic, OpenAI and Gemini use laravel/ai's + own. Copying more vendor code to cover them is explicitly rejected — Part B's + reader-side heartbeat is provider-agnostic and closes the gap properly. + +## Part B — durable turns + +A turn should not die because a tab closed. The job runs it; a store holds its +events; the browser reads from the store at whatever sequence it left off. + +Chosen transport: **DB-backed, host-polled.** No new infrastructure beyond the +queue and database a host already runs. Broadcasting was rejected because +replay-after-reload still requires a stored backlog, making it the DB store +*plus* a Reverb/Pusher/Redis dependency rather than an alternative to it. A +cache-backed list was rejected because eviction silently loses a turn and +file/array drivers do not share state between the worker and web processes. + +The durable path is **additive**. `continueConversation()` is unchanged and +still works synchronously; the job calls it, so there is one turn +implementation rather than two. + +### Data model + +**`ai_turn_runs`** + +| Column | Notes | +| --- | --- | +| `id` | | +| `public_id` | ULID, unique — the handle a host puts in a URL, following `ai_conversations.public_id` | +| `ai_conversation_id` | FK, indexed | +| `status` | string(20), cast to `AiTurnRunStatus` | +| `prompt` | text — the user message the job replays | +| `last_polled_at` | nullable timestamp — the abandonment signal | +| `cancel_requested_at` | nullable timestamp — an explicit cancel | +| `started_at`, `finished_at` | nullable timestamps | +| `error_message` | nullable text | +| timestamps | | + +**`ai_turn_events`** + +| Column | Notes | +| --- | --- | +| `id` | | +| `ai_turn_run_id` | FK, indexed | +| `sequence` | unsigned int; unique with the run id | +| `payload` | json — one event array as `continueConversation()` yields it | +| `created_at` | | + +**`AiTurnRunStatus`**: `Queued`, `Running`, `Completed`, `Failed`, `Cancelled`, +`Abandoned`, with `isTerminal()`. + +There is deliberately **no `last_sequence` column** on the run. The reader asks +for events after a sequence it already holds and terminates on status plus an +empty read; a per-event counter update would be a second write per event +buying nothing. + +### Components + +**`Services/Conversation/TurnRunStore`** — the only writer, owned by the job: + +``` +open(AiConversation, string $message): AiTurnRun +markRunning(AiTurnRun): void +append(AiTurnRun, array $event): int // returns the assigned sequence +finish(AiTurnRun, AiTurnRunStatus, ?string $error = null): void +eventsAfter(AiTurnRun, int $sequence, int $limit = 200): Collection +touchPoll(AiTurnRun): void +requestCancel(AiTurnRun): void +shouldStop(AiTurnRun): bool +``` + +The sequence counter lives in memory on the store instance, seeded at `open()`, +because the job is the sole writer for the life of a run. + +`shouldStop()` is the cancellation signal and is **throttled**: it re-queries at +most every two seconds and returns its cached answer in between. It is +consulted on every stream event, so an unthrottled query would put a database +round-trip in the token loop. + +**`Jobs/RunConversationTurnJob`** — constructed with the run id (not the model, +so a queued payload stays small). `handle()` marks the run running, binds +`usingCancellationCheck(fn () => $store->shouldStop($run))`, appends every +yielded event, and finishes the run with the status the stop reason implies: +`Cancelled` when a cancel was requested, `Abandoned` when nobody polled, +`Completed` otherwise. `failed()` marks the run `Failed` with the exception +message, so a worker that dies does not leave a reader waiting forever. + +**`Services/Conversation/TurnEventStream`** — the reader generator a host's +endpoint consumes. Each pass: touch `last_polled_at`, read events after the +cursor, yield each with its sequence attached, advance the cursor. When a read +comes back empty it re-reads the run's status, and **if terminal it drains once +more before breaking** — the job appends an event and then marks the run +terminal, so reading status before events would drop the final event. While the +run is live and quiet it yields `['type' => 'heartbeat']` on the same +`heartbeat_seconds` cadence, which is what makes Part A's benefit +provider-agnostic on this path. A `turns.max_stream_seconds` ceiling bounds the +generator so an endpoint cannot hang forever. + +**`AiPersonaConversationService`** gains three methods, none of which change the +existing five-argument constructor (`AiPersonaConversationServiceTest` builds +anonymous subclasses against that exact signature): + +``` +dispatchTurn(AiConversation, string $message): AiTurnRun +resumeTurn(AiTurnRun, int $after = 0): Generator +cancelTurn(AiTurnRun): void +``` + +**`SseFrameEncoder`** emits `id: \n` before the data line for any +event carrying a `_seq` key, and strips `_seq` from the JSON payload. `_seq` is +encoder-only metadata, never part of the documented event shape. This gives an +`EventSource` consumer automatic `Last-Event-ID` resumption; the published +fetch-based client reads it manually. + +**`Console/Commands/PruneTurnEventsCommand`** (`ai:prune-turn-events`) deletes +terminal runs older than `turns.retention_days`, cascading to their events. +Scheduled daily at 03:15, alongside `ai:prune-provider-exchanges`. + +### Cancellation semantics + +`connection_aborted()` is meaningless in a worker — it reports 0 — so a +detached turn needs a different signal. A run is stopped when either: + +- `cancel_requested_at` is set (an explicit host-driven cancel), or +- nothing has polled it for `turns.abandon_after_seconds` (default 30), + measured from `last_polled_at`, or from `created_at` while that is still null + so a run gets a grace period before its first reader connects. + +Closing the tab therefore stops the GPU within ~30s, and a reload *inside* that +window reattaches to the same run rather than starting a new one. That resume +property is the whole point of Part B. + +### Host wiring + +The package still ships no routes. The documented shape is two endpoints: + +```php +// Start a turn. +$run = $chat->dispatchTurn($conversation, $request->string('message')); +return ['run' => $run->public_id]; + +// Stream it, from the beginning or from wherever the browser left off. +return response()->stream(function () use ($chat, $run, $after, $encoder) { + foreach ($encoder->encode($chat->resumeTurn($run, $after)) as $frame) { + echo $frame; + ob_get_level() > 0 && ob_flush(); + flush(); + } +}, headers: [...]); +``` + +A reload calls the second endpoint again with the last sequence it saw. + +## Configuration + +```php +'conversations' => [ + // ... + 'heartbeat_seconds' => (int) env('CODE_TALKER_HEARTBEAT_SECONDS', 5), +], + +'turns' => [ + 'queue' => env('CODE_TALKER_TURN_QUEUE'), // null = default + 'abandon_after_seconds' => (int) env('CODE_TALKER_TURN_ABANDON_SECONDS', 30), + 'poll_interval_ms' => (int) env('CODE_TALKER_TURN_POLL_MS', 250), + 'max_stream_seconds' => (int) env('CODE_TALKER_TURN_MAX_STREAM_SECONDS', 900), + 'retention_days' => (int) env('CODE_TALKER_TURN_RETENTION_DAYS', 7), +], +``` + +Every nested read uses an inline default, because Laravel skips +`mergeConfigFrom` entirely when a host has cached config — a host that +published `code-talker.php` before these keys existed would otherwise resolve +`null` in production only. + +## Frontend contract + +- **README** — the turn-events table gains `heartbeat`, with a note that + `SseFrameEncoder` renders it as a `: ping` comment frame; a new section + documents running a turn as a job and resuming it. +- **`code-talker.d.ts`** — deliberately does **not** add `heartbeat` to the + `ChatStreamEvent` union. That union describes what arrives over the wire, and + a comment frame never does; declaring it would force consumers to handle a + case they cannot receive. A comment in the declarations records why, so the + omission does not read as an oversight later. +- **`code-talker-stream.ts`** — unchanged for heartbeats (it already filters on + `data:`); gains sequence tracking so a caller can resume. + +## Error handling + +| Failure | Behaviour | +| --- | --- | +| Body is not resource-backed | Trait delegates to the parent parser; no heartbeats, no regression | +| Stream times out mid-frame | Partial line held in the buffer; the frame completes on the next read | +| Worker dies mid-run | `failed()` marks the run `Failed`; the reader sees a terminal status and stops | +| Nobody polls a live run | Run stops within `abandon_after_seconds`; partial output persisted by the 0.15.0 recorder | +| Reader outlives `max_stream_seconds` | Generator yields a terminal `error` frame and returns | +| Reader asks for a pruned run | Treated as terminal; the host renders the stored transcript instead | + +## Testing + +**Part A** +- A socket pair with a deliberate idle gap: heartbeats are yielded during the + gap, and a frame split across the gap arrives intact (the regression the + partial-line buffer exists to prevent). +- A non-resource-backed body falls through to the parent parser unchanged. +- `heartbeat_seconds = 0` disables the override entirely. +- The runner yields a `heartbeat` frame, records no heartbeat in + `ai_llm_messages`, and does not reset the step clock. +- The encoder renders `: ping\n\n` and does not treat it as terminal. +- The max-duration guard trips on a heartbeat with no provider event. + +**Part B** +- `dispatchTurn()` creates a queued run and dispatches the job. +- The job appends every event the service yields, in order, and finishes the + run `Completed`. +- A reader replays from sequence 0 and from mid-sequence, and a second reader + starting at `after: N` sees exactly the tail — the reload case. +- A reader that stops polling abandons the run; the turn stops and its partial + output is persisted and flagged incomplete. +- The final event is never dropped when the job finishes between the reader's + event read and its status read. +- An explicit cancel stops the run and marks it `Cancelled`. +- `failed()` marks the run `Failed` and the reader terminates. +- `ai:prune-turn-events` removes terminal runs past retention and their events, + and leaves live runs alone. + +## Out of scope + +- A broadcasting transport. The store is the prerequisite for one; adding it + later needs no schema change. +- Multi-viewer fan-out. Two readers on one run both work, but `last_polled_at` + makes them indistinguishable, so neither can be cancelled independently. +- Resuming a turn whose worker died mid-generation. The run is marked failed + and the browser sees an error; regenerating is the host's call. +- Extending gateway-level heartbeats to Anthropic/OpenAI/Gemini. Part B's + reader-side heartbeat covers those paths without copying vendor code. diff --git a/openspec/changes/durable-turns-and-heartbeats/deferred-findings.md b/openspec/changes/durable-turns-and-heartbeats/deferred-findings.md new file mode 100644 index 0000000..75606de --- /dev/null +++ b/openspec/changes/durable-turns-and-heartbeats/deferred-findings.md @@ -0,0 +1,26 @@ +# Deferred findings + +Minor findings raised during review of this change and deliberately not fixed. +The whole-branch review triaged every one of them as safe to defer; they are +recorded here so they are not lost with the review workspace. + +- (task 1) TeeingStream::record() has only indirect coverage via RawExchangeChatIntegrationTest; a direct unit test would be ~10 lines. +- (task 1) the processTextStream() Heartbeat-forwarding branch is untested; reordering the loop body so the error sniff runs first would throw on a Heartbeat (no ArrayAccess). +- (task 1) heartbeats are not yet visible end-to-end (that is Task 2's runner hop). +- (task 2) drainAndDecode() maps ANY ':'-prefixed line to a heartbeat; a future second comment-frame type would be miscounted. ': ping' would be tighter. +- (task 2) the new d.ts JSDoc block floats unattached between PageReloadEvent and the union's own docblock. Stylistic; typecheck passes. +- (task 3) no 'sequence' => 'integer' cast on AiTurnEvent; under PDO::ATTR_EMULATE_PREPARES it returns a string. Siblings do cast their ints. +- (task 3) ai_turn_events.created_at is nullable with an app-level default, where the sibling ai_conversation_messages uses ->useCurrent(). A raw DB::table insert would leave NULL. +- (task 3) neither foreignId is constrained(), so deleting a run leaves orphan events at the DB level. Task 8's prune command already deletes events by hand, consistent with the package's existing doctrine. +- (task 4) touchPoll()/requestCancel() use query-builder update(), so the caller's in-memory $run keeps pre-update attributes. Internally harmless — shouldStop()/stopStatusFor() always fresh(). +- (task 4) stopStatusFor() returns Abandoned for a deleted run and for a healthy run called without a prior shouldStop() === true. Consistent with shouldStop() and carried by the docblock contract. +- (task 4) an externally written cancel is seen by the job's store only on its next re-read, so cancel latency is bounded at the throttle interval rather than zero. +- (task 5) the success-path finish() is unguarded, so in the outrun-retry_after degraded mode a run failed by failed() can be flipped back to Completed by the still-streaming first worker. Status flaps; no sequence corruption. +- (task 5) the post-loop shouldStop() re-read can mislabel a stop-vs-complete race in two narrow windows (abandoned-then-polled reads as Completed; completed-then-threshold-crossed reads as Abandoned). Label only — events and the recorded message are intact either way. +- (task 6) a run deleted mid-stream ends as a cleanly finished turn with [DONE] and no error frame. Matches the design doc's error-handling table; revisit if pruning ever races live readers. +- (task 6) $event->payload + ['_seq' => ...] is an array union, so a stored payload containing a '_seq' key would override the real sequence. Writer-controlled today; theoretical. +- (task 6) the stub's eventsAfter ignores its $sequence argument, so the new tests cannot catch a drain that fails to advance the cursor it passes to the store. Real-store termination argument still holds. +- (task 7) resumeTurn() is lazy — app(TurnEventStream::class) and the first touchPoll() do not happen until the caller advances the generator. +- (task 7) sync queue driver makes the returned $run stale (in-memory Queued, DB terminal). +- (task 7) test file import ordering broke the file's grouping. Cosmetic. +- (task 8) the two deletes are not wrapped in a transaction unlike the package's other hand-cascades. Partial failure self-heals because events are deleted first, so leftover runs are simply re-selected next sweep. diff --git a/openspec/changes/durable-turns-and-heartbeats/design.md b/openspec/changes/durable-turns-and-heartbeats/design.md new file mode 100644 index 0000000..4a716e3 --- /dev/null +++ b/openspec/changes/durable-turns-and-heartbeats/design.md @@ -0,0 +1,102 @@ +# Design + +The full working design, with the vendored-code analysis behind it, is +`docs/superpowers/specs/2026-08-31-durable-turns-and-heartbeats-design.md`; the +implementation plan is +`docs/superpowers/plans/2026-08-31-durable-turns-and-heartbeats.md`. What +follows is the record of the decisions those documents made and why. + +## Why the heartbeat has to live inside the SSE read + +`ParsesServerSentEvents::readLine()` blocks on a byte-at-a-time read of the +response body. While it blocks, `ConversationTurnRunner`'s `foreach` over +`$agent->stream()` is suspended inside it, and so is everything upstream — the +service, and the host's controller loop. A heartbeat therefore cannot be +yielded from any of them. The read is the only seam. + +Three facts were verified against the vendored code before committing to this: + +1. Guzzle routes `stream => true` to `StreamHandler`, not the cURL handler + (`Proxy::wrapStreaming`), so the body wraps a real socket resource and + `stream_set_timeout()` applies. +2. `TextGenerationLoop::stream()` forwards every yielded event verbatim, so a + package-defined `StreamEvent` subclass reaches the runner untouched. +3. `readLine()` returns its buffer on an empty read. A naive timeout would + therefore hand the parser a partial line (`data: {"cho`) that starts with + `data:`, fails `json_decode` silently, and leaves its remainder to be + dropped as a line without the prefix. **The frame would be lost.** The + partial-line buffer that survives an idle window is the load-bearing detail + of the whole trait. + +`stream_set_timeout` was chosen over `stream_select` because the parent already +reads a byte at a time; a `select` per byte would double the syscall count for +no benefit. The timeout flag is checked *before* `feof()`, because a socket can +report EOF after a read timeout — treating that as the end would turn every +silent gap into a truncated turn. + +A body with no resource behind it (a `PumpStream`, a host's custom handler) +delegates to the parent parser. That check happens **before** `detach()`, +because detaching cannot be undone: a failed detach would leave no body to fall +back to. + +## Why detection costs two heartbeats + +PHP only flips `connection_aborted()` after a write to a dead socket has been +attempted. The first heartbeat is that write; the second observes the result. +At the default interval that is roughly ten seconds — against the 100–500 +seconds observed in the field, which is the number this change exists to fix. + +## Why the durable transport is the database + +The alternatives were weighed and rejected: + +- **Broadcasting** (Reverb/Pusher/Redis) still needs a stored backlog to + replay after a reload, which makes it the database store *plus* an + infrastructure dependency this package does not currently have — not an + alternative to the store. The store is its prerequisite, and adding a + broadcast layer later needs no schema change. +- **A cache-backed list** needs no migration, but eviction silently loses a + turn mid-generation, and file/array drivers do not share state between the + queue worker and the web process at all. + +The database costs a poll query per active viewer at `poll_interval_ms`. That +is the price of having no new infrastructure, and it is paid only by hosts who +opt into the detached path. + +## Why abandonment is a poll timestamp + +`connection_aborted()` reports 0 in a worker, so a detached turn has no signal +that its reader left. Reading is therefore what keeps a run alive: every pass +of `TurnEventStream` stamps `last_polled_at`, and a run nobody stamps for +`turns.abandon_after_seconds` stops. + +The timestamp is measured from `created_at` while `last_polled_at` is still +null. A run dispatched a moment ago has no reader by definition, and killing it +before its first reader connects would make the feature unusable. + +`shouldStop()` is consulted on every stream event, so it re-reads at most every +two seconds and returns a cached answer in between — an unthrottled check would +put a database round-trip inside the token loop. + +## Why the reader drains after seeing a terminal status + +The job appends its last event and *then* marks the run finished. A reader that +checks status before reading events can see "finished" while the final event is +still unread, and drop it — silently, on every turn. The order is therefore: +read events, and only if empty re-read the status, and if terminal read events +once more before stopping. + +## Why `heartbeat` is absent from the TypeScript event union + +`ChatStreamEvent` describes what arrives over the wire. `SseFrameEncoder` +writes a heartbeat as an SSE comment, which never arrives as a message, so a +wire consumer cannot receive one and should not be made to handle it. A host +consuming the structured events directly, without the SSE encoding, does see +`{ type: 'heartbeat' }` — the README documents it there. A comment in the +declarations records the omission so it does not read as an oversight. + +## Why there is no `last_sequence` column + +The reader asks for events after a sequence it already holds and stops on +status plus an empty read. A per-event counter update on the run would be a +second write per event buying nothing. diff --git a/openspec/changes/durable-turns-and-heartbeats/proposal.md b/openspec/changes/durable-turns-and-heartbeats/proposal.md new file mode 100644 index 0000000..6a0a106 --- /dev/null +++ b/openspec/changes/durable-turns-and-heartbeats/proposal.md @@ -0,0 +1,33 @@ +## Why + +A streamed turn lives and dies with the browser's HTTP connection, and the server only learns the browser is gone when it next writes to the socket. With a large-context local model, gaps between provider events run 100–500 seconds, so a turn abandoned at t=14s keeps generating until t=104s — burning GPU the whole time — and is then thrown away. + +Two fixes already in this release stop the *data* loss: an interrupted turn is now persisted with whatever it produced and flagged incomplete, and it is logged as `aborted` rather than as a clean success. Neither stops the waste, and neither makes a turn survive a reload. This change addresses both. + +The blocking read is `laravel/ai`'s `ParsesServerSentEvents::readLine()`, a byte-at-a-time blocking read on the response body. While it blocks, the whole turn is suspended inside it — nothing in this package, and nothing in a host's controller, is running to emit anything. That constraint decides the shape of both halves of this change. + +## What Changes + +- Adds a heartbeat during provider silence. A `HeartbeatsIdleSseReads` trait bounds the SSE read with `stream_set_timeout()` and emits a `Heartbeat` stream event on each idle window; `ConversationTurnRunner` forwards it as a `heartbeat` turn event and `SseFrameEncoder` renders it as an SSE comment (`: ping`), which every existing consumer ignores without being taught to. Configurable via `conversations.heartbeat_seconds` (default 5, `0` disables). +- The heartbeat carries a partial-line buffer across idle windows. Without it a timeout mid-frame would hand the parser half a `data:` line, which fails `json_decode` silently and leaves its remainder to be dropped — losing the frame outright. +- Fixes a latent bug the heartbeat exposes: the maximum-stream-duration guard was only evaluated when a provider event arrived, so a stalled stream could run well past `conversations.max_stream_seconds` unnoticed. Heartbeats give the guard something to run on. +- Adds detached turns. `AiPersonaConversationService::dispatchTurn()` runs a turn as a queued job that appends each yielded event to `ai_turn_events`; `resumeTurn()` streams them back from any sequence, so a browser reload resumes the turn instead of killing it. `cancelTurn()` stops one early. +- Frames each detached event with an SSE `id:` carrying its sequence, so a reconnecting consumer resumes from `Last-Event-ID` rather than replaying the turn. The published client reports it through a new `onSequence` callback. +- Replaces `connection_aborted()` for detached turns, which reports 0 in a queue worker and would mean a turn never stops. A run is abandoned when nothing has read it for `turns.abandon_after_seconds` (default 30) — so closing the tab still stops generation, and a reload inside that window reattaches to the same run. +- Adds `ai:prune-turn-events` (scheduled daily at 03:15) to clear finished runs past `turns.retention_days`. + +The synchronous path is untouched. `continueConversation()` behaves exactly as before, and the job calls it — there is one turn implementation, not two. + +## Capabilities + +### Modified Capabilities +- `chat-turn-library`: adds `heartbeat` to the turn event vocabulary and its SSE comment-frame encoding; adds the detached-turn lifecycle (dispatch, resume from a sequence, cancel, abandon) alongside the existing synchronous call; adds sequence-bearing SSE framing. + +## Impact + +- **Code**: `Heartbeat` stream event and `HeartbeatsIdleSseReads` trait; `ReasoningOpenAiCompatibleGateway`, `ConversationTurnRunner`, and `SseFrameEncoder` modified; `AiTurnRun`/`AiTurnEvent` models with two migrations; `TurnRunStore`, `TurnEventStream`, `RunConversationTurnJob`, `PruneTurnEventsCommand`; three new methods on `AiPersonaConversationService`. +- **Migrations**: `ai_turn_runs` and `ai_turn_events`. Hosts re-publish and migrate even if they never call `dispatchTurn()` — `ai:prune-turn-events` is scheduled by default and expects the tables. +- **Config**: `conversations.heartbeat_seconds`; a new `turns` block (`queue`, `abandon_after_seconds`, `poll_interval_ms`, `max_stream_seconds`, `retention_days`). +- **Frontend contract**: README turn-events table gains `heartbeat`; `code-talker-stream.ts` gains `onSequence`. `heartbeat` is deliberately **not** added to the `ChatStreamEvent` union in `code-talker.d.ts` — that union describes what arrives over the wire, and a comment frame never does. +- **Scope limit**: gateway-level heartbeats reach `openai-compatible` and `lm-studio` systems only, because that is the one gateway this package overrides. A detached turn heartbeats for every provider, because its beat is measured against the store rather than a socket. +- **Not in scope**: a broadcasting transport (the store is its prerequisite and needs no schema change to add one); multi-viewer fan-out; resuming a turn whose worker died mid-generation. diff --git a/openspec/changes/durable-turns-and-heartbeats/specs/chat-turn-library/spec.md b/openspec/changes/durable-turns-and-heartbeats/specs/chat-turn-library/spec.md new file mode 100644 index 0000000..0a1e5bf --- /dev/null +++ b/openspec/changes/durable-turns-and-heartbeats/specs/chat-turn-library/spec.md @@ -0,0 +1,140 @@ +## MODIFIED Requirements + +### Requirement: A turn is a library call yielding structured events + +The chat turn SHALL be driven directly as a library call that yields structured events, so a host can deliver them over any transport. + +#### Scenario: Driving a turn + +- **WHEN** a host continues a conversation with a message +- **THEN** it receives an iterable of event arrays, each carrying a `type` +- **AND** the event vocabulary is `status`, `message_start`, `content_block_delta`, `reasoning_block_delta`, `message_delta`, `message_stop`, `tool_use_progress`, `page_reload`, `heartbeat`, and `error` + +#### Scenario: No transport encoding leaks into the turn + +- **WHEN** a turn yields an event +- **THEN** it is a structured array, not a wire-encoded string, so the caller chooses the encoding + +### Requirement: Server-sent events remain available as a helper + +The package SHALL ship an encoder that turns the event stream into the documented server-sent-event framing, so a host preserving the existing wire format does not reimplement it. + +#### Scenario: Encoding a finished turn + +- **WHEN** a completed turn's events are encoded +- **THEN** each is emitted as `data: \n\n` +- **AND** the stream ends with the literal `data: [DONE]\n\n` + +#### Scenario: An error terminates the stream on its own + +- **WHEN** a turn emits an error event +- **THEN** the encoded stream ends without the `[DONE]` sentinel + +#### Scenario: A heartbeat is encoded as a comment + +- **WHEN** a heartbeat event is encoded +- **THEN** it is emitted as the comment frame `: ping\n\n` rather than a data frame +- **AND** it does not terminate the stream, so a turn carrying heartbeats still ends with the sentinel + +#### Scenario: A sequenced event carries a resumable id + +- **WHEN** an event carrying a sequence is encoded +- **THEN** the frame is preceded by `id: ` +- **AND** the sequence does not appear inside the event's JSON payload + +## ADDED Requirements + +### Requirement: A silent provider does not mean a silent connection + +A turn SHALL emit a heartbeat while the provider produces nothing, so an intermediary does not time out mid-answer and an abandoned connection is detected in seconds rather than minutes. + +#### Scenario: The provider goes quiet mid-turn + +- **WHEN** a configured heartbeat interval elapses with no provider event +- **THEN** the turn yields a `heartbeat` event +- **AND** continues reading the provider stream from where it paused + +#### Scenario: A frame spanning a silent gap is not lost + +- **WHEN** the provider stops part way through emitting an event frame and resumes after a heartbeat +- **THEN** the completed frame is parsed as a single event, with nothing dropped + +#### Scenario: Heartbeats are not part of the record + +- **WHEN** a turn that emitted heartbeats is logged +- **THEN** the stored provider events contain no heartbeat + +#### Scenario: Heartbeats are disabled + +- **WHEN** the heartbeat interval is configured as `0` +- **THEN** the turn reads the provider stream exactly as it did before heartbeats existed + +### Requirement: The stream duration guard applies during silence + +The maximum-stream-duration guard SHALL be evaluated while the provider is silent, not only when it emits. + +#### Scenario: A stalled stream exceeds its budget + +- **WHEN** a turn passes its maximum stream duration with no provider event since +- **THEN** the turn is stopped and reported as a duration failure, without waiting for the provider to emit again + +### Requirement: A turn can outlive the connection that started it + +A turn SHALL be dispatchable as a background run whose events are recorded, so a caller can disconnect and reattach without destroying it. + +#### Scenario: Dispatching a turn + +- **WHEN** a host dispatches a turn for a conversation +- **THEN** it receives a run with a shareable public identifier +- **AND** the turn is executed in the background, recording each event it yields in order + +#### Scenario: Reattaching after a reload + +- **WHEN** a caller streams a run from the last sequence it received +- **THEN** it receives every event recorded after that sequence and nothing it already had +- **AND** continues to receive events until the run ends + +#### Scenario: The final event survives the run finishing mid-read + +- **WHEN** a run records its last event and finishes between a reader's two reads +- **THEN** the reader still receives that event before stopping + +#### Scenario: Cancelling a dispatched turn + +- **WHEN** a host cancels a run +- **THEN** the turn stops generating +- **AND** whatever it produced is persisted and flagged incomplete + +### Requirement: A dispatched turn nobody is reading stops + +Because connection state is unavailable to a background worker, a dispatched run SHALL treat the absence of a reader as the signal to stop. + +#### Scenario: The reader goes away + +- **WHEN** no caller has read a run for the configured abandonment window +- **THEN** the turn stops generating and the run is recorded as abandoned + +#### Scenario: A run is not killed before its first reader connects + +- **WHEN** a run has been dispatched but never read, and the abandonment window has not yet elapsed since it was created +- **THEN** it continues running + +#### Scenario: Reading keeps a run alive + +- **WHEN** a caller is streaming a run +- **THEN** the run is not abandoned for as long as the reading continues + +#### Scenario: A worker dies mid-run + +- **WHEN** the process running a turn fails +- **THEN** the run is recorded as failed, so a reader stops rather than waiting indefinitely + +### Requirement: Recorded turn events are retained for a bounded window + +Recorded runs and their events SHALL be prunable, so the store does not grow without limit. + +#### Scenario: Pruning finished runs + +- **WHEN** the retention command runs +- **THEN** finished runs older than the retention window are removed with their events +- **AND** runs that are still executing are left alone, however old diff --git a/openspec/changes/durable-turns-and-heartbeats/tasks.md b/openspec/changes/durable-turns-and-heartbeats/tasks.md new file mode 100644 index 0000000..c8c4c8c --- /dev/null +++ b/openspec/changes/durable-turns-and-heartbeats/tasks.md @@ -0,0 +1,59 @@ +## 1. Heartbeats during idle provider reads + +- [x] 1.1 Add `Services/LaravelAi/Streaming/Heartbeat`, a `StreamEvent` reporting `type: 'heartbeat'` +- [x] 1.2 Add `Services/LaravelAi/Concerns/HeartbeatsIdleSseReads` overriding `parseServerSentEvents()` with a `stream_set_timeout()`-bounded read whose partial-line buffer survives an idle window +- [x] 1.3 Delegate to the parent parser when the interval is `0` or the body has no stream resource, checked before `detach()` +- [x] 1.4 Check `timed_out` before `feof()`, so a read timeout is never mistaken for the end of the stream +- [x] 1.5 Use the trait in `ReasoningOpenAiCompatibleGateway` and pass a `Heartbeat` through `processTextStream()` +- [x] 1.6 Add `conversations.heartbeat_seconds` (default 5, `0` disables) + +## 2. The turn forwards heartbeats + +- [x] 2.1 `ConversationTurnRunner` yields `['type' => 'heartbeat']` without appending to `$events` or logging it +- [x] 2.2 A heartbeat does not reset the step clock, and does reach the max-duration guard +- [x] 2.3 `SseFrameEncoder` renders `heartbeat` as `": ping\n\n"`, non-terminal +- [x] 2.4 Document the event in the README; record in `code-talker.d.ts` why it is absent from `ChatStreamEvent` + +## 3. Turn run schema and models + +- [x] 3.1 Migration: `ai_turn_runs` — `public_id`, `ai_conversation_id`, `status`, `prompt`, `last_polled_at`, `cancel_requested_at`, `started_at`, `finished_at`, `error_message` +- [x] 3.2 Migration: `ai_turn_events` — `ai_turn_run_id`, `sequence`, `payload`, unique on (run, sequence) +- [x] 3.3 Add `AiTurnRunStatus` with `isTerminal()` +- [x] 3.4 Add `AiTurnRun` (ULID `public_id`, enum + timestamp casts, `events()` ordered by sequence) and `AiTurnEvent` + +## 4. TurnRunStore + +- [x] 4.1 `open`, `markRunning`, `append`, `finish`, `eventsAfter`, `touchPoll`, `requestCancel` +- [x] 4.2 `shouldStop()` throttled to one read every two seconds, so it never queries per token +- [x] 4.3 Abandonment measured from `last_polled_at`, or `created_at` while that is null +- [x] 4.4 `stopStatusFor()` distinguishing `Cancelled` from `Abandoned` +- [x] 4.5 Add the `turns` config block + +## 5. RunConversationTurnJob + +- [x] 5.1 Constructed with the run id; drives the existing `continueConversation()` and appends every yielded event +- [x] 5.2 Binds `usingCancellationCheck()` to the store's stop signal +- [x] 5.3 Finishes the run `Completed`, `Cancelled` or `Abandoned` as the stop reason implies +- [x] 5.4 `failed()` marks the run `Failed`, so a dead worker does not leave a reader polling forever + +## 6. TurnEventStream and resumable framing + +- [x] 6.1 Reader generator replaying from any sequence and following a live run +- [x] 6.2 Drains once more after seeing a terminal status, so the job's final event is never dropped +- [x] 6.3 Stamps `last_polled_at` each pass; emits a provider-agnostic `heartbeat` while quiet; bounded by `turns.max_stream_seconds` +- [x] 6.4 `SseFrameEncoder` emits `id: ` from `_seq` and strips it from the payload + +## 7. Service entry points + +- [x] 7.1 `dispatchTurn()`, `resumeTurn()`, `cancelTurn()`, resolving collaborators inside the methods so the five-argument constructor is unchanged + +## 8. Retention + +- [x] 8.1 `ai:prune-turn-events` removing terminal runs past `turns.retention_days` and their events, leaving live runs alone +- [x] 8.2 Register the command and schedule it daily at 03:15 + +## 9. Contract and documentation + +- [x] 9.1 `code-talker-stream.ts` reports the SSE `id:` through a new `onSequence` callback +- [x] 9.2 README: the `heartbeat` event, the detached-turn workflow, and the `turns.*` config +- [x] 9.3 CHANGELOG: fold both halves into the 0.15.0 entry, including the new migrations under Breaking Changes diff --git a/resources/js/code-talker-stream.ts b/resources/js/code-talker-stream.ts index 290d5f4..8192969 100644 --- a/resources/js/code-talker-stream.ts +++ b/resources/js/code-talker-stream.ts @@ -64,6 +64,12 @@ export interface ChatTurnCallbacks { * delivered through onText/onReasoning is still valid. */ onError?: (error: ChatTurnError) => void; + /** + * The sequence of the last event received, present only for a turn + * dispatched with `dispatchTurn()`. Pass it back as `after` when + * reconnecting so the turn resumes rather than replays. + */ + onSequence?: (sequence: number) => void; } export interface ChatTurn { @@ -213,6 +219,18 @@ async function consume(body: ReadableStream, callbacks: ChatTurnCall * @return true when the stream is finished and reading should stop. */ function dispatch(frame: string, callbacks: ChatTurnCallbacks): boolean { + const idLine = frame.split('\n').find((line) => line.startsWith('id:')); + + if (idLine !== undefined) { + const parsed = Number.parseInt(idLine.slice(3).trim(), 10); + + if (!Number.isNaN(parsed)) { + // Recorded so a caller reconnecting after a dropped connection can + // resume from here instead of replaying the whole turn. + callbacks.onSequence?.(parsed); + } + } + const data = frame .split('\n') .filter((line) => line.startsWith('data:')) diff --git a/resources/js/types/code-talker.d.ts b/resources/js/types/code-talker.d.ts index 67ec077..07be624 100644 --- a/resources/js/types/code-talker.d.ts +++ b/resources/js/types/code-talker.d.ts @@ -32,14 +32,24 @@ export interface ChatMessage { reasoning_content: string | null; /** Ordered content runs; null on messages stored before blocks existed. */ blocks: MessageBlock[] | null; + /** + * The reply was never finished — the browser hung up, or the server's + * duration guard cut it off. `content` may be empty or stop mid-sentence; + * render it as interrupted rather than as an answer. + */ + incomplete: boolean; } // --------------------------------------------------------------------------- // Stream events // --------------------------------------------------------------------------- -/** Why the turn stopped. */ -export type StopReason = 'end_turn' | 'max_tokens' | 'tool_use'; +/** + * Why the turn stopped. `incomplete` means the turn never finished — the + * connection dropped, or the server's duration guard cut the generation off — + * so whatever content arrived stops mid-answer. + */ +export type StopReason = 'end_turn' | 'max_tokens' | 'tool_use' | 'incomplete'; /** * Why the turn failed. Absent for a recoverable in-stream provider error and @@ -113,6 +123,21 @@ export interface PageReloadEvent { type: 'page_reload'; } +/** + * `heartbeat` is deliberately absent from this union. The server yields it as + * a turn event, but `SseFrameEncoder` writes it as an SSE comment (`: ping`), + * which never arrives as a message — so a wire consumer cannot receive one and + * should not be made to handle it. A host consuming the events directly, + * without the SSE encoding, will see `{ type: 'heartbeat' }`. + * + * A turn dispatched with `dispatchTurn()` frames each stored event with an SSE + * `id:` line carrying its sequence — present only on that path, never for + * `continueConversation()`. The published client's `ChatTurnCallbacks` reports + * it through `onSequence?: (sequence: number) => void`: the sequence of the + * last event received, to pass back as `after` when reconnecting so the turn + * resumes rather than replays. + */ + /** Every event the message endpoint emits, discriminated on `type`. */ export type ChatStreamEvent = | StatusEvent diff --git a/src/CodeTalkerServiceProvider.php b/src/CodeTalkerServiceProvider.php index 1a205f2..06f3f1b 100644 --- a/src/CodeTalkerServiceProvider.php +++ b/src/CodeTalkerServiceProvider.php @@ -6,6 +6,7 @@ use Jvjvjv\CodeTalker\Console\Commands\BackfillConversationUsageCommand; use Jvjvjv\CodeTalker\Console\Commands\CompleteIdleConversationsCommand; use Jvjvjv\CodeTalker\Console\Commands\PruneProviderExchangesCommand; +use Jvjvjv\CodeTalker\Console\Commands\PruneTurnEventsCommand; use Jvjvjv\CodeTalker\Console\Commands\ReadProviderExchangeCommand; use Jvjvjv\CodeTalker\Console\Commands\SyncConversationUsageCommand; use Jvjvjv\CodeTalker\Jobs\BackfillConversationUsageJob; @@ -198,6 +199,7 @@ public function boot(): void BackfillConversationUsageCommand::class, SyncConversationUsageCommand::class, PruneProviderExchangesCommand::class, + PruneTurnEventsCommand::class, CompleteIdleConversationsCommand::class, ReadProviderExchangeCommand::class, ]); @@ -217,6 +219,10 @@ public function boot(): void ->dailyAt('03:00') ->withoutOverlapping(); + Schedule::command('ai:prune-turn-events') + ->dailyAt('03:15') + ->withoutOverlapping(); + Schedule::command('ai:complete-idle-conversations') ->everyFifteenMinutes() ->withoutOverlapping(); diff --git a/src/Console/Commands/PruneTurnEventsCommand.php b/src/Console/Commands/PruneTurnEventsCommand.php new file mode 100644 index 0000000..90534e7 --- /dev/null +++ b/src/Console/Commands/PruneTurnEventsCommand.php @@ -0,0 +1,70 @@ +info('Turn event retention is disabled; nothing pruned.'); + + return self::SUCCESS; + } + + $terminal = array_values(array_map( + static fn (AiTurnRunStatus $status): string => $status->value, + array_filter(AiTurnRunStatus::cases(), static fn (AiTurnRunStatus $s): bool => $s->isTerminal()), + )); + + $cutoff = now()->subDays($days); + $pruned = 0; + + do { + // Only finished runs: a turn still generating is not garbage, + // however long it has been going. + $runIds = AiTurnRun::query() + ->whereIn('status', $terminal) + ->where('created_at', '<', $cutoff) + ->orderBy('id') + ->limit(self::PAGE_SIZE) + ->pluck('id'); + + if ($runIds->isEmpty()) { + break; + } + + // Events before runs: a failure between the two leaves runs whose + // events are gone, and the next sweep re-selects and finishes them. + AiTurnEvent::query()->whereIn('ai_turn_run_id', $runIds)->delete(); + AiTurnRun::query()->whereIn('id', $runIds)->delete(); + + $pruned += $runIds->count(); + } while ($runIds->count() === self::PAGE_SIZE); + + if ($pruned === 0) { + $this->info('No turn runs past retention.'); + + return self::SUCCESS; + } + + $this->info("Pruned {$pruned} turn run(s)."); + + return self::SUCCESS; + } +} diff --git a/src/Enums/AiInteractionStatus.php b/src/Enums/AiInteractionStatus.php index af2d2f1..6111b4d 100644 --- a/src/Enums/AiInteractionStatus.php +++ b/src/Enums/AiInteractionStatus.php @@ -6,4 +6,12 @@ enum AiInteractionStatus: string { case Success = 'success'; case Error = 'error'; + + /** + * The turn ran but never finished: the caller (usually the browser) hung + * up mid-stream. Not an error — nothing failed, and the tokens it burned + * still count towards the conversation's usage — but not a success either, + * because the answer it was producing is incomplete. + */ + case Aborted = 'aborted'; } diff --git a/src/Enums/AiTurnRunStatus.php b/src/Enums/AiTurnRunStatus.php new file mode 100644 index 0000000..92e7ad7 --- /dev/null +++ b/src/Enums/AiTurnRunStatus.php @@ -0,0 +1,26 @@ + false, + self::Completed, self::Failed, self::Cancelled, self::Abandoned => true, + }; + } +} diff --git a/src/Jobs/RunConversationTurnJob.php b/src/Jobs/RunConversationTurnJob.php new file mode 100644 index 0000000..e224501 --- /dev/null +++ b/src/Jobs/RunConversationTurnJob.php @@ -0,0 +1,132 @@ +onQueue(config('code-talker.turns.queue') ?: null); + } + + public function handle(AiPersonaConversationService $chat, TurnRunStore $store): void + { + $run = AiTurnRun::find($this->turnRunId); + + if ($run === null || $run->status->isTerminal()) { + return; + } + + $store->markRunning($run); + + $errorSeen = false; + $errorMessage = null; + + try { + $events = $chat + ->usingCancellationCheck(fn (): bool => $store->shouldStop($run)) + ->continueConversation($run->conversation, $run->prompt); + + foreach ($events as $event) { + // A heartbeat is consumed here, never stored: it reports that + // the provider was silent, not that anything happened, and + // SseFrameEncoder transmits it as a comment with no id — so a + // stored one would burn a sequence number a reader never sees, + // leaving a reconnecting client's `after` cursor behind the + // real sequence and replaying events it already had. + // TurnEventStream synthesizes its own heartbeat whenever the + // store is quiet, which is what a reader actually needs. + if (($event['type'] ?? null) === 'heartbeat') { + continue; + } + + // continueConversation() catches provider failures internally + // and yields a terminal error event instead of throwing, so + // the generator completes normally — remember the failure or + // the run would be recorded as a clean completion. + if (($event['type'] ?? null) === 'error') { + $errorSeen = true; + $errorMessage = $event['message'] ?? null; + } + + $store->append($run, $event); + } + } catch (Throwable $exception) { + $store->finish($run, AiTurnRunStatus::Failed, $exception->getMessage()); + + throw $exception; + } + + // Terminal status precedence: an error event outranks a stop request + // (in practice disjoint — a client abort yields no error event, and a + // provider failure ends the turn before any stop is noticed — but the + // precedence is explicit so that stays true), and only a run that saw + // neither is Completed. + if ($errorSeen) { + $store->finish($run, AiTurnRunStatus::Failed, $errorMessage); + } elseif ($store->shouldStop($run)) { + $store->finish($run, $store->stopStatusFor($run)); + } else { + $store->finish($run, AiTurnRunStatus::Completed); + } + } + + /** + * A worker that dies leaves a reader polling forever unless the run is + * closed out here. + */ + public function failed(?Throwable $exception): void + { + $run = AiTurnRun::find($this->turnRunId); + + if ($run === null || $run->status->isTerminal()) { + return; + } + + app(TurnRunStore::class)->finish( + $run, + AiTurnRunStatus::Failed, + $exception?->getMessage(), + ); + } +} diff --git a/src/Models/AiTurnEvent.php b/src/Models/AiTurnEvent.php new file mode 100644 index 0000000..1cfe944 --- /dev/null +++ b/src/Models/AiTurnEvent.php @@ -0,0 +1,49 @@ +created_at === null) { + $event->created_at = Carbon::now(); + } + }); + } + + protected $fillable = [ + 'ai_turn_run_id', + 'sequence', + 'payload', + 'created_at', + ]; + + protected function casts(): array + { + return [ + 'payload' => 'array', + 'created_at' => 'datetime', + ]; + } + + public function run(): BelongsTo + { + return $this->belongsTo(AiTurnRun::class, 'ai_turn_run_id'); + } +} diff --git a/src/Models/AiTurnRun.php b/src/Models/AiTurnRun.php new file mode 100644 index 0000000..e89f0c8 --- /dev/null +++ b/src/Models/AiTurnRun.php @@ -0,0 +1,64 @@ +public_id)) { + $run->public_id = (string) Str::ulid(); + } + }); + } + + protected function casts(): array + { + return [ + 'status' => AiTurnRunStatus::class, + 'last_polled_at' => 'datetime', + 'cancel_requested_at' => 'datetime', + 'started_at' => 'datetime', + 'finished_at' => 'datetime', + ]; + } + + public function conversation(): BelongsTo + { + return $this->belongsTo(AiConversation::class, 'ai_conversation_id'); + } + + public function events(): HasMany + { + return $this->hasMany(AiTurnEvent::class, 'ai_turn_run_id')->orderBy('sequence'); + } +} diff --git a/src/Services/AiPersonaConversationService.php b/src/Services/AiPersonaConversationService.php index 25b5deb..98758eb 100644 --- a/src/Services/AiPersonaConversationService.php +++ b/src/Services/AiPersonaConversationService.php @@ -5,9 +5,13 @@ use Generator; use RuntimeException; use Jvjvjv\CodeTalker\Enums\AiConversationStatus; +use Jvjvjv\CodeTalker\Jobs\RunConversationTurnJob; use Jvjvjv\CodeTalker\Models\AiPersona; use Jvjvjv\CodeTalker\Models\AiConversation; use Jvjvjv\CodeTalker\Models\AiConversationMessage; +use Jvjvjv\CodeTalker\Models\AiTurnRun; +use Jvjvjv\CodeTalker\Services\Conversation\TurnEventStream; +use Jvjvjv\CodeTalker\Services\Conversation\TurnRunStore; use Jvjvjv\CodeTalker\Services\ChatBot\Conversation\ConversationTitle; use Jvjvjv\CodeTalker\Services\ChatBot\Conversation\ConversationTurnRunner; use Jvjvjv\CodeTalker\Services\ChatBot\Conversation\RequestPayloadBuilder; @@ -247,6 +251,62 @@ public function continueConversation(AiConversation $conversation, string $userM } } + /** + * Run a turn detached from the caller's connection. + * + * The turn becomes a queued job that writes its events to a store; the + * browser reads them with resumeTurn() and can reconnect at any point. Use + * this instead of continueConversation() when a turn is long enough that a + * reload or a flaky connection should not destroy it. + * + * The store and reader are resolved here rather than injected: this + * service's five-argument constructor is depended on by host apps and by + * tests that subclass it, so collaborators are built from what it has. + */ + public function dispatchTurn(AiConversation $conversation, string $message): AiTurnRun + { + $run = app(TurnRunStore::class)->open($conversation, $message); + + // afterCommit(): a host may wrap this call in its own transaction, and + // with a Redis/SQS queue a worker could pick the job up before the run + // row commits — the job would find nothing and return, leaving the run + // Queued while a reader polls heartbeats until the stream ceiling. + // With no transaction open the job dispatches immediately, so this is + // safe unconditionally. + RunConversationTurnJob::dispatch($run->id)->afterCommit(); + + return $run; + } + + /** + * Stream a dispatched turn's events, starting after the given sequence. + * + * Yields the turn's stored events — the shapes continueConversation() + * produces — each carrying a `_seq` the encoder turns into an SSE id, plus + * events the stream synthesizes while waiting on the store: a `heartbeat` + * per beat of silence, and a `max_stream_duration` error at the reader + * ceiling. Synthesized events are never stored and carry no `_seq`, so + * read that key conditionally. A browser that reconnects passes back the + * last sequence it saw and misses nothing in between. + * + * @return Generator> + */ + public function resumeTurn(AiTurnRun $run, int $after = 0): Generator + { + yield from app(TurnEventStream::class)->stream($run, $after); + } + + /** + * Ask a running turn to stop. + * + * The worker notices within a couple of seconds and stops generating; + * whatever the turn produced by then is persisted and flagged incomplete. + */ + public function cancelTurn(AiTurnRun $run): void + { + app(TurnRunStore::class)->requestCancel($run); + } + /** * Wall-clock seconds elapsed since the turn started. Extracted so tests can * drive the max-stream-duration guard deterministically. diff --git a/src/Services/ChatBot/ChatBotPresenter.php b/src/Services/ChatBot/ChatBotPresenter.php index 69e4593..d8361af 100644 --- a/src/Services/ChatBot/ChatBotPresenter.php +++ b/src/Services/ChatBot/ChatBotPresenter.php @@ -21,7 +21,7 @@ class ChatBotPresenter * The visible transcript: everything except the system prompt, which is * instructions rather than something anyone said. * - * @return array + * @return array */ public function transcript(?AiConversation $conversation): array { @@ -39,6 +39,11 @@ public function transcript(?AiConversation $conversation): array 'content' => $message->content, 'reasoning_content' => $message->reasoning_content, 'blocks' => $message->blocks, + // A reply the model never finished — the browser hung up, or + // the duration guard cut it off. Its content, if any, stops + // mid-sentence; render it as interrupted rather than as an + // answer. + 'incomplete' => (bool) ($message->metadata['incomplete'] ?? false), ]) ->all(); } diff --git a/src/Services/ChatBot/Conversation/ConversationTurnRunner.php b/src/Services/ChatBot/Conversation/ConversationTurnRunner.php index fe0c705..abcc299 100644 --- a/src/Services/ChatBot/Conversation/ConversationTurnRunner.php +++ b/src/Services/ChatBot/Conversation/ConversationTurnRunner.php @@ -9,6 +9,7 @@ use Jvjvjv\CodeTalker\Services\LaravelAi\AiSystemProviderConfigurator; use Jvjvjv\CodeTalker\Services\LaravelAi\CodeTalkerAgent; use Jvjvjv\CodeTalker\Services\LaravelAi\StreamTranslator; +use Jvjvjv\CodeTalker\Services\LaravelAi\Streaming\Heartbeat; use Jvjvjv\CodeTalker\Services\RawExchange\RawExchangeContext; use Jvjvjv\CodeTalker\Services\RawExchange\RawExchangeFrame; use Laravel\Ai\Messages\AssistantMessage; @@ -136,16 +137,25 @@ public function run( break; } - Log::debug('Chat bot API stream event', [ - 'conversation_id' => $conversation->id, - 'ai_persona_id' => $conversation->ai_persona_id, - 'ai_system_id' => $conversation->ai_system_id, - 'turn_number' => $turnNumber, - 'attempt' => $attempt, - 'event_type' => class_basename($event), - ]); - - $events[] = $event; + // A tick, not model output. It is deliberately not logged + // and not appended to $events — a turn with minute-long + // gaps would otherwise fill response_data with hundreds of + // them — and it deliberately does not reset the step + // clock, which would put the duration guard out of reach. + $isHeartbeat = $event instanceof Heartbeat; + + if (! $isHeartbeat) { + Log::debug('Chat bot API stream event', [ + 'conversation_id' => $conversation->id, + 'ai_persona_id' => $conversation->ai_persona_id, + 'ai_system_id' => $conversation->ai_system_id, + 'turn_number' => $turnNumber, + 'attempt' => $attempt, + 'event_type' => class_basename($event), + ]); + + $events[] = $event; + } if ($event instanceof ToolResultEvent) { $recordedToolResults[] = $event->toolResult->toArray(); @@ -204,6 +214,12 @@ public function run( break; } + if ($isHeartbeat) { + yield ['type' => 'heartbeat']; + + continue; + } + if ($event instanceof ToolCallEvent) { $toolCalls[] = [ 'id' => $event->toolCall->id, diff --git a/src/Services/ChatBot/Conversation/TurnRecorder.php b/src/Services/ChatBot/Conversation/TurnRecorder.php index 31e1f10..c8df127 100644 --- a/src/Services/ChatBot/Conversation/TurnRecorder.php +++ b/src/Services/ChatBot/Conversation/TurnRecorder.php @@ -35,12 +35,21 @@ public function recordCompletedTurn( ): void { $text = $blocks->text(); $reasoning = $blocks->reasoning(); + $incompleteReason = $this->incompleteReason($outcome); - // Persist whatever was produced even when the turn was cut off by - // the max-duration guard — including reasoning-only content, so a - // model stuck deliberating without ever answering doesn't vanish - // without a trace. - if ($text !== '' || $reasoning !== '') { + // Persist whatever the turn produced, and record the turn itself even + // when it produced nothing visible. + // + // A turn cut short still happened: its tool calls changed state on the + // host's side, and the user is owed something other than silence under + // their question. Writing only when there was text or reasoning meant a + // turn abandoned during prompt processing — which, on a large context, + // is where most of a turn's wall-clock time goes — vanished entirely, + // leaving a user message with no reply beneath it and no record that + // anything had ever run. + $producedSomething = $text !== '' || $reasoning !== '' || $outcome->toolCalls !== []; + + if ($producedSomething || $incompleteReason !== null) { AiConversationMessage::create([ 'ai_conversation_id' => $conversation->id, 'role' => 'assistant', @@ -61,6 +70,12 @@ public function recordCompletedTurn( 'input_tokens' => $inputTokens ?: null, 'output_tokens' => $outputTokens ?: null, 'model' => $conversation->aiSystem->model, + // The flag a host renders "this reply was interrupted" + // from. Always present, so a host can read it without + // having to distinguish "complete" from "stored before + // this existed". + 'incomplete' => $incompleteReason !== null, + 'incomplete_reason' => $incompleteReason, ], ]); } @@ -82,12 +97,21 @@ public function recordCompletedTurn( 'input_token_price_snapshot' => $pricingSnapshot['input_token_price_snapshot'], 'output_token_price_snapshot' => $pricingSnapshot['output_token_price_snapshot'], 'duration_ms' => $outcome->durationMs, - 'status' => $outcome->maxDurationExceeded ? AiInteractionStatus::Error : AiInteractionStatus::Success, + // A turn that was cut short is never logged as a success: doing so + // made a truncated turn indistinguishable from a clean one in + // every dashboard the logs feed. + 'status' => match (true) { + $outcome->maxDurationExceeded => AiInteractionStatus::Error, + $outcome->clientAborted => AiInteractionStatus::Aborted, + default => AiInteractionStatus::Success, + }, ]; if ($outcome->maxDurationExceeded) { $log['error_message'] = $outcome->maxDurationMessage; $log['provider_metadata'] = ['error_reason' => 'max_stream_duration']; + } elseif ($outcome->clientAborted) { + $log['provider_metadata'] = ['error_reason' => 'client_aborted']; } AiInteractionLog::create($log); @@ -95,6 +119,18 @@ public function recordCompletedTurn( $this->conversationUsageService->syncConversation($conversation->fresh()); } + /** + * Why the turn stopped short, or null if it ran to completion. + */ + private function incompleteReason(TurnOutcome $outcome): ?string + { + return match (true) { + $outcome->maxDurationExceeded => 'max_stream_duration', + $outcome->clientAborted => 'client_aborted', + default => null, + }; + } + /** * Record a turn that never completed — an unsupported provider, an * unrecoverable in-stream error event, or any other provider failure. diff --git a/src/Services/ChatBot/SseFrameEncoder.php b/src/Services/ChatBot/SseFrameEncoder.php index be82ce7..10d7212 100644 --- a/src/Services/ChatBot/SseFrameEncoder.php +++ b/src/Services/ChatBot/SseFrameEncoder.php @@ -23,12 +23,28 @@ public function encode(iterable $events): Generator $failed = false; foreach ($events as $event) { + // A comment frame, not a data frame: it exists to put a byte on + // the wire during a silent gap, and every SSE consumer ignores it + // without being taught to. + if (($event['type'] ?? null) === 'heartbeat') { + yield ": ping\n\n"; + + continue; + } + // An error event is terminal on its own — nothing follows it, and // the stream is not sentinel-terminated. Consumers rely on this to // tell a failed turn from a finished one. $failed = ($event['type'] ?? null) === 'error'; - yield 'data: ' . json_encode($event) . "\n\n"; + // `_seq` is framing metadata, not part of the event vocabulary: it + // becomes the SSE id a reconnecting consumer resumes from, and + // never reaches the browser inside the payload. + $sequence = $event['_seq'] ?? null; + unset($event['_seq']); + + yield ($sequence === null ? '' : 'id: ' . $sequence . "\n") + . 'data: ' . json_encode($event) . "\n\n"; } if (! $failed) { diff --git a/src/Services/Conversation/TurnEventStream.php b/src/Services/Conversation/TurnEventStream.php new file mode 100644 index 0000000..6f952a2 --- /dev/null +++ b/src/Services/Conversation/TurnEventStream.php @@ -0,0 +1,95 @@ +> + */ + public function stream(AiTurnRun $run, int $after = 0): Generator + { + $pollMicroseconds = max(1, (int) config('code-talker.turns.poll_interval_ms', 250)) * 1000; + $heartbeatSeconds = (int) config('code-talker.conversations.heartbeat_seconds', 5); + $maxSeconds = (int) config('code-talker.turns.max_stream_seconds', 900); + + $startedAt = microtime(true); + $lastEmittedAt = $startedAt; + + while (true) { + $this->store->touchPoll($run); + + $events = $this->store->eventsAfter($run, $after); + + if ($events->isNotEmpty()) { + foreach ($events as $event) { + /** @var AiTurnEvent $event */ + $after = $event->sequence; + $lastEmittedAt = microtime(true); + + yield $event->payload + ['_seq' => $event->sequence]; + } + + continue; + } + + // Nothing new. Read the status only now, and drain before stopping: + // the job appends its last event and *then* marks the run finished, + // so checking status first would drop that event. The drain pages + // until it comes back empty — eventsAfter() caps each read, and a + // backlog larger than one page must not be truncated. + if ($run->fresh()?->status->isTerminal() ?? true) { + do { + $drained = $this->store->eventsAfter($run, $after); + + foreach ($drained as $event) { + /** @var AiTurnEvent $event */ + $after = $event->sequence; + + yield $event->payload + ['_seq' => $event->sequence]; + } + } while ($drained->isNotEmpty()); + + return; + } + + if ($maxSeconds > 0 && microtime(true) - $startedAt > $maxSeconds) { + yield [ + 'type' => 'error', + 'message' => "The turn exceeded the maximum stream duration of {$maxSeconds}s.", + 'reason' => 'max_stream_duration', + ]; + + return; + } + + if ($heartbeatSeconds > 0 && microtime(true) - $lastEmittedAt >= $heartbeatSeconds) { + $lastEmittedAt = microtime(true); + + // Provider-agnostic, unlike the gateway's own heartbeat: this + // one fires for every provider, because it is measured against + // the store rather than a socket. + yield ['type' => 'heartbeat']; + } + + usleep($pollMicroseconds); + } + } +} diff --git a/src/Services/Conversation/TurnRunStore.php b/src/Services/Conversation/TurnRunStore.php new file mode 100644 index 0000000..e80780a --- /dev/null +++ b/src/Services/Conversation/TurnRunStore.php @@ -0,0 +1,158 @@ + $conversation->id, + 'status' => AiTurnRunStatus::Queued, + 'prompt' => $message, + ]); + } + + public function markRunning(AiTurnRun $run): void + { + $run->forceFill([ + 'status' => AiTurnRunStatus::Running, + 'started_at' => now(), + ])->save(); + + $this->sequence = (int) $run->events()->max('sequence'); + } + + /** + * Single-writer contract: sequences come from an in-memory counter seeded + * by markRunning(), so exactly one store instance may write a given run. + * Concurrent writers collide on the unique (ai_turn_run_id, sequence) + * index and throw rather than silently corrupting a reader's replay. + * + * @param array $event + */ + public function append(AiTurnRun $run, array $event): int + { + $sequence = ++$this->sequence; + + AiTurnEvent::create([ + 'ai_turn_run_id' => $run->id, + 'sequence' => $sequence, + 'payload' => $event, + ]); + + return $sequence; + } + + public function finish(AiTurnRun $run, AiTurnRunStatus $status, ?string $error = null): void + { + $run->forceFill([ + 'status' => $status, + 'finished_at' => now(), + 'error_message' => $error, + ])->save(); + } + + /** + * @return Collection + */ + public function eventsAfter(AiTurnRun $run, int $sequence, int $limit = 200): Collection + { + return AiTurnEvent::query() + ->where('ai_turn_run_id', $run->id) + ->where('sequence', '>', $sequence) + ->orderBy('sequence') + ->limit($limit) + ->get(); + } + + public function touchPoll(AiTurnRun $run): void + { + AiTurnRun::query()->whereKey($run->id)->update(['last_polled_at' => now()]); + } + + public function requestCancel(AiTurnRun $run): void + { + AiTurnRun::query()->whereKey($run->id)->update(['cancel_requested_at' => now()]); + + $this->cachedShouldStop = null; + } + + /** + * Whether the turn should stop generating: someone cancelled it, or nobody + * is reading it any more. + */ + public function shouldStop(AiTurnRun $run): bool + { + $now = microtime(true); + + if ($this->cachedShouldStop !== null && $now - $this->shouldStopCheckedAt < $this->stopCheckInterval) { + return $this->cachedShouldStop; + } + + $this->shouldStopCheckedAt = $now; + + return $this->cachedShouldStop = $this->readShouldStop($run); + } + + /** + * Which terminal status a stopped run earned. + */ + public function stopStatusFor(AiTurnRun $run): AiTurnRunStatus + { + return $run->fresh()?->cancel_requested_at !== null + ? AiTurnRunStatus::Cancelled + : AiTurnRunStatus::Abandoned; + } + + private function readShouldStop(AiTurnRun $run): bool + { + $fresh = $run->fresh(); + + if ($fresh === null || $fresh->cancel_requested_at !== null) { + return true; + } + + $seconds = (int) config('code-talker.turns.abandon_after_seconds', 30); + + if ($seconds <= 0) { + return false; + } + + // Measured from created_at while nothing has polled yet: a run + // dispatched a moment ago has no reader by definition, and killing it + // before its reader connects would make the feature unusable. + $since = $fresh->last_polled_at ?? $fresh->created_at; + + return $since !== null && $since->diffInSeconds(now()) > $seconds; + } +} diff --git a/src/Services/ConversationUsageService.php b/src/Services/ConversationUsageService.php index c366b33..59ab3cd 100644 --- a/src/Services/ConversationUsageService.php +++ b/src/Services/ConversationUsageService.php @@ -46,7 +46,12 @@ public function buildUsageSummary(AiConversation $conversation): array { $logs = AiInteractionLog::query() ->where('ai_conversation_id', $conversation->id) - ->where('status', AiInteractionStatus::Success->value) + // Aborted turns count too: the browser hanging up does not refund + // the tokens the provider already generated. + ->whereIn('status', [ + AiInteractionStatus::Success->value, + AiInteractionStatus::Aborted->value, + ]) ->where(function ($query): void { $query->whereNotNull('input_tokens') ->orWhereNotNull('output_tokens'); diff --git a/src/Services/LaravelAi/Concerns/HeartbeatsIdleSseReads.php b/src/Services/LaravelAi/Concerns/HeartbeatsIdleSseReads.php new file mode 100644 index 0000000..60bdde5 --- /dev/null +++ b/src/Services/LaravelAi/Concerns/HeartbeatsIdleSseReads.php @@ -0,0 +1,137 @@ +|Heartbeat> + */ + protected function parseServerSentEvents($streamBody): Generator + { + $seconds = (int) config('code-talker.conversations.heartbeat_seconds', 5); + + // Checked before detaching, because detach() cannot be undone: a body + // with no resource behind it (a PumpStream, a host's custom handler) + // must reach the parent parser with its body intact. + if ($seconds <= 0 || ! is_string($streamBody->getMetadata('stream_type'))) { + yield from parent::parseServerSentEvents($streamBody); + + return; + } + + // Raw-exchange capture tees every byte the parser reads. Detaching + // takes the resource out from under the tee, so the reader below + // feeds it the bytes itself — otherwise enabling heartbeats would + // silently blank ai_provider_exchanges.raw_response. + $tee = $streamBody instanceof TeeingStream ? $streamBody : null; + + $resource = $streamBody->detach(); + + if (! is_resource($resource)) { + return; + } + + try { + yield from $this->readSseWithHeartbeats($resource, $seconds, $tee); + } finally { + // Nothing else holds it once detached. + fclose($resource); + } + } + + /** + * @param resource $resource + * @return Generator|Heartbeat> + */ + private function readSseWithHeartbeats($resource, int $seconds, ?TeeingStream $tee = null): Generator + { + stream_set_timeout($resource, $seconds); + + $buffer = ''; + $emptyReads = 0; + + while (true) { + $byte = fread($resource, 1); + + if ($byte === false || $byte === '') { + // A timed-out read surfaces as false on some platforms and as + // '' on others, so both are checked against timed_out — and + // before feof(), because a socket can report EOF after a read + // timeout. Treating either as the end would turn every silent + // gap into a truncated turn. + if (stream_get_meta_data($resource)['timed_out'] ?? false) { + $emptyReads = 0; + + yield new Heartbeat(strtolower((string) Str::uuid7()), time()); + + continue; + } + + if ($byte === false || feof($resource)) { + return; + } + + if (++$emptyReads >= self::MAX_EMPTY_READS) { + return; + } + + continue; + } + + $emptyReads = 0; + $buffer .= $byte; + $tee?->record($byte); + + if ($byte !== "\n") { + continue; + } + + $line = trim($buffer); + $buffer = ''; + + if ($line === '' || ! str_starts_with($line, 'data:')) { + continue; + } + + $data = trim(substr($line, 5)); + + if ($data === '[DONE]') { + return; + } + + $decoded = json_decode($data, true); + + if (json_last_error() === JSON_ERROR_NONE && $decoded !== null) { + yield $decoded; + } + } + } +} diff --git a/src/Services/LaravelAi/ReasoningOpenAiCompatibleGateway.php b/src/Services/LaravelAi/ReasoningOpenAiCompatibleGateway.php index 11e81cd..c7e2a61 100644 --- a/src/Services/LaravelAi/ReasoningOpenAiCompatibleGateway.php +++ b/src/Services/LaravelAi/ReasoningOpenAiCompatibleGateway.php @@ -3,6 +3,8 @@ namespace Jvjvjv\CodeTalker\Services\LaravelAi; use Generator; +use Jvjvjv\CodeTalker\Services\LaravelAi\Concerns\HeartbeatsIdleSseReads; +use Jvjvjv\CodeTalker\Services\LaravelAi\Streaming\Heartbeat; use Laravel\Ai\Contracts\Providers\TextProvider; use Laravel\Ai\Gateway\OpenAiCompatible\OpenAiCompatibleGateway; use Laravel\Ai\Gateway\StepContext; @@ -39,6 +41,8 @@ */ class ReasoningOpenAiCompatibleGateway extends OpenAiCompatibleGateway { + use HeartbeatsIdleSseReads; + /** * Stream text for a single Chat Completions step. * @@ -103,6 +107,14 @@ protected function processTextStream( $responseModel = $model; foreach ($this->parseServerSentEvents($streamBody) as $data) { + // A tick, not model output: forward it and read on. Everything + // below this line assumes $data is a decoded provider frame. + if ($data instanceof Heartbeat) { + yield $data->withInvocationId($invocationId); + + continue; + } + // OpenAI-shaped errors nest under an "error" key. LM Studio's own // engine-level failures (e.g. "context size exceeded") instead // arrive as a flat frame — {"code", "message", "type"} with no diff --git a/src/Services/LaravelAi/StreamTranslator.php b/src/Services/LaravelAi/StreamTranslator.php index 79d9351..7eba99e 100644 --- a/src/Services/LaravelAi/StreamTranslator.php +++ b/src/Services/LaravelAi/StreamTranslator.php @@ -30,6 +30,12 @@ class StreamTranslator private ?string $lastReason = null; + /** + * Whether a provider request is open — a StreamStart has arrived without + * its StreamEnd. True at the moment a turn is cut off mid-generation. + */ + private bool $streamOpen = false; + private Usage $usage; public function __construct() @@ -71,11 +77,15 @@ public function translate(StreamEvent $event): array */ public function finish(): array { + // Read before the synthetic message_start below, which opens a stream + // that no StreamEnd will ever close. + $stopReason = $this->stopReason(); + $events = $this->messageStarted ? [] : $this->onStreamStart(); $events[] = [ 'type' => 'message_delta', - 'delta' => ['stop_reason' => $this->stopReason()], + 'delta' => ['stop_reason' => $stopReason], 'usage' => [ 'input_tokens' => $this->inputTokens() ?: null, 'output_tokens' => $this->outputTokens() ?: null, @@ -89,9 +99,18 @@ public function finish(): array /** * The last finish reason mapped to the legacy Anthropic-style stop_reason. + * + * A turn that never saw a StreamEnd — the browser hung up, or the duration + * guard cut the generation off — reports 'incomplete'. It used to fall + * through to 'end_turn', which made a truncated turn indistinguishable + * from a clean one everywhere the reason is recorded. */ public function stopReason(): string { + if ($this->streamOpen || $this->lastReason === null) { + return 'incomplete'; + } + return match ($this->lastReason) { 'tool_calls' => 'tool_use', 'length' => 'max_tokens', @@ -122,6 +141,11 @@ public function outputTokens(): int */ private function onStreamStart(): array { + // Set before the message_start guard below: the agentic loop opens one + // provider request per step, and only the first of them emits a + // browser-visible message_start. + $this->streamOpen = true; + if ($this->messageStarted) { return []; } @@ -145,6 +169,7 @@ private function onStreamEnd(StreamEnd $event): array { $this->usage = $this->usage->add($event->usage); $this->lastReason = $event->reason; + $this->streamOpen = false; return []; } diff --git a/src/Services/LaravelAi/Streaming/Heartbeat.php b/src/Services/LaravelAi/Streaming/Heartbeat.php new file mode 100644 index 0000000..5e8ab4a --- /dev/null +++ b/src/Services/LaravelAi/Streaming/Heartbeat.php @@ -0,0 +1,35 @@ + + */ + public function toArray(): array + { + return [ + 'id' => $this->id, + 'invocation_id' => $this->invocationId, + 'type' => 'heartbeat', + 'timestamp' => $this->timestamp, + ]; + } +} diff --git a/src/Services/RawExchange/TeeingStream.php b/src/Services/RawExchange/TeeingStream.php index 5b63482..cb48fb8 100644 --- a/src/Services/RawExchange/TeeingStream.php +++ b/src/Services/RawExchange/TeeingStream.php @@ -28,6 +28,20 @@ public function __construct( ) { } + /** + * Record bytes a consumer read from the underlying resource directly. + * + * The heartbeat SSE reader detaches the resource so it can fread() with a + * stream timeout — those reads never pass through read() above, so the + * reader feeds the tee itself. Flushing still happens on close/destruct. + */ + public function record(string $bytes): void + { + if ($bytes !== '') { + $this->buffer .= $bytes; + } + } + public function read($length): string { $data = $this->stream->read($length); diff --git a/tests/Feature/AiPersonaConversationServiceTest.php b/tests/Feature/AiPersonaConversationServiceTest.php index 1c07f41..dbb56b5 100644 --- a/tests/Feature/AiPersonaConversationServiceTest.php +++ b/tests/Feature/AiPersonaConversationServiceTest.php @@ -7,8 +7,12 @@ use Illuminate\Support\Facades\Queue; use Illuminate\Support\Facades\Schema; use Jvjvjv\CodeTalker\CodeTalkerServiceProvider; +use Jvjvjv\CodeTalker\Enums\AiTurnRunStatus; use Jvjvjv\CodeTalker\Jobs\ProcessAiMemoryJob; +use Jvjvjv\CodeTalker\Jobs\RunConversationTurnJob; use Jvjvjv\CodeTalker\Models\AiPersona; +use Jvjvjv\CodeTalker\Models\AiTurnRun; +use Jvjvjv\CodeTalker\Services\Conversation\TurnRunStore; use Jvjvjv\CodeTalker\Models\AiConversationMessage; use Jvjvjv\CodeTalker\Models\AiInteractionLog; use Jvjvjv\CodeTalker\Models\AiLlmMessage; @@ -20,6 +24,7 @@ use Jvjvjv\CodeTalker\Services\LaravelAi\AgentFactory; use Jvjvjv\CodeTalker\Services\LaravelAi\AiSystemProviderConfigurator; use Jvjvjv\CodeTalker\Services\LaravelAi\CodeTalkerAgent; +use Jvjvjv\CodeTalker\Services\LaravelAi\Streaming\Heartbeat; use Jvjvjv\CodeTalker\Services\RawExchange\RawExchangeContext; use Jvjvjv\CodeTalker\Tests\TestCase; use Closure; @@ -36,7 +41,9 @@ use Laravel\Ai\Responses\Data\Usage; use Laravel\Ai\Streaming\Events\Error; use Laravel\Ai\Streaming\Events\ReasoningDelta; +use Laravel\Ai\Streaming\Events\StreamEnd; use Laravel\Ai\Streaming\Events\StreamStart; +use Laravel\Ai\Streaming\Events\TextDelta; use RuntimeException; class AiPersonaConversationServiceTest extends TestCase @@ -128,6 +135,14 @@ private function drainAndDecode(iterable $stream, array &$rawLines = []): array foreach ((new SseFrameEncoder())->encode($stream) as $line) { $rawLines[] = $line; + // A heartbeat travels as an SSE comment, not a data frame. Map it + // back to its structured form so assertions can still count it. + if (str_starts_with($line, ':')) { + $events[] = ['type' => 'heartbeat']; + + continue; + } + $payload = trim(str_replace('data: ', '', $line)); if ($payload === '[DONE]') { @@ -517,8 +532,14 @@ protected function streamElapsedSeconds(float $startedAt): float $this->assertSame(1, AiLlmMessage::where('direction', 'response')->count()); // Nothing streamed before the guard tripped on the very first event, - // so there is no content to preserve. - $this->assertNull(AiConversationMessage::where('role', 'assistant')->first()); + // so there is no content to preserve — but the turn is still recorded, + // flagged as interrupted, rather than leaving the user's message with + // nothing beneath it. + $message = AiConversationMessage::where('role', 'assistant')->first(); + $this->assertNotNull($message); + $this->assertSame('', $message->content); + $this->assertTrue($message->metadata['incomplete']); + $this->assertSame('max_stream_duration', $message->metadata['incomplete_reason']); } public function test_the_max_duration_guard_preserves_partial_reasoning_content(): void @@ -653,50 +674,142 @@ protected function streamElapsedSeconds(float $startedAt): float $this->assertGreaterThan(1, count(array_unique($service->seenStartedAt))); } - public function test_a_client_abort_stops_the_turn_and_persists_the_partial_response(): void + /** + * A service whose cancellation check fires once the given number of stream + * events has been consumed — the browser hanging up mid-turn, made + * deterministic. The guard is consulted at the top of each iteration, so + * `$events` events are processed before the loop breaks. + */ + private function abortingAfter(int $events): AiPersonaConversationService + { + $checks = 0; + + return $this->app->make(AiPersonaConversationService::class) + ->usingCancellationCheck(static function () use (&$checks, $events): bool { + return $checks++ >= $events; + }); + } + + public function test_a_client_abort_before_any_content_still_records_an_interrupted_message(): void { Queue::fake(); CodeTalkerAgent::fake(['This turn is cancelled by the browser mid-stream']); $persona = $this->makePersona(); - // Simulate the browser hanging up (Cancel button / ESC): the abort guard - // trips after the first stream event so a partial response is captured. - $service = new class( - $this->app->make(AgentFactory::class), - $this->app->make(AiMemoryService::class), - $this->app->make(ConversationUsageService::class), - $this->app->make(RawExchangeContext::class), - $this->app->make(AiSystemProviderConfigurator::class), - ) extends AiPersonaConversationService { - private int $checks = 0; - - protected function clientAborted(): bool - { - return $this->checks++ > 0; - } - }; + // Abort with only the StreamStart consumed: the model was still + // processing the prompt and had emitted nothing. This used to persist + // nothing at all, so the user's message sat in the transcript with no + // reply beneath it and no record that a turn had ever run. + $service = $this->abortingAfter(1); $conversation = $service->startConversation($persona); $events = $this->drainAndDecode($service->continueConversation($conversation, 'Hi')); - // A client abort is a clean stop, not a failure: no error event is emitted. + // A client abort is a clean stop, not a provider failure: no error + // event is emitted (and by then the browser is gone anyway). $this->assertNull(collect($events)->firstWhere('type', 'error')); - // The turn is still recorded (request + partial response), and the - // interaction log is not marked as an error. + // Only one attempt runs — the abort short-circuits any continuation loop. + $this->assertSame(1, AiLlmMessage::where('direction', 'request')->count()); $this->assertSame(1, AiLlmMessage::where('direction', 'response')->count()); + // The turn never finished, so it is not reported as one that did. + $response = AiLlmMessage::where('direction', 'response')->first(); + $this->assertSame('incomplete', $response->response_data['stop_reason']); + $log = AiInteractionLog::first(); $this->assertNotNull($log); - $this->assertSame('success', $log->status->value); + $this->assertSame('aborted', $log->status->value); + $this->assertSame('client_aborted', $log->provider_metadata['error_reason']); - // Only one attempt runs — the abort short-circuits any continuation loop. - $this->assertSame(1, AiLlmMessage::where('direction', 'request')->count()); + // The interrupted reply is visible in the transcript, so the host can + // render "this reply was interrupted" instead of silence. + $message = AiConversationMessage::where('role', 'assistant')->first(); + $this->assertNotNull($message); + $this->assertSame('', $message->content); + $this->assertTrue($message->metadata['incomplete']); + $this->assertSame('client_aborted', $message->metadata['incomplete_reason']); Queue::assertNotPushed(ProcessAiMemoryJob::class); } + public function test_a_client_abort_after_a_tool_call_persists_the_tool_call(): void + { + Queue::fake(); + Http::fake([ + 'https://example.com/page' => Http::response( + 'Hi

Body text.

', + 200, + ['Content-Type' => 'text/html; charset=UTF-8'], + ), + ]); + + CodeTalkerAgent::fake([ + new ToolCall('tool-1', 'fetch-web-page', ['url' => 'https://example.com/page']), + 'Summary after reading the page', + ]); + + $persona = $this->makePersona( + ['allowed_tools' => ['fetch-web-page']], + ['tools_enabled' => true], + ); + + // Abort with StreamStart and the ToolCall consumed. The tool may well + // have run and changed state on the host's side; dropping the turn + // would leave the next turn's history with no record the call was ever + // made, and the model free to contradict itself about it. + $service = $this->abortingAfter(2); + + $conversation = $service->startConversation($persona); + $this->drainAndDecode($service->continueConversation($conversation, 'Read the page')); + + $message = AiConversationMessage::where('role', 'assistant')->first(); + $this->assertNotNull($message); + $this->assertSame('', $message->content); + $this->assertSame('fetch-web-page', $message->tool_calls[0]['name']); + $this->assertTrue($message->metadata['incomplete']); + + $this->assertSame('aborted', AiInteractionLog::first()->status->value); + } + + public function test_a_client_abort_after_a_text_delta_persists_the_partial_text(): void + { + Queue::fake(); + CodeTalkerAgent::fake(['This turn is cancelled by the browser mid-stream']); + + $persona = $this->makePersona(); + + // StreamStart, TextStart, then the first TextDelta. + $service = $this->abortingAfter(3); + + $conversation = $service->startConversation($persona); + $this->drainAndDecode($service->continueConversation($conversation, 'Hi')); + + $message = AiConversationMessage::where('role', 'assistant')->first(); + $this->assertNotNull($message); + $this->assertSame('This', $message->content); + $this->assertTrue($message->metadata['incomplete']); + $this->assertSame('client_aborted', $message->metadata['incomplete_reason']); + } + + public function test_a_completed_turn_is_not_flagged_as_interrupted(): void + { + Queue::fake(); + CodeTalkerAgent::fake(['All done here']); + + $persona = $this->makePersona(); + $service = $this->app->make(AiPersonaConversationService::class); + + $conversation = $service->startConversation($persona); + $this->drainAndDecode($service->continueConversation($conversation, 'Hi')); + + $message = AiConversationMessage::where('role', 'assistant')->first(); + $this->assertSame('All done here', $message->content); + $this->assertFalse($message->metadata['incomplete']); + $this->assertSame('success', AiInteractionLog::first()->status->value); + } + public function test_a_non_recoverable_provider_error_event_fails_the_turn_instead_of_logging_success(): void { Queue::fake(); @@ -781,4 +894,165 @@ public function test_provider_failures_emit_the_legacy_error_event_and_log_the_f Queue::assertNotPushed(ProcessAiMemoryJob::class); } + + /** + * Install a gateway that emits heartbeats between its text deltas — a + * model that is slow to produce tokens rather than one that has stopped. + */ + private function fakeHeartbeatingGateway(int $beats): void + { + CodeTalkerAgent::fake([]); + + $gateway = new class([], $beats) extends FakeTextGateway { + public function __construct(array $responses, private int $beats) + { + parent::__construct($responses); + } + + public function generateStreamStep( + string $invocationId, + TextProvider $provider, + string $model, + ?string $instructions, + array $messages, + array $tools, + ?array $schema, + ?TextGenerationOptions $options, + ?int $timeout, + StepContext $stepContext, + ): Generator { + yield (new StreamStart(uniqid('', true), $provider->name(), $model, time())) + ->withInvocationId($invocationId); + + for ($i = 0; $i < $this->beats; $i++) { + yield (new Heartbeat(uniqid('', true), time()))->withInvocationId($invocationId); + } + + yield (new TextDelta(uniqid('', true), 'm1', 'Done', time())) + ->withInvocationId($invocationId); + + yield (new StreamEnd(uniqid('', true), 'stop', new Usage(), time())) + ->withInvocationId($invocationId); + + return new StepResponse( + 'Done', [], FinishReason::Stop, new Usage(), new Meta($provider->name(), $model), + ); + } + }; + + $manager = $this->app->make(AiManager::class); + (Closure::bind(function () use ($gateway): void { + $this->fakeAgentGateways[CodeTalkerAgent::class] = $gateway; + }, $manager, $manager::class))(); + } + + public function test_a_heartbeat_reaches_the_browser_but_never_the_stored_events(): void + { + Queue::fake(); + $this->fakeHeartbeatingGateway(beats: 3); + + $persona = $this->makePersona(); + $service = $this->app->make(AiPersonaConversationService::class); + + $conversation = $service->startConversation($persona); + $events = $this->drainAndDecode($service->continueConversation($conversation, 'Hi')); + + $this->assertSame(3, count(array_filter($events, fn ($e) => ($e['type'] ?? null) === 'heartbeat'))); + + // The stored event log is a record of what the model did, not of how + // long it took to do it. + $logged = AiLlmMessage::where('direction', 'response')->first()->response_data['events']; + $this->assertNotContains('heartbeat', array_column($logged, 'type')); + + // The answer itself is unaffected. + $this->assertSame('Done', AiConversationMessage::where('role', 'assistant')->first()->content); + } + + public function test_the_max_duration_guard_trips_on_a_heartbeat_with_no_provider_event(): void + { + Queue::fake(); + config()->set('code-talker.conversations.max_stream_seconds', 60); + $this->fakeHeartbeatingGateway(beats: 3); + + $persona = $this->makePersona(); + + // Elapsed time only goes over budget after the StreamStart, so the + // guard has nothing but heartbeats to trip on. + $service = new class( + $this->app->make(AgentFactory::class), + $this->app->make(AiMemoryService::class), + $this->app->make(ConversationUsageService::class), + $this->app->make(RawExchangeContext::class), + $this->app->make(AiSystemProviderConfigurator::class), + ) extends AiPersonaConversationService { + private int $calls = 0; + + protected function streamElapsedSeconds(float $startedAt): float + { + return ++$this->calls > 1 ? 9999.0 : 0.0; + } + }; + + $conversation = $service->startConversation($persona); + $events = $this->drainAndDecode($service->continueConversation($conversation, 'Hi')); + + $error = collect($events)->firstWhere('type', 'error'); + $this->assertNotNull($error); + $this->assertSame('max_stream_duration', $error['reason']); + } + + public function test_dispatching_a_turn_queues_a_job_against_a_new_run(): void + { + Queue::fake(); + + $persona = $this->makePersona(); + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($persona); + + $run = $service->dispatchTurn($conversation, 'Hi there'); + + $this->assertSame(AiTurnRunStatus::Queued, $run->status); + $this->assertSame('Hi there', $run->prompt); + $this->assertNotEmpty($run->public_id); + + Queue::assertPushed( + RunConversationTurnJob::class, + fn (RunConversationTurnJob $job): bool => $job->turnRunId === $run->id, + ); + } + + public function test_resuming_a_turn_streams_its_stored_events(): void + { + Queue::fake(); + config()->set('code-talker.turns.poll_interval_ms', 1); + + $persona = $this->makePersona(); + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($persona); + + $run = $service->dispatchTurn($conversation, 'Hi'); + + $store = $this->app->make(TurnRunStore::class); + $store->markRunning($run); + $store->append($run, ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi']]); + $store->finish($run, AiTurnRunStatus::Completed); + + $events = iterator_to_array($service->resumeTurn($run), false); + + $this->assertSame(['content_block_delta'], array_column($events, 'type')); + } + + public function test_cancelling_a_turn_marks_it_for_the_worker(): void + { + Queue::fake(); + + $persona = $this->makePersona(); + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($persona); + + $run = $service->dispatchTurn($conversation, 'Hi'); + $service->cancelTurn($run); + + $this->assertNotNull($run->fresh()->cancel_requested_at); + } } diff --git a/tests/Feature/AiTurnRunModelTest.php b/tests/Feature/AiTurnRunModelTest.php new file mode 100644 index 0000000..1b515be --- /dev/null +++ b/tests/Feature/AiTurnRunModelTest.php @@ -0,0 +1,108 @@ +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + } + + private function conversation(): AiConversation + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + return AiConversation::create([ + 'ai_system_id' => $system->id, + 'feature' => 'persona:test', + ]); + } + + public function test_a_run_gets_a_public_id_and_casts_its_status(): void + { + $run = AiTurnRun::create([ + 'ai_conversation_id' => $this->conversation()->id, + 'status' => AiTurnRunStatus::Queued, + 'prompt' => 'Hello', + ]); + + $this->assertNotEmpty($run->public_id); + $this->assertSame(AiTurnRunStatus::Queued, $run->fresh()->status); + $this->assertFalse($run->status->isTerminal()); + } + + public function test_terminal_statuses_are_the_ones_a_reader_stops_on(): void + { + $this->assertTrue(AiTurnRunStatus::Completed->isTerminal()); + $this->assertTrue(AiTurnRunStatus::Failed->isTerminal()); + $this->assertTrue(AiTurnRunStatus::Cancelled->isTerminal()); + $this->assertTrue(AiTurnRunStatus::Abandoned->isTerminal()); + $this->assertFalse(AiTurnRunStatus::Queued->isTerminal()); + $this->assertFalse(AiTurnRunStatus::Running->isTerminal()); + } + + public function test_events_belong_to_a_run_and_keep_their_payload_shape(): void + { + $run = AiTurnRun::create([ + 'ai_conversation_id' => $this->conversation()->id, + 'status' => AiTurnRunStatus::Running, + 'prompt' => 'Hello', + ]); + + AiTurnEvent::create([ + 'ai_turn_run_id' => $run->id, + 'sequence' => 1, + 'payload' => ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi']], + ]); + + $event = $run->events()->first(); + + $this->assertSame(1, $event->sequence); + $this->assertSame('Hi', $event->payload['delta']['text']); + } + + public function test_a_run_cannot_reuse_a_sequence(): void + { + $run = AiTurnRun::create([ + 'ai_conversation_id' => $this->conversation()->id, + 'status' => AiTurnRunStatus::Running, + 'prompt' => 'Hello', + ]); + + AiTurnEvent::create(['ai_turn_run_id' => $run->id, 'sequence' => 1, 'payload' => ['type' => 'a']]); + + $this->expectException(\Illuminate\Database\QueryException::class); + + AiTurnEvent::create(['ai_turn_run_id' => $run->id, 'sequence' => 1, 'payload' => ['type' => 'b']]); + } +} diff --git a/tests/Feature/ChatTurnLibraryTest.php b/tests/Feature/ChatTurnLibraryTest.php index 5f8e24b..4cd7320 100644 --- a/tests/Feature/ChatTurnLibraryTest.php +++ b/tests/Feature/ChatTurnLibraryTest.php @@ -98,6 +98,22 @@ public function test_an_empty_turn_still_terminates(): void ); } + public function test_a_heartbeat_is_encoded_as_a_comment_frame(): void + { + $frames = iterator_to_array((new SseFrameEncoder())->encode([ + ['type' => 'heartbeat'], + ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi']], + ]), false); + + // A comment frame: every SSE consumer ignores it, including the + // published client, which only reads lines beginning with "data:". + $this->assertSame(": ping\n\n", $frames[0]); + $this->assertStringStartsWith('data: {', $frames[1]); + + // A heartbeat is not an error, so the turn still terminates normally. + $this->assertSame("data: [DONE]\n\n", $frames[2]); + } + // ------------------------------------------------------------ access rules public function test_an_inactive_bot_cannot_open_a_conversation(): void diff --git a/tests/Feature/ConversationUsageServiceTest.php b/tests/Feature/ConversationUsageServiceTest.php new file mode 100644 index 0000000..48a2d8c --- /dev/null +++ b/tests/Feature/ConversationUsageServiceTest.php @@ -0,0 +1,83 @@ +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + // AiConversation::booted() assigns a uuid, but no package migration + // creates the column (host apps add it themselves). + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + } + + private function conversation(): AiConversation + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + return AiConversation::create([ + 'ai_system_id' => $system->id, + 'feature' => 'chat-bot:test', + ]); + } + + private function log(AiConversation $conversation, AiInteractionStatus $status, int $in, int $out): void + { + AiInteractionLog::create([ + 'ai_system_id' => $conversation->ai_system_id, + 'ai_conversation_id' => $conversation->id, + 'feature' => $conversation->feature, + 'model' => 'claude-sonnet-4-6', + 'input_tokens' => $in, + 'output_tokens' => $out, + 'status' => $status, + ]); + } + + public function test_aborted_turns_still_count_towards_conversation_usage(): void + { + $conversation = $this->conversation(); + + $this->log($conversation, AiInteractionStatus::Success, 100, 20); + // The browser hanging up mid-stream does not refund the tokens the + // provider had already generated, so an aborted turn is still billed. + $this->log($conversation, AiInteractionStatus::Aborted, 300, 40); + // A turn that never reached the provider is not. + $this->log($conversation, AiInteractionStatus::Error, 999, 999); + + $usage = $this->app->make(ConversationUsageService::class)->buildUsageSummary($conversation); + + $this->assertSame(400, $usage['input_tokens']); + $this->assertSame(60, $usage['output_tokens']); + $this->assertSame(460, $usage['total_tokens']); + } +} diff --git a/tests/Feature/PruneTurnEventsCommandTest.php b/tests/Feature/PruneTurnEventsCommandTest.php new file mode 100644 index 0000000..4abe1ff --- /dev/null +++ b/tests/Feature/PruneTurnEventsCommandTest.php @@ -0,0 +1,93 @@ +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + } + + private function makeRun(AiTurnRunStatus $status, int $daysOld): AiTurnRun + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + $conversation = AiConversation::create([ + 'ai_system_id' => $system->id, + 'feature' => 'persona:test', + ]); + + $run = AiTurnRun::create([ + 'ai_conversation_id' => $conversation->id, + 'status' => $status, + 'prompt' => 'Hi', + ]); + + $run->forceFill(['created_at' => now()->subDays($daysOld)])->save(); + + AiTurnEvent::create(['ai_turn_run_id' => $run->id, 'sequence' => 1, 'payload' => ['type' => 'a']]); + + return $run; + } + + public function test_it_removes_old_terminal_runs_and_their_events(): void + { + config()->set('code-talker.turns.retention_days', 7); + + $old = $this->makeRun(AiTurnRunStatus::Completed, daysOld: 10); + $recent = $this->makeRun(AiTurnRunStatus::Completed, daysOld: 1); + $live = $this->makeRun(AiTurnRunStatus::Running, daysOld: 10); + + $this->artisan('ai:prune-turn-events')->assertExitCode(0); + + $this->assertNull(AiTurnRun::find($old->id)); + $this->assertSame(0, AiTurnEvent::where('ai_turn_run_id', $old->id)->count()); + + $this->assertNotNull(AiTurnRun::find($recent->id)); + + // A long-running turn is not garbage, however old the row is. + $this->assertNotNull(AiTurnRun::find($live->id)); + } + + public function test_zero_retention_days_disables_pruning_instead_of_deleting_everything(): void + { + config()->set('code-talker.turns.retention_days', 0); + + $old = $this->makeRun(AiTurnRunStatus::Completed, daysOld: 365); + + $this->artisan('ai:prune-turn-events')->assertExitCode(0); + + $this->assertNotNull(AiTurnRun::find($old->id)); + $this->assertSame(1, AiTurnEvent::where('ai_turn_run_id', $old->id)->count()); + } +} diff --git a/tests/Feature/ReasoningOpenAiCompatibleGatewayTest.php b/tests/Feature/ReasoningOpenAiCompatibleGatewayTest.php index c79993a..b7095da 100644 --- a/tests/Feature/ReasoningOpenAiCompatibleGatewayTest.php +++ b/tests/Feature/ReasoningOpenAiCompatibleGatewayTest.php @@ -6,6 +6,7 @@ use Illuminate\Contracts\Events\Dispatcher; use Jvjvjv\CodeTalker\Services\LaravelAi\ReasoningOpenAiCompatibleGateway; use Jvjvjv\CodeTalker\Services\LaravelAi\ReasoningOpenAiCompatibleProvider; +use Jvjvjv\CodeTalker\Services\LaravelAi\Streaming\Heartbeat; use Jvjvjv\CodeTalker\Tests\TestCase; use Laravel\Ai\AiManager; use Laravel\Ai\Streaming\Events\Error as ErrorEvent; @@ -136,4 +137,90 @@ public function test_the_openai_compatible_driver_resolves_to_the_reasoning_prov $this->assertInstanceOf(ReasoningOpenAiCompatibleProvider::class, $provider); } + + /** + * A gateway exposing the protected SSE parser, so a test can drive the + * generator one step at a time. Stepping matters: the generator suspends on + * each yield, which is what lets a single-threaded test write the second + * half of a frame *after* observing the heartbeat for the gap. + */ + private function parsingGateway(): object + { + return new class($this->app->make(Dispatcher::class)) extends ReasoningOpenAiCompatibleGateway + { + public function parse($body): \Generator + { + return $this->parseServerSentEvents($body); + } + }; + } + + public function test_an_idle_gap_yields_a_heartbeat_without_losing_the_frame_that_spans_it(): void + { + config()->set('code-talker.conversations.heartbeat_seconds', 1); + + [$readEnd, $writeEnd] = stream_socket_pair(STREAM_PF_UNIX, STREAM_SOCK_STREAM, 0); + + // Half a frame, then silence — exactly the shape that used to be lost. + fwrite($writeEnd, 'data: {"choices":[{"delta":{"con'); + + $events = $this->parsingGateway()->parse(Utils::streamFor($readEnd)); + + // Runs until the first yield: the read times out and reports a beat. + $events->rewind(); + $this->assertInstanceOf(Heartbeat::class, $events->current()); + $this->assertSame('heartbeat', $events->current()->toArray()['type']); + + // The rest of the frame arrives after the gap and must parse intact. + fwrite($writeEnd, 'tent":"Hello"}}]}' . "\n\n"); + + $events->next(); + $this->assertSame( + [['delta' => ['content' => 'Hello']]], + $events->current()['choices'], + ); + + fclose($writeEnd); + $events->next(); + $this->assertFalse($events->valid()); + } + + public function test_heartbeats_are_disabled_by_a_zero_interval(): void + { + config()->set('code-talker.conversations.heartbeat_seconds', 0); + + $sse = 'data: {"choices":[{"delta":{"content":"Hi"}}]}' . "\n\n" + . 'data: [DONE]' . "\n\n"; + + $parsed = iterator_to_array($this->parsingGateway()->parse(Utils::streamFor($sse)), false); + + $this->assertCount(1, $parsed); + $this->assertSame('Hi', $parsed[0]['choices'][0]['delta']['content']); + } + + public function test_a_body_without_a_stream_resource_falls_back_to_the_parent_parser(): void + { + config()->set('code-talker.conversations.heartbeat_seconds', 1); + + // A PumpStream has no underlying resource, so detaching it would leave + // nothing to read; the parser must delegate instead. The pump returns + // null once drained — PumpStream's contract for EOF; an empty string + // would never flip eof() and the parent parser would spin forever. + $body = new \GuzzleHttp\Psr7\PumpStream(function (): ?string { + static $sent = false; + + if ($sent) { + return null; + } + + $sent = true; + + return 'data: {"choices":[{"delta":{"content":"Hi"}}]}' . "\n\n"; + }); + + $parsed = iterator_to_array($this->parsingGateway()->parse($body), false); + + $this->assertCount(1, $parsed); + $this->assertSame('Hi', $parsed[0]['choices'][0]['delta']['content']); + } } diff --git a/tests/Feature/RunConversationTurnJobTest.php b/tests/Feature/RunConversationTurnJobTest.php new file mode 100644 index 0000000..af6bd5e --- /dev/null +++ b/tests/Feature/RunConversationTurnJobTest.php @@ -0,0 +1,263 @@ +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + } + + private function persona(): AiPersona + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + return AiPersona::create([ + 'ai_system_id' => $system->id, + 'name' => 'Test Bot', + 'slug' => 'test-bot', + 'prompt_template' => 'You are {{persona_name}}.', + 'is_active' => true, + ]); + } + + public function test_the_job_records_every_event_and_completes_the_run(): void + { + Queue::fake(); + CodeTalkerAgent::fake(['Hello there']); + + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($this->persona()); + + $run = $this->app->make(TurnRunStore::class)->open($conversation, 'Hi'); + + $this->app->make(RunConversationTurnJob::class, ['turnRunId' => $run->id]) + ->handle($service, $this->app->make(TurnRunStore::class)); + + $run->refresh(); + $this->assertSame(AiTurnRunStatus::Completed, $run->status); + $this->assertNotNull($run->finished_at); + + $types = $run->events()->get()->pluck('payload.type')->all(); + $this->assertContains('content_block_delta', $types); + $this->assertContains('message_stop', $types); + + // Sequences are contiguous from 1, which is what a resuming reader + // relies on to know it missed nothing. + $this->assertSame(range(1, $run->events()->count()), $run->events()->pluck('sequence')->all()); + + // The turn itself behaved exactly as the synchronous path does. + $this->assertSame('Hello there', AiConversationMessage::where('role', 'assistant')->first()->content); + } + + public function test_a_cancelled_run_stops_and_is_marked_cancelled(): void + { + Queue::fake(); + CodeTalkerAgent::fake(['This answer is cancelled part way through']); + + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($this->persona()); + + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($conversation, 'Hi'); + $store->requestCancel($run); + + $this->app->make(RunConversationTurnJob::class, ['turnRunId' => $run->id]) + ->handle($service, $this->app->make(TurnRunStore::class)); + + $this->assertSame(AiTurnRunStatus::Cancelled, $run->fresh()->status); + + // 0.15.0's recorder keeps whatever the turn produced, flagged. + $message = AiConversationMessage::where('role', 'assistant')->first(); + $this->assertNotNull($message); + $this->assertTrue($message->metadata['incomplete']); + } + + /** + * Enable faked mode, then swap in the given gateway — the same pattern + * AiPersonaConversationServiceTest uses for its stream-shaped fakes. + */ + private function installGateway(FakeTextGateway $gateway): void + { + CodeTalkerAgent::fake([]); + + $manager = $this->app->make(AiManager::class); + (Closure::bind(function () use ($gateway): void { + $this->fakeAgentGateways[CodeTalkerAgent::class] = $gateway; + }, $manager, $manager::class))(); + } + + public function test_heartbeats_are_consumed_and_never_stored_as_turn_events(): void + { + Queue::fake(); + + // A gateway that beats between its text deltas — a model slow to + // produce tokens rather than one that has stopped. + $this->installGateway(new class([]) extends FakeTextGateway { + public function generateStreamStep( + string $invocationId, + TextProvider $provider, + string $model, + ?string $instructions, + array $messages, + array $tools, + ?array $schema, + ?TextGenerationOptions $options, + ?int $timeout, + StepContext $stepContext, + ): Generator { + yield (new StreamStart(uniqid('', true), $provider->name(), $model, time())) + ->withInvocationId($invocationId); + + yield (new TextDelta(uniqid('', true), 'm1', 'Hel', time())) + ->withInvocationId($invocationId); + + for ($i = 0; $i < 3; $i++) { + yield (new Heartbeat(uniqid('', true), time()))->withInvocationId($invocationId); + } + + yield (new TextDelta(uniqid('', true), 'm1', 'lo', time())) + ->withInvocationId($invocationId); + + yield (new StreamEnd(uniqid('', true), 'stop', new Usage(), time())) + ->withInvocationId($invocationId); + + return new StepResponse( + 'Hello', [], FinishReason::Stop, new Usage(), new Meta($provider->name(), $model), + ); + } + }); + + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($this->persona()); + $run = $this->app->make(TurnRunStore::class)->open($conversation, 'Hi'); + + $this->app->make(RunConversationTurnJob::class, ['turnRunId' => $run->id]) + ->handle($service, $this->app->make(TurnRunStore::class)); + + $run->refresh(); + $this->assertSame(AiTurnRunStatus::Completed, $run->status); + + // No beat reached the store — SseFrameEncoder never transmits one, so + // a stored beat would burn a sequence a reconnecting reader never saw. + $types = $run->events()->get()->pluck('payload.type'); + $this->assertNotContains('heartbeat', $types->all()); + $this->assertContains('content_block_delta', $types->all()); + + // And the text events hold contiguous sequences with no gaps. + $this->assertSame(range(1, $run->events()->count()), $run->events()->pluck('sequence')->all()); + } + + public function test_an_in_stream_provider_error_finishes_the_run_failed(): void + { + Queue::fake(); + + // Mirrors LM Studio returning HTTP 200 then a non-recoverable SSE + // "event: error" — continueConversation() converts it into a terminal + // error event rather than throwing, so the generator ends normally. + $this->installGateway(new class([]) extends FakeTextGateway { + public function generateStreamStep( + string $invocationId, + TextProvider $provider, + string $model, + ?string $instructions, + array $messages, + array $tools, + ?array $schema, + ?TextGenerationOptions $options, + ?int $timeout, + StepContext $stepContext, + ): Generator { + yield (new StreamStart(uniqid('', true), $provider->name(), $model, time())) + ->withInvocationId($invocationId); + + yield (new Error(uniqid('', true), 'unknown_error', 'Context size has been exceeded.', false, time())) + ->withInvocationId($invocationId); + + return new StepResponse('', [], FinishReason::Stop, new Usage(), new Meta($provider->name(), $model)); + } + }); + + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($this->persona()); + $run = $this->app->make(TurnRunStore::class)->open($conversation, 'Hi'); + + $this->app->make(RunConversationTurnJob::class, ['turnRunId' => $run->id]) + ->handle($service, $this->app->make(TurnRunStore::class)); + + $run->refresh(); + $this->assertSame(AiTurnRunStatus::Failed, $run->status); + $this->assertStringContainsString('Context size has been exceeded', $run->error_message); + + // The terminal error event itself is stored, so a reader replaying the + // run sees the same failure the run's status reports. + $this->assertContains('error', $run->events()->get()->pluck('payload.type')->all()); + } + + public function test_a_failed_job_marks_the_run_failed_so_a_reader_stops_waiting(): void + { + Queue::fake(); + + $service = $this->app->make(AiPersonaConversationService::class); + $conversation = $service->startConversation($this->persona()); + $run = $this->app->make(TurnRunStore::class)->open($conversation, 'Hi'); + + $this->app->make(RunConversationTurnJob::class, ['turnRunId' => $run->id]) + ->failed(new \RuntimeException('worker died')); + + $run->refresh(); + $this->assertSame(AiTurnRunStatus::Failed, $run->status); + $this->assertSame('worker died', $run->error_message); + } +} diff --git a/tests/Feature/StreamTranslatorTest.php b/tests/Feature/StreamTranslatorTest.php index 80d4480..4bc636b 100644 --- a/tests/Feature/StreamTranslatorTest.php +++ b/tests/Feature/StreamTranslatorTest.php @@ -85,6 +85,38 @@ public function test_reason_mapping_covers_tool_calls_and_length(): void $this->assertSame('length', $lengthTranslator->lastReason()); } + public function test_stop_reason_is_incomplete_when_no_stream_end_was_ever_seen(): void + { + $translator = new StreamTranslator(); + + // A turn cut off before the provider finished — the browser hung up, + // or the duration guard tripped. Reporting 'end_turn' here made a + // truncated turn indistinguishable from a clean one in the logs. + $this->assertSame('incomplete', $translator->stopReason()); + + $translator->translate($this->streamStart()); + $translator->translate(new TextDelta('e1', 'm1', 'I', time())); + + $this->assertSame('incomplete', $translator->stopReason()); + } + + public function test_stop_reason_is_incomplete_while_a_later_step_is_still_open(): void + { + $translator = new StreamTranslator(); + + $translator->translate($this->streamStart()); + $translator->translate($this->streamEnd('tool_calls')); + + $this->assertSame('tool_use', $translator->stopReason()); + + // The agentic loop opened another provider request that never ended: + // the turn as a whole did not complete, whatever the last finished + // step reported. + $translator->translate($this->streamStart()); + + $this->assertSame('incomplete', $translator->stopReason()); + } + public function test_finish_without_any_stream_start_still_emits_message_start_first(): void { $translator = new StreamTranslator(); diff --git a/tests/Feature/TurnEventStreamTest.php b/tests/Feature/TurnEventStreamTest.php new file mode 100644 index 0000000..cde599d --- /dev/null +++ b/tests/Feature/TurnEventStreamTest.php @@ -0,0 +1,232 @@ +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + + config()->set('code-talker.turns.poll_interval_ms', 1); + } + + private function conversation(): AiConversation + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + return AiConversation::create([ + 'ai_system_id' => $system->id, + 'feature' => 'persona:test', + ]); + } + + public function test_a_finished_run_replays_from_the_beginning(): void + { + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($this->conversation(), 'Hi'); + $store->markRunning($run); + $store->append($run, ['type' => 'message_start']); + $store->append($run, ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi']]); + $store->finish($run, AiTurnRunStatus::Completed); + + $events = iterator_to_array($this->app->make(TurnEventStream::class)->stream($run, 0), false); + + $this->assertSame(['message_start', 'content_block_delta'], array_column($events, 'type')); + $this->assertSame([1, 2], array_column($events, '_seq')); + } + + public function test_a_reload_resumes_from_the_last_sequence_it_saw(): void + { + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($this->conversation(), 'Hi'); + $store->markRunning($run); + $store->append($run, ['type' => 'a']); + $store->append($run, ['type' => 'b']); + $store->append($run, ['type' => 'c']); + $store->finish($run, AiTurnRunStatus::Completed); + + $events = iterator_to_array($this->app->make(TurnEventStream::class)->stream($run, 1), false); + + $this->assertSame(['b', 'c'], array_column($events, 'type')); + } + + public function test_the_final_event_survives_a_run_finishing_mid_poll(): void + { + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($this->conversation(), 'Hi'); + $store->markRunning($run); + $store->append($run, ['type' => 'first']); + + $events = $this->app->make(TurnEventStream::class)->stream($run, 0); + + $events->rewind(); + $this->assertSame('first', $events->current()['type']); + + // The job's last act: append, then mark finished. A reader that read + // status before events would drop 'last' entirely. + $store->append($run, ['type' => 'last']); + $store->finish($run, AiTurnRunStatus::Completed); + + $events->next(); + $this->assertSame('last', $events->current()['type']); + + $events->next(); + $this->assertFalse($events->valid()); + } + + public function test_reading_marks_the_run_as_polled_so_it_is_not_abandoned(): void + { + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($this->conversation(), 'Hi'); + $store->markRunning($run); + $store->append($run, ['type' => 'a']); + $store->finish($run, AiTurnRunStatus::Completed); + + iterator_to_array($this->app->make(TurnEventStream::class)->stream($run, 0), false); + + $this->assertNotNull($run->fresh()->last_polled_at); + } + + /** + * A single-threaded test cannot interleave an append between the reader's + * empty read and its status check, so the race the drain exists for is + * pinned with a scripted store instead: empty on the first read, rows on + * the next, against a run that is already terminal. Remove the drain and + * this fails — the reader would return on the empty read and never see + * the rows the job appended just before finishing. + */ + public function test_the_terminal_drain_reads_events_that_landed_after_the_empty_read(): void + { + $run = $this->terminalRun(); + + $stub = $this->scriptedStore([ + new Collection(), + new Collection([ + $this->event(1, 'mid-a'), + $this->event(2, 'mid-b'), + ]), + ]); + + $events = iterator_to_array((new TurnEventStream($stub))->stream($run, 0), false); + + $this->assertSame(['mid-a', 'mid-b'], array_column($events, 'type')); + $this->assertSame([1, 2], array_column($events, '_seq')); + } + + /** + * eventsAfter() caps each read at 200 rows. The drain must page until a + * read comes back empty; a drain that reads once would truncate a >200 + * backlog and the encoder would still append [DONE] — a cleanly finished + * turn silently missing the end of its answer. + */ + public function test_the_terminal_drain_pages_through_a_backlog_larger_than_one_read(): void + { + $run = $this->terminalRun(); + + $fullPage = new Collection(); + foreach (range(1, 200) as $sequence) { + $fullPage->push($this->event($sequence, 'bulk')); + } + + $stub = $this->scriptedStore([ + new Collection(), + $fullPage, + new Collection([ + $this->event(201, 'tail-a'), + $this->event(202, 'tail-b'), + ]), + ]); + + $events = iterator_to_array((new TurnEventStream($stub))->stream($run, 0), false); + + $this->assertCount(202, $events); + $this->assertSame('tail-b', $events[201]['type']); + $this->assertSame(202, $events[201]['_seq']); + $this->assertSame(range(1, 202), array_column($events, '_seq')); + } + + private function terminalRun(): AiTurnRun + { + $store = $this->app->make(TurnRunStore::class); + $run = $store->open($this->conversation(), 'Hi'); + $store->markRunning($run); + $store->finish($run, AiTurnRunStatus::Completed); + + return $run; + } + + private function event(int $sequence, string $type): AiTurnEvent + { + return new AiTurnEvent([ + 'sequence' => $sequence, + 'payload' => ['type' => $type], + ]); + } + + /** + * A store whose reads follow a script, then come back empty forever. + * + * @param array> $pages + */ + private function scriptedStore(array $pages): TurnRunStore + { + return new class ($pages) extends TurnRunStore { + /** @param array> $pages */ + public function __construct(private array $pages) + { + parent::__construct(); + } + + public function eventsAfter(AiTurnRun $run, int $sequence, int $limit = 200): Collection + { + return array_shift($this->pages) ?? new Collection(); + } + }; + } + + public function test_sequences_become_sse_ids_and_never_leak_into_the_payload(): void + { + $frames = iterator_to_array((new SseFrameEncoder())->encode([ + ['type' => 'content_block_delta', 'delta' => ['text' => 'Hi'], '_seq' => 7], + ]), false); + + $this->assertSame("id: 7\ndata: " . json_encode([ + 'type' => 'content_block_delta', + 'delta' => ['text' => 'Hi'], + ]) . "\n\n", $frames[0]); + } +} diff --git a/tests/Feature/TurnRunStoreTest.php b/tests/Feature/TurnRunStoreTest.php new file mode 100644 index 0000000..a1b244d --- /dev/null +++ b/tests/Feature/TurnRunStoreTest.php @@ -0,0 +1,162 @@ +loadLaravelMigrations(); + } + + protected function setUp(): void + { + parent::setUp(); + + if (!Schema::hasColumn('ai_conversations', 'uuid')) { + Schema::table('ai_conversations', function ($table): void { + $table->string('uuid')->nullable(); + }); + } + } + + private function conversation(): AiConversation + { + $system = AiSystem::create([ + 'name' => 'Test System', + 'provider' => 'anthropic', + 'api_key' => 'sk-ant-test', + 'model' => 'claude-sonnet-4-6', + 'max_tokens' => 1024, + 'is_active' => true, + ]); + + return AiConversation::create([ + 'ai_system_id' => $system->id, + 'feature' => 'persona:test', + ]); + } + + private function store(): TurnRunStore + { + return $this->app->make(TurnRunStore::class); + } + + public function test_appended_events_are_sequenced_from_one(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $this->assertSame(1, $store->append($run, ['type' => 'message_start'])); + $this->assertSame(2, $store->append($run, ['type' => 'content_block_delta'])); + + $this->assertSame( + ['message_start', 'content_block_delta'], + $store->eventsAfter($run, 0)->pluck('payload.type')->all(), + ); + } + + public function test_events_after_a_sequence_returns_only_the_tail(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $store->append($run, ['type' => 'a']); + $store->append($run, ['type' => 'b']); + $store->append($run, ['type' => 'c']); + + $this->assertSame(['b', 'c'], $store->eventsAfter($run, 1)->pluck('payload.type')->all()); + $this->assertTrue($store->eventsAfter($run, 3)->isEmpty()); + } + + public function test_a_run_nobody_polls_is_stopped_once_the_grace_period_lapses(): void + { + config()->set('code-talker.turns.abandon_after_seconds', 30); + + // A zero-interval store: this test travels through Carbon test time, so + // the wall-clock throttle cache must be out of the way for the second + // shouldStop() call to read fresh state. + $store = new TurnRunStore(0.0); + $run = $store->open($this->conversation(), 'Hello'); + + // Freshly opened and never polled: the reader has not connected yet. + $this->assertFalse($store->shouldStop($run)); + + // Still never polled, but now well past the grace period. + Carbon::setTestNow(now()->addSeconds(31)); + $this->assertTrue($store->shouldStop($run)); + $this->assertSame(AiTurnRunStatus::Abandoned, $store->stopStatusFor($run)); + + Carbon::setTestNow(); + } + + public function test_polling_keeps_a_run_alive(): void + { + config()->set('code-talker.turns.abandon_after_seconds', 30); + + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + Carbon::setTestNow(now()->addSeconds(29)); + $store->touchPoll($run); + + Carbon::setTestNow(now()->addSeconds(20)); + $this->assertFalse($store->shouldStop($run)); + + Carbon::setTestNow(); + } + + public function test_an_explicit_cancel_stops_the_run(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $store->requestCancel($run); + + $this->assertTrue($store->shouldStop($run)); + $this->assertSame(AiTurnRunStatus::Cancelled, $store->stopStatusFor($run)); + } + + public function test_should_stop_is_throttled_so_it_never_queries_per_token(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $store->shouldStop($run); + + $queries = 0; + \Illuminate\Support\Facades\DB::listen(function () use (&$queries): void { + $queries++; + }); + + for ($i = 0; $i < 50; $i++) { + $store->shouldStop($run); + } + + $this->assertSame(0, $queries); + } + + public function test_finishing_records_the_status_and_the_error(): void + { + $store = $this->store(); + $run = $store->open($this->conversation(), 'Hello'); + + $store->finish($run, AiTurnRunStatus::Failed, 'provider exploded'); + + $run->refresh(); + $this->assertSame(AiTurnRunStatus::Failed, $run->status); + $this->assertSame('provider exploded', $run->error_message); + $this->assertNotNull($run->finished_at); + } +}