diff --git a/docs/README.md b/docs/README.md index be476c3..7141dcb 100644 --- a/docs/README.md +++ b/docs/README.md @@ -39,7 +39,7 @@ docs/ │ └── design-system.md │ ├── adr/ # Architecture Decision Records -│ └── 0001-*.md through 0015-*.md +│ └── 0001-*.md through 0016-*.md │ ├── Feature Plans/ # Future feature plans │ ├── claude-plugin-packaging.md diff --git a/docs/SYSTEM_ARCHITECTURE.md b/docs/SYSTEM_ARCHITECTURE.md index 06783d3..da70c20 100644 --- a/docs/SYSTEM_ARCHITECTURE.md +++ b/docs/SYSTEM_ARCHITECTURE.md @@ -1,14 +1,14 @@ --- document_id: doc.system-architecture -last_verified: 2026-03-06 -tokens_estimate: 950 +last_verified: 2026-05-18 +tokens_estimate: 1050 tags: - architecture - system - overview anchors: - id: overview - summary: "Desktop Next.js + Electron app; SQLite, agentic-flow, RuVector" + summary: "Desktop Next.js + Electron app; SQLite, Agent SDK, RuVector" - id: data-model summary: "Project → Workflow → Activity → Card; knowledge items; orchestration entities" - id: data-flow @@ -43,7 +43,7 @@ Single-user desktop app. Next.js serves UI + API. Electron wraps it. SQLite stor ┌─────────────────────────────────────────────────────────────┐ │ Electron Shell │ │ ┌──────────────────────────────────────────────────────┐ │ -│ │ Next.js 15 (standalone) │ │ +│ │ Next.js 16 (standalone) │ │ │ │ ┌──────────┐ ┌──────────┐ ┌───────────────────┐ │ │ │ │ │ React 19 │ │ API │ │ Claude Agent SDK │ │ │ │ │ │ UI │──│ Routes │──│ (in-process, │ │ │ @@ -73,7 +73,7 @@ Single-user desktop app. Next.js serves UI + API. Electron wraps it. SQLite stor | UI | React 19 + Tailwind v4 + shadcn/ui | `app/`, `components/` | | API | Next.js App Router (route handlers) | `app/api/` | | Database | SQLite via better-sqlite3 (local, no Supabase) | `lib/db/` | -| Planning LLM | Anthropic Claude (streaming + non-streaming) | `lib/llm/` | +| Planning LLM | Claude Agent SDK `query()` with CLI fallback | `lib/llm/` | | Build agents | `@anthropic-ai/claude-agent-sdk` `query()` (streaming, in-process) | `lib/orchestration/` | | Agent definitions | agentic-flow registry (system prompts only) | `node_modules/agentic-flow/` | | Embeddings | all-MiniLM-L6-v2 (ONNX WASM, local) | `lib/memory/embedding.ts` | @@ -106,7 +106,7 @@ Full schema details: [data-contracts-reference.md](domains/data-contracts-refere ## Data Flow (Summaries) ### Planning -`User chat → Anthropic streaming API → stream-action-parser → PlanningAction[] → validate → apply → SQLite` +`User chat → resolve credential → Agent SDK query() or claude CLI fallback → stream-action-parser → PlanningAction[] → validate → apply → SQLite` Detail: [planning-reference.md](domains/planning-reference.md) @@ -138,15 +138,17 @@ Two distinct connection patterns exist. | Concern | Planning LLM | Build Agent | |---------|-------------|-------------| -| Auth | `ANTHROPIC_API_KEY` | `ANTHROPIC_API_KEY` | -| SDK | `@anthropic-ai/sdk` (Messages API) | `@anthropic-ai/claude-agent-sdk` | -| Call style | `messages.create` / `messages.stream` | `query()` — async iterator | +| Auth | `ANTHROPIC_API_KEY`, `~/.dossier/config`, Claude CLI settings, or CLI subprocess fallback | `ANTHROPIC_API_KEY` | +| SDK | `@anthropic-ai/claude-agent-sdk`; no-credential fallback uses `claude -p` | `@anthropic-ai/claude-agent-sdk` | +| Call style | `query()` result text; CLI uses JSON / stream-json stdout | `query()` — async iterator | | Streaming | Optional (non-streaming for simple calls, streaming for chat) | Always streaming (`for await` over messages) | -| Tools | None (text output only) | Read, Write, Edit, Bash, Glob, Grep, WebFetch, WebSearch | -| CWD | N/A | `worktree_path` (repo clone) | +| Tools | WebSearch always; Read, Glob, Grep only when repo `cwd` is available | Read, Write, Edit, Bash, Glob, Grep, WebFetch, WebSearch | +| CWD | Repo clone when connected; otherwise none | `worktree_path` (repo clone) | | Lifecycle | Request-response per chat turn | Fire-and-forget; result via in-process webhook callback | | Model | `claude-haiku-4-5-20251001` (configurable) | `claude-sonnet-4-5-20250929` (configurable) | +Credentialed planning passes API keys via `ANTHROPIC_API_KEY`; OAuth tokens from Claude Code settings are passed as `CLAUDE_CODE_OAUTH_TOKEN` and `ANTHROPIC_AUTH_TOKEN`. See [ADR 0016](adr/0016-planning-agent-two-auth-paths.md). + --- ## Key Directories diff --git a/docs/docs-index.yaml b/docs/docs-index.yaml index fc6899f..604295b 100644 --- a/docs/docs-index.yaml +++ b/docs/docs-index.yaml @@ -6,17 +6,17 @@ docs_index: documents: - id: doc.system-architecture path: SYSTEM_ARCHITECTURE.md - tokens_estimate: 1200 + tokens_estimate: 1250 tags: [architecture, system, overview] anchors: - id: overview - summary: "Self-deployable Next.js app; SQLite + agentic-flow + RuVector" + summary: "Self-deployable Next.js app; SQLite + Agent SDK + RuVector" - id: data-flow - summary: "Planning LLM → actions → DbAdapter; Build → orchestration → agents" + summary: "Planning → Agent SDK/CLI → actions; Build → Agent SDK → agents" - id: endpoints - summary: "REST API under /api/projects; map, actions, artifacts, cards" + summary: "REST API under /api; projects, docs, setup, orchestration" depends_on: [] - last_verified: "2026-02-18" + last_verified: "2026-05-18" - id: doc.data-contracts path: domains/data-contracts-reference.md @@ -58,25 +58,25 @@ docs_index: - id: doc.api-reference path: domains/api-reference.md - tokens_estimate: 400 + tokens_estimate: 500 tags: [api, endpoints] anchors: - id: endpoints - summary: "REST API under /api/projects; map, actions, artifacts, cards" + summary: "REST API under /api; projects, docs, setup, orchestration" depends_on: [doc.system-architecture] - last_verified: "2026-02-18" + last_verified: "2026-05-18" - id: doc.planning path: domains/planning-reference.md - tokens_estimate: 700 + tokens_estimate: 950 tags: [planning, llm, chat] anchors: - id: contract summary: "Planning LLM converts ideas to PlanningAction[]; never code-gen" - id: modes - summary: "Scaffold, populate, full; mode selected by map state" + summary: "Scaffold, populate, full, finalize; mode selected by map state" depends_on: [doc.system-architecture, doc.data-contracts] - last_verified: "2026-02-18" + last_verified: "2026-05-18" - id: doc.mutation path: domains/mutation-reference.md @@ -104,15 +104,15 @@ docs_index: - id: doc.orchestration path: domains/orchestration-reference.md - tokens_estimate: 750 + tokens_estimate: 1000 tags: [orchestration, build, runs] anchors: - id: contract summary: "OrchestrationRun → CardAssignment; checks before approval; PR user-gated" - id: flow - summary: "createRun → assignments → agentic-flow → checks → approval → PR" + summary: "createRun → assignments → Agent SDK → checks → approval → PR" depends_on: [doc.system-architecture, doc.data-contracts] - last_verified: "2026-02-18" + last_verified: "2026-05-18" - id: doc.testing path: testing-reference.md diff --git a/docs/domains/api-reference.md b/docs/domains/api-reference.md index 14b6ec6..804e6ce 100644 --- a/docs/domains/api-reference.md +++ b/docs/domains/api-reference.md @@ -1,13 +1,13 @@ --- document_id: doc.api-reference -last_verified: 2026-02-18 -tokens_estimate: 400 +last_verified: 2026-05-18 +tokens_estimate: 500 tags: - api - endpoints anchors: - id: endpoints - summary: "REST API under /api/projects; map, actions, artifacts, cards" + summary: "REST API under /api; projects, docs, setup, orchestration" ttl_expires_on: null --- # API Reference @@ -24,6 +24,8 @@ ttl_expires_on: null | Group | Base Path | Purpose | |-------|-----------|---------| | Projects | `/api/projects` | CRUD projects | +| Setup & GitHub | `/api/setup`, `/api/github/*` | Local credentials and GitHub OAuth/PAT flows | +| Docs | `/api/docs` | Docs panel index and doc content under `docs/` | | Map | `/api/projects/[id]/map` | Canonical map snapshot | | Actions | `/api/projects/[id]/actions` | Submit planning actions | | Chat | `/api/projects/[id]/chat`, `/chat/stream` | Planning LLM | @@ -31,6 +33,8 @@ ttl_expires_on: null | Card knowledge | `/api/projects/[id]/cards/[cardId]/{requirements,facts,assumptions,questions}` | Knowledge items | | Planned files | `/api/projects/[id]/cards/[cardId]/planned-files` | Card planned files | | Files | `/api/projects/[id]/files` | File tree (planned or repo); `?source=repo` for produced code | +| Orchestration | `/api/projects/[id]/orchestration/*` | Build trigger, resume, approvals, PR candidates | +| Developer utilities | `/api/dev/restart-and-open` | Local-only project preview server | ## Related - [data-contracts-reference.md](data-contracts-reference.md) diff --git a/docs/domains/orchestration-reference.md b/docs/domains/orchestration-reference.md index 1938655..d62d2b8 100644 --- a/docs/domains/orchestration-reference.md +++ b/docs/domains/orchestration-reference.md @@ -1,7 +1,7 @@ --- document_id: doc.orchestration -last_verified: 2026-03-06 -tokens_estimate: 950 +last_verified: 2026-05-18 +tokens_estimate: 1000 tags: - orchestration - build @@ -10,7 +10,7 @@ anchors: - id: contract summary: "OrchestrationRun → CardAssignment; checks before approval; PR user-gated" - id: flow - summary: "createRun → assignments → agentic-flow → checks → approval → PR" + summary: "createRun → assignments → Agent SDK → checks → approval → PR" - id: policy summary: "SystemPolicyProfile: required_checks, protected_paths, forbidden_paths" ttl_expires_on: null @@ -41,7 +41,7 @@ User trigger (card | workflow) → createRun (validate policy, capture snapshots; worktree_root = clone path) → createFeatureBranch per card → createAssignment per card (feature_branch, worktree_path, allowed_paths, forbidden_paths) - → dispatch to agentic-flow (cwd = worktree_path) + → dispatch to in-process Claude Agent SDK (cwd = worktree_path) → agents write files, commit to feature branch → GET /api/projects/[id]/files?source=repo surfaces produced files with diff status → execute checks (dependency, security, policy, lint, unit, integration, e2e) @@ -80,7 +80,8 @@ dispatch.ts → createAgenticFlowClient() → SDK query() | `lib/orchestration/create-run.ts` | createRun; policy validation; snapshot capture | | `lib/orchestration/create-assignment.ts` | CardAssignment per card | | `lib/orchestration/trigger-build.ts` | Entry point; clones repo, creates branch, populates worktree_path | -| `lib/orchestration/dispatch.ts` | Dispatch to agentic-flow | +| `lib/orchestration/dispatch.ts` | Build assignment payload and dispatch to execution client | +| `lib/orchestration/agentic-flow-client.ts` | Direct Agent SDK `query()` execution using agentic-flow prompts | | `lib/orchestration/execute-checks.ts` | Run required checks | | `lib/orchestration/approval-gates.ts` | Check pass before approval request | | `lib/orchestration/create-approval-request.ts` | ApprovalRequest creation | diff --git a/docs/domains/planning-reference.md b/docs/domains/planning-reference.md index 34afdf5..a35a1d8 100644 --- a/docs/domains/planning-reference.md +++ b/docs/domains/planning-reference.md @@ -1,7 +1,7 @@ --- document_id: doc.planning -last_verified: 2026-03-06 -tokens_estimate: 750 +last_verified: 2026-05-18 +tokens_estimate: 950 tags: - planning - llm @@ -10,9 +10,9 @@ anchors: - id: contract summary: "Planning LLM converts ideas to PlanningAction[]; never code-gen" - id: modes - summary: "Scaffold, populate, full; mode selected by map state" + summary: "Scaffold, populate, full, finalize; mode selected by map state" - id: flow - summary: "Chat → Claude → stream-action-parser → actions" + summary: "Chat → Agent SDK or CLI → stream-action-parser → actions" ttl_expires_on: null --- # Planning Domain Reference @@ -47,35 +47,51 @@ Mode selected by `lib/llm/planning-prompt.ts` based on map state. ### Flow ``` -User message → POST /chat/stream +User message → POST /api/projects/[projectId]/chat/stream → buildPlanningSystemPrompt() | buildScaffoldSystemPrompt() | buildPopulateSystemPrompt() | buildFinalizeSystemPrompt() - → Claude API (streaming) + → resolvePlanningCredential() + → if credential exists: planning-sdk-runner query() + - WebSearch always + - Read, Glob, Grep only when repo cwd is available + → else if Claude CLI is authenticated: claude -p --output-format stream-json → stream-action-parser (parse JSON blocks) → PlanningAction[] emitted - → POST /actions (validate + apply) + → POST /api/projects/[projectId]/actions (validate + apply) ``` ### Per-Card Finalize Flow ``` -User clicks "Finalize" on card → POST /cards/[cardId]/finalize - → Assemble: project-wide docs + card context + e2e tests - → Return finalization package for review - → User edits (optional) - → POST /cards/[cardId]/finalize/confirm - → Set card.finalized_at → card is build-ready +GET /api/projects/[projectId]/cards/[cardId]/finalize + → Assemble review package: project docs + card artifacts + requirements + planned files + +User clicks "Finalize" on card → POST /api/projects/[projectId]/cards/[cardId]/finalize + → SSE finalize_progress: link project docs to card + → SSE action: createContextArtifact for generated e2e/context artifacts + → SSE finalize_progress: set card.finalized_at and ingest memory when enabled + → SSE phase_complete: card_finalize_complete + → SSE done ``` +Constraints enforced before POST streaming starts: +- Card must belong to the project and not already be finalized. +- Project must be finalized first. +- Card must have at least one requirement and one planned file. +- `NEXT_PUBLIC_PLANNING_LLM_ENABLED=true` must enable the planning LLM. + ### Key Files | File | Purpose | |------|---------| | `lib/llm/planning-prompt.ts` | System prompts; mode selection | | `lib/llm/stream-action-parser.ts` | Parse streaming JSON → actions | | `lib/llm/build-preview-response.ts` | Preview response before apply | -| `lib/llm/claude-client.ts` | Planning LLM client (Messages API) | -| `lib/llm/planning-credential.ts` | Resolves ANTHROPIC_API_KEY from env or ~/.dossier/config | +| `lib/llm/claude-client.ts` | Planning auth routing; Agent SDK path with CLI subprocess fallback | +| `lib/llm/planning-sdk-runner.ts` | Agent SDK `query()` runner; read-only planning tools | +| `lib/llm/planning-sdk-bridge.ts` | Converts SDK result text into planning response shape | +| `lib/llm/planning-credential.ts` | Resolves env/config/Claude CLI credentials for planning | +| `lib/llm/run-llm-substep.ts` | Finalize sub-step runner that filters generated actions | | `app/api/projects/[id]/chat/route.ts` | Non-streaming chat | | `app/api/projects/[id]/chat/stream/route.ts` | Streaming chat (scaffold, populate, finalize) | -| `app/api/projects/[id]/cards/[cardId]/finalize/route.ts` | Per-card finalize endpoint | +| `app/api/projects/[id]/cards/[cardId]/finalize/route.ts` | GET review package and POST SSE card finalization | ### Response Types - `clarification`: Questions only; `actions: []` @@ -88,6 +104,7 @@ User clicks "Finalize" on card → POST /cards/[cardId]/finalize - [ ] No action proposes code generation (validate-action rejects) - [ ] Prompt instructs LLM to use existing IDs from context - [ ] User actions follow-up after populate (agent prompts for View Details, Build, etc.) +- [ ] Finalize docs match GET package plus POST SSE behavior; no separate confirm route ## Related - [mutation-reference.md](mutation-reference.md) diff --git a/docs/reference/api-endpoints.md b/docs/reference/api-endpoints.md index 692099b..3271fba 100644 --- a/docs/reference/api-endpoints.md +++ b/docs/reference/api-endpoints.md @@ -536,6 +536,38 @@ Resumes a previously blocked assignment after user input is provided. --- +## Documentation Browser + +### GET /api/docs + +Returns documentation metadata from `docs/docs-index.yaml` for the in-app Docs panel, or returns one document's Markdown content when `path` is provided. + +**Query params:** + +| Param | Description | +|-------|-------------| +| `path` | Optional path relative to `docs/`, for example `domains/planning-reference.md` | + +**Response without `path`:** `200` +```json +{ + "documents": [ + { "id": "doc.planning", "path": "domains/planning-reference.md", "tags": ["planning", "llm", "chat"] } + ] +} +``` + +**Response with `path`:** `200` +```json +{ "content": "# Planning Domain Reference\n..." } +``` + +Constraints: +- Paths are normalized and confined under the repository `docs/` directory. +- Missing docs return `404`; invalid traversal paths return `400`. + +--- + ## Developer Utilities ### POST /api/dev/restart-and-open