From b30aca28880388221afc9fc46372a224d1e10c8c Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Mon, 30 Mar 2026 16:15:23 +0000 Subject: [PATCH] docs: refresh API, planning auth, and developer runbooks Co-authored-by: rwliebs --- README.md | 4 +- docs/development-reference.md | 49 ++++- docs/domains/api-reference.md | 9 +- docs/domains/planning-reference.md | 27 ++- docs/reference/api-endpoints.md | 220 ++++++++++++++++++++++ docs/reference/configuration-reference.md | 33 +++- 6 files changed, 325 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index c6f3d40..044800b 100644 --- a/README.md +++ b/README.md @@ -141,7 +141,7 @@ Your browser will open to **http://localhost:3000**. First run creates `~/.dossi On first run you'll be guided through API key setup, or navigate directly to `/setup`. -**Prerequisites:** Node.js 20+, [Anthropic API key](https://console.anthropic.com/). A [GitHub token](https://github.com/settings/tokens) with `repo` scope is needed only if you want agents to open PRs; agents can write files locally without it. +**Prerequisites:** Node.js 20+, and Anthropic credentials. For planning, you can use an [Anthropic API key](https://console.anthropic.com/) or an authenticated Claude CLI install. Build orchestration requires `ANTHROPIC_API_KEY`. A [GitHub token](https://github.com/settings/tokens) with `repo` scope is needed only if you want agents to open PRs; agents can write files locally without it. > **Windows users:** Dossier uses native SQLite, which requires a C++ compiler during install. Install [Visual Studio Build Tools](https://visualstudio.microsoft.com/downloads/) with the **"Desktop development with C++"** workload, or run this from an elevated PowerShell: > ```powershell @@ -165,7 +165,7 @@ On first run you'll be guided through API key setup, or navigate directly to `/s Configuration (API keys, GitHub token) can be set via the web UI at `/setup` or by editing `~/.dossier/config`. See [.env.example](.env.example) for all options. -NEW: Use your Claude MAX account directly, no extra setup required (if you've got Claude Code installed locally.) +NEW: You can use your Claude CLI authentication for planning workflows. Build orchestration still requires `ANTHROPIC_API_KEY`. --- diff --git a/docs/development-reference.md b/docs/development-reference.md index 6e783d4..4a43d7b 100644 --- a/docs/development-reference.md +++ b/docs/development-reference.md @@ -1,6 +1,6 @@ --- document_id: doc.development -last_verified: 2026-02-18 +last_verified: 2026-03-30 tokens_estimate: 700 tags: - development @@ -46,6 +46,7 @@ pnpm install # or: npm install - **Development**: Copy `.env.example` to `.env.local`; fill `ANTHROPIC_API_KEY`, `GITHUB_TOKEN` - **Self-deploy**: Use `/setup` or edit `~/.dossier/config` - Precedence: `process.env` > `.env.local` > `~/.dossier/config` +- Planning can run through Claude CLI auth when API key is absent; build orchestration still requires `ANTHROPIC_API_KEY` --- @@ -89,6 +90,52 @@ Override: `DOSSIER_DATA_DIR` or `SQLITE_PATH` --- +## Operational Runbook (Common Tasks) + +### Sync local clone after merging on GitHub + +Use: + +`POST /api/projects/[projectId]/repo/sync` + +This fetches and syncs local `` in `~/.dossier/repos//` to `origin/`. + +### Push a completed card branch + +Use: + +`POST /api/projects/[projectId]/cards/[cardId]/push` + +Requires: +- Connected repository on project +- Completed assignment for the card +- `GITHUB_TOKEN` configured + +### View app from cloned project during development + +Use: + +`POST /api/dev/restart-and-open` with `{ "projectId": "..." }` + +Behavior: +- Dev-only (`NODE_ENV=development`) +- Starts `npm run dev` in clone path +- Picks first free port in `3001..3010` + +--- + +## Troubleshooting + +- **Planning works but build trigger fails with Anthropic credential error** + - Planning may be running via Claude CLI auth. + - Build path uses orchestration/Agent SDK and requires `ANTHROPIC_API_KEY`. +- **Push/sync endpoints failing with auth errors** + - Verify `GITHUB_TOKEN` is set in env or `~/.dossier/config`. +- **Finalize blocked** + - Card finalize requires project finalized state, at least one requirement, and at least one planned file. + +--- + ## Verification - [ ] `npm run dev` starts on 3000 - [ ] `npm run test` passes diff --git a/docs/domains/api-reference.md b/docs/domains/api-reference.md index 14b6ec6..8c33b1c 100644 --- a/docs/domains/api-reference.md +++ b/docs/domains/api-reference.md @@ -1,6 +1,6 @@ --- document_id: doc.api-reference -last_verified: 2026-02-18 +last_verified: 2026-03-30 tokens_estimate: 400 tags: - api @@ -30,7 +30,14 @@ ttl_expires_on: null | Artifacts | `/api/projects/[id]/artifacts` | Context artifacts | | Card knowledge | `/api/projects/[id]/cards/[cardId]/{requirements,facts,assumptions,questions}` | Knowledge items | | Planned files | `/api/projects/[id]/cards/[cardId]/planned-files` | Card planned files | +| Finalize | `/api/projects/[id]/cards/[cardId]/finalize` | Card finalization package + SSE finalize pipeline | +| Orchestration build | `/api/projects/[id]/orchestration/build` | Trigger card/workflow build runs | +| Orchestration runs | `/api/projects/[id]/orchestration/runs` | Create/list runs, assignments, checks | +| Approvals & PR candidates | `/api/projects/[id]/orchestration/{approvals,pull-requests}` | Human-gated approval + PR metadata lifecycle | +| Repo sync/push | `/api/projects/[id]/repo/sync`, `/cards/[cardId]/push` | Sync local base branch and push card feature branch | | Files | `/api/projects/[id]/files` | File tree (planned or repo); `?source=repo` for produced code | +| Setup & GitHub | `/api/setup`, `/api/setup/status`, `/api/github/repos` | First-run credentials and GitHub repo list/create | +| Docs & memory | `/api/docs`, `/api/projects/[id]/memory` | Docs index/content and memory inspection | ## Related - [data-contracts-reference.md](data-contracts-reference.md) diff --git a/docs/domains/planning-reference.md b/docs/domains/planning-reference.md index 34afdf5..bd2d9a1 100644 --- a/docs/domains/planning-reference.md +++ b/docs/domains/planning-reference.md @@ -1,6 +1,6 @@ --- document_id: doc.planning -last_verified: 2026-03-06 +last_verified: 2026-03-30 tokens_estimate: 750 tags: - planning @@ -25,6 +25,7 @@ ttl_expires_on: null - INVARIANT: Planning LLM outputs PlanningAction[] only; never production code or file contents - INVARIANT: Code-generation intents rejected; respond with clarification redirect - INVARIANT: IDs in actions must exist in current map state; new entities get fresh UUIDs +- INVARIANT: Planning auth can use API key or Claude CLI; build orchestration remains API-key-based ### Boundaries - ALLOWED: createWorkflow, createActivity, createCard, updateCard, linkContextArtifact, upsertCardPlannedFile, createContextArtifact, etc. @@ -49,20 +50,28 @@ Mode selected by `lib/llm/planning-prompt.ts` based on map state. ``` User message → POST /chat/stream → buildPlanningSystemPrompt() | buildScaffoldSystemPrompt() | buildPopulateSystemPrompt() | buildFinalizeSystemPrompt() - → Claude API (streaming) + → auth resolution (API key or Claude CLI) + → Claude call (SDK stream or CLI stream-json) → stream-action-parser (parse JSON blocks) → PlanningAction[] emitted → POST /actions (validate + apply) ``` +### Auth + Transport Paths + +| Path | When used | Transport | +|------|-----------|-----------| +| API key path | `ANTHROPIC_API_KEY` available via env/config/CLI settings file | Anthropic SDK bridge (`runPlanningQuery` / `streamPlanningQuery`) | +| CLI path | No API key resolved, but `claude` CLI is installed/authenticated | `claude -p` subprocess (`--output-format json` or `stream-json`) | + +Build agents do not use the CLI path; they require `ANTHROPIC_API_KEY`. + ### Per-Card Finalize Flow ``` User clicks "Finalize" on card → POST /cards/[cardId]/finalize - → Assemble: project-wide docs + card context + e2e tests - → Return finalization package for review - → User edits (optional) - → POST /cards/[cardId]/finalize/confirm - → Set card.finalized_at → card is build-ready + → SSE pipeline: link project docs + generate e2e test artifact + stamp finalized_at + → Optional memory ingest for build retrieval context + → Card becomes build-ready ``` ### Key Files @@ -71,8 +80,8 @@ User clicks "Finalize" on card → POST /cards/[cardId]/finalize | `lib/llm/planning-prompt.ts` | System prompts; mode selection | | `lib/llm/stream-action-parser.ts` | Parse streaming JSON → actions | | `lib/llm/build-preview-response.ts` | Preview response before apply | -| `lib/llm/claude-client.ts` | Planning LLM client (Messages API) | -| `lib/llm/planning-credential.ts` | Resolves ANTHROPIC_API_KEY from env or ~/.dossier/config | +| `lib/llm/claude-client.ts` | Planning auth/router (API-key SDK path or Claude CLI subprocess path) | +| `lib/llm/planning-credential.ts` | Resolves planning credential from env/config/`~/.claude/settings.json` | | `app/api/projects/[id]/chat/route.ts` | Non-streaming chat | | `app/api/projects/[id]/chat/stream/route.ts` | Streaming chat (scaffold, populate, finalize) | | `app/api/projects/[id]/cards/[cardId]/finalize/route.ts` | Per-card finalize endpoint | diff --git a/docs/reference/api-endpoints.md b/docs/reference/api-endpoints.md index 3b0fe12..257f7ee 100644 --- a/docs/reference/api-endpoints.md +++ b/docs/reference/api-endpoints.md @@ -34,6 +34,58 @@ All errors return JSON: --- +## Setup & Environment + +### GET /api/setup/status + +Return whether first-run setup is required. + +**Response:** `200` +```json +{ + "needsSetup": true, + "missingKeys": ["ANTHROPIC_API_KEY", "GITHUB_TOKEN"], + "configPath": "/home/user/.dossier/config", + "anthropicViaCli": false +} +``` + +Notes: +- `anthropicViaCli=true` means planning can run via installed Claude CLI even when API key is missing. +- Build orchestration still requires `ANTHROPIC_API_KEY`. + +### POST /api/setup + +Persist credentials to `~/.dossier/config`. + +**Request body:** +```json +{ + "anthropicApiKey": "sk-ant-...", + "githubToken": "ghp_..." +} +``` + +At least one key is required. + +### GET /api/github/repos + +List repositories visible to configured `GITHUB_TOKEN`. + +### POST /api/github/repos + +Create a GitHub repository for the authenticated user. + +**Request body:** +```json +{ + "name": "my-repo", + "private": true +} +``` + +--- + ## Project Management ### GET /api/projects @@ -148,6 +200,19 @@ Submit planning actions. Validates, applies, and persists. Rejects on first fail Code-generation intents are rejected. +### POST /api/projects/[projectId]/actions/preview + +Dry-run action batch. Validates and returns previews without DB mutation. + +**Response:** `200` +```json +{ + "success": true, + "previews": [{ "summary": "Create workflow Checkout" }], + "summary": ["Create workflow Checkout"] +} +``` + --- ## Context Artifacts @@ -271,6 +336,159 @@ Delete planned file. --- +## Chat & Finalization + +### POST /api/projects/[projectId]/chat + +Non-streaming planning endpoint. Supports `scaffold`, `populate`, `finalize` modes. + +**Request body:** +```json +{ + "message": "Create checkout workflow", + "mode": "scaffold", + "workflow_id": "uuid-when-populate" +} +``` + +### POST /api/projects/[projectId]/chat/stream + +Streaming SSE planning endpoint. Emits action/progress events as planning runs. + +### GET /api/projects/[projectId]/cards/[cardId]/finalize + +Return card finalization package (card, project docs, linked artifacts, requirements, planned files). + +### POST /api/projects/[projectId]/cards/[cardId]/finalize + +Streaming SSE card finalization: +1. Links project docs to card +2. Generates e2e test context artifact +3. Sets `card.finalized_at` + +Possible validation responses include: +- Project not finalized +- No card requirements +- No planned files + +### GET /api/projects/[projectId]/cards/[cardId]/context-artifacts + +List artifacts linked to a card. + +### GET /api/projects/[projectId]/cards/[cardId]/produced-files + +List files produced by the completed assignment for a card (`added|modified`). + +--- + +## Build & Orchestration + +### POST /api/projects/[projectId]/orchestration/build + +Trigger build run for a workflow or single card. + +**Request body:** +```json +{ + "scope": "card", + "card_id": "uuid", + "initiated_by": "user", + "trigger_type": "manual" +} +``` + +**Response:** `202` +```json +{ + "runId": "uuid", + "assignmentIds": ["uuid"], + "message": "Build started", + "outcome_type": "success" +} +``` + +### Runs + +- `GET /api/projects/[projectId]/orchestration/runs?scope=card|workflow&status=...&limit=...` +- `POST /api/projects/[projectId]/orchestration/runs` +- `GET /api/projects/[projectId]/orchestration/runs/[runId]` +- `PATCH /api/projects/[projectId]/orchestration/runs/[runId]` + +### Assignments + +- `GET /api/projects/[projectId]/orchestration/runs/[runId]/assignments` +- `POST /api/projects/[projectId]/orchestration/runs/[runId]/assignments` +- `GET /api/projects/[projectId]/orchestration/runs/[runId]/assignments/[assignmentId]` +- `POST /api/projects/[projectId]/orchestration/runs/[runId]/assignments/[assignmentId]/dispatch` + +### Checks + +- `GET /api/projects/[projectId]/orchestration/runs/[runId]/checks` +- `POST /api/projects/[projectId]/orchestration/runs/[runId]/checks` +- `GET /api/projects/[projectId]/orchestration/runs/[runId]/checks/[checkId]` + +### Approvals & PR candidates + +- `GET /api/projects/[projectId]/orchestration/approvals?run_id=` +- `POST /api/projects/[projectId]/orchestration/approvals` +- `GET /api/projects/[projectId]/orchestration/approvals/[approvalId]` +- `PATCH /api/projects/[projectId]/orchestration/approvals/[approvalId]` +- `GET /api/projects/[projectId]/orchestration/pull-requests?run_id=` +- `POST /api/projects/[projectId]/orchestration/pull-requests` +- `GET /api/projects/[projectId]/orchestration/pull-requests/[prId]` +- `PATCH /api/projects/[projectId]/orchestration/pull-requests/[prId]` + +### Recovery & Webhooks + +- `POST /api/projects/[projectId]/orchestration/resume-blocked` +- `POST /api/projects/[projectId]/orchestration/webhooks/agentic-flow` + +--- + +## Repository Operations + +### POST /api/projects/[projectId]/repo/sync + +Fetch/clone (if needed) and sync local base branch to `origin/`. +Use after merging PRs remotely to refresh local clone state. + +### POST /api/projects/[projectId]/cards/[cardId]/push + +Push completed card assignment branch from local clone to origin. + +Requires: +- Connected repository +- Completed assignment for the card +- `GITHUB_TOKEN` configured + +--- + +## Docs & Memory + +### GET /api/docs + +List docs from `docs/docs-index.yaml`. + +### GET /api/docs?path=product/user-workflows-reference.md + +Return raw markdown content for a specific docs path. + +### GET /api/projects/[projectId]/memory + +Return stored memory units for project plus storage paths (SQLite + RuVector). + +--- + +## Dev-only Utilities + +### POST /api/dev/restart-and-open + +Development-only endpoint (`NODE_ENV=development`) used by "View on server": +- Requires `{ "projectId": "uuid" }` +- Starts `npm run dev` in project clone +- Uses first free port in `3001..3010` +- Opens browser tab after startup delay + ## Project Files (Planned + Repository) ### GET /api/projects/[projectId]/files @@ -319,3 +537,5 @@ File tree for the project. Two modes via `source` query param. - **Mutations**: All map changes go through the actions endpoint; no direct writes. - **Auth**: No auth/RLS; endpoints use anon access (single-user desktop app). - **Database**: SQLite only; no Supabase or Postgres. +- **Planning credential model**: API key and Claude CLI paths are supported for planning endpoints. +- **Build credential model**: Build/orchestration routes require `ANTHROPIC_API_KEY` and do not use CLI-only fallback. diff --git a/docs/reference/configuration-reference.md b/docs/reference/configuration-reference.md index f5197b8..5e7614d 100644 --- a/docs/reference/configuration-reference.md +++ b/docs/reference/configuration-reference.md @@ -1,6 +1,6 @@ --- document_id: doc.configuration -last_verified: 2026-02-18 +last_verified: 2026-03-30 tokens_estimate: 600 tags: - configuration @@ -23,19 +23,44 @@ ttl_expires_on: null - INVARIANT: Config precedence: `process.env` > `.env.local` > `~/.dossier/config` - INVARIANT: Self-deploy uses `~/.dossier/config`; dev uses `.env.local` -- Anthropic credential: we accept **API key** first (env, then `~/.dossier/config`). If none is set, we use your **installed Claude CLI** config: `~/.claude/settings.json` (or `CLAUDE_CONFIG_DIR`/settings.json). We read `env.ANTHROPIC_API_KEY` or `env.ANTHROPIC_AUTH_TOKEN` from that file so you don’t need to paste a key if Claude Code is already configured. +- Planning auth supports two paths: + 1. API-key path (`ANTHROPIC_API_KEY` from env/config), or + 2. Claude CLI path (installed/authenticated `claude` CLI, with optional credential read from `~/.claude/settings.json`). +- Build orchestration requires `ANTHROPIC_API_KEY` (env/config). Claude CLI fallback does not satisfy build dispatch. --- ## Required -Anthropic credential (API key or Claude CLI config) and GitHub token: +Credential requirements depend on subsystem: | Variable | Purpose | |----------|---------| -| ANTHROPIC_API_KEY | Planning LLM and build (set in env or `~/.dossier/config`; or we use your Claude CLI `~/.claude/settings.json` when no key is set) | +| ANTHROPIC_API_KEY | Required for build orchestration; also valid for planning LLM | | GITHUB_TOKEN | Push branches, create PRs; [github.com/settings/tokens](https://github.com/settings/tokens) `repo` scope | +Planning can additionally use Claude CLI auth when `ANTHROPIC_API_KEY` is absent. + +--- + +## Credential Behavior by Subsystem + +| Subsystem | Credential resolution | Notes | +|----------|------------------------|-------| +| Planning chat (`/api/projects/[projectId]/chat`, `/chat/stream`) | `ANTHROPIC_API_KEY` (env) → `ANTHROPIC_API_KEY` (`~/.dossier/config`) → `~/.claude/settings.json` (`ANTHROPIC_API_KEY` or `ANTHROPIC_AUTH_TOKEN`) → Claude CLI subprocess if no credential but CLI is installed | Setup status reports this as `anthropicViaCli: true` when CLI path is available | +| Build orchestration (`/api/projects/[projectId]/orchestration/build`) | `ANTHROPIC_API_KEY` (env/config) | Uses Claude Agent SDK query path; no CLI-only fallback | + +--- + +## Troubleshooting + +- **Planning works, build fails with Anthropic credential error** + - Cause: CLI auth satisfies planning but build still needs `ANTHROPIC_API_KEY`. + - Fix: Set `ANTHROPIC_API_KEY` in `/setup`, `.env.local`, or `~/.dossier/config`. +- **`/setup` keeps requiring `GITHUB_TOKEN`** + - Cause: Token missing from both env and config. + - Fix: Add token in `/setup` or config file. Some repo endpoints return `401`/`503` without it. + --- ## Optional