From 0afb7e3ec8c40262c57e3c562a83b575ea2f7949 Mon Sep 17 00:00:00 2001 From: Maxi Date: Mon, 7 Sep 2026 08:31:43 +0100 Subject: [PATCH] feat: replace gh with the REST API, and hold tracker credentials in the cloud `gh` is no longer installable on the machine the runner runs on, which took GitHub polling, labelling and commenting with it. Nothing about that was load-bearing: `gh` lived in exactly one file, and the six calls Sentinel0 actually made are plain REST. `GitHubRestClient` makes them over `fetch`. It lives in `packages/common` rather than the orchestrator because the cloud API needs three of the same calls to answer the dashboard's pickers, and two clients would be two places that know GitHub's field names. The token arrives through a **provider function**, never a constructor string: a PAT is a constant and would not need one, but a GitHub App installation token expires hourly, and the provider is the shape that serves both -- moving to an App changes how a token is obtained and nothing else in the file. Two REST behaviours are not `gh` behaviours, and both fail silently: `GET /repos/{o}/{r}/issues` returns pull requests as well as issues, so unfiltered every open PR would also raise a `ticket` trigger; and `gh`'s `reviewRequests` merged users and teams where REST splits them into `requested_reviewers` and `requested_teams`, so reading only the first makes a route targeting a review-owning team silently never match. Both are pinned by tests. `getPullRequestDiff` is deleted rather than ported. It has had no callers since the Hermes pivot -- fetching a diff is the agent's job under that boundary -- so porting it would have re-crossed the boundary in the wrong direction. What remains is five calls: list issues, list pulls, ensure/add/remove labels, and post a comment. Labels are the half that cannot be given up, because `sentinel0:in-progress` is the loop guard and a runner that cannot write labels re-fires every route on every cycle. The comment exists for `postFailure`, the one case an agent structurally cannot report on its own behalf. Credentials move from each machine to the organization, encrypted at rest with AES-256-GCM under `SENTINEL0_SECRET_KEY`. `crypto.ts` throws rather than deriving a key from nothing: a deployment that quietly encrypted every token under a guessable constant would look exactly like a working one. The cloud is a **credential broker, not a data proxy**. `GET /v1/runner/integrations` hands the runner its tokens and the runner still calls `api.github.com` itself, so trigger collection survives a cloud outage the same way cached routes do -- proxying the data too would have made polling stop whenever Railway did. That endpoint is the only one anywhere returning a plaintext secret, and is runner-scoped for exactly that reason: the `snt_usr_` key a browser holds is rejected before the handler runs. `IntegrationStore` keeps tokens in memory only. Routes and projects cache to `~/.sentinel0/` so an outage cannot stop dispatch and the same argument applies here, but a decrypted PAT in a file is a worse trade than a runner that needs its control plane once at boot. `GITHUB_TOKEN` and `LINEAR_API_KEY` remain as an explicit fallback for the two cases the cloud cannot serve -- no control plane at all, and a restart mid-outage -- and the cloud outranks them whenever it has an answer, or a rotated credential would lose to a stale local one. A failed refresh keeps what was already loaded, for the same reason. Precedence is project override, then organization default, resolved by `resolveIntegration` in `common` so the cloud and the dashboard cannot disagree about it. `PUT /v1/integrations/:provider` verifies a token against its provider before storing it and returns the account it authenticated as: a credential saved unverified fails hours later as a 401 in a log nobody reads, while whoever pasted it has already left the screen believing it worked. Storing the token in the cloud is also what lets the dashboard stop asking people to type things it could look up. *Add project* offered a free-form JSON textarea hinted "use {}", which asked an operator to know the filter schema, spell a repository slug correctly, and guess which labels a repository has -- three mistakes discovered hours later as an empty poll. It now offers the repositories the credential can reach and the chosen repository's real labels, read through the API because a browser must never hold a token that grants writing to someone's repositories. The JSON escape hatch stays, behind a disclosure. The organization pages gain the design's second-level rail (`showOrgRail`). That group had outgrown a flat list: with Integrations added there are five destinations, each a settings surface rather than a view of work, and promoting all five into the primary sidebar would have buried Runs and Routes among them. Settings splits three ways -- Slack moves to Integrations beside the credentials it belongs with, and the runner table to a page of its own, having been the only table on a page of forms. `end` on a rail entry is derived rather than hand-set. On all of them the rail goes blank on a child page like `/projects/new`; on none of them Settings stays lit across every page nested under it. Only entries another entry sits under need it, which is a fact about the list rather than a flag to remember. Also: `errorMessage` moves out of `index.ts` now that a second module needs it, and `HostExecutor` is left in place but is no longer constructed anywhere -- the runner genuinely executes nothing now. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01RbctixjBYjPL823X6CEMd3 --- CLAUDE.md | 70 ++- README.md | 4 +- docs/api.md | 88 ++++ docs/dashboard.md | 22 + docs/deploy-cloud.md | 17 +- docs/getting-started.md | 2 +- docs/routes.md | 13 +- packages/cli/src/commands/init.ts | 38 +- packages/cli/src/commands/preflight.ts | 48 +- packages/cli/vitest.config.ts | 8 +- packages/cloud-api/src/crypto.ts | 104 ++++ packages/cloud-api/src/integrations.ts | 235 +++++++++ .../src/migrations/007_integrations.sql | 44 ++ packages/cloud-api/src/routes/runner.ts | 20 + packages/cloud-api/src/routes/user.ts | 175 +++++++ packages/cloud-api/test/crypto.test.ts | 90 ++++ packages/cloud-api/vitest.config.ts | 7 + packages/cloud-dashboard/src/App.tsx | 4 + packages/cloud-dashboard/src/api/endpoints.ts | 45 ++ packages/cloud-dashboard/src/api/types.ts | 20 + .../src/components/AppShell.tsx | 11 +- .../src/components/OrgRail.tsx | 75 +++ .../src/screens/Integrations.tsx | 446 ++++++++++++++++++ .../src/screens/ProjectNew.tsx | 267 +++++++++-- .../cloud-dashboard/src/screens/Runners.tsx | 157 ++++++ .../cloud-dashboard/src/screens/Settings.tsx | 265 +---------- packages/cloud-dashboard/src/styles.css | 218 +++++++++ .../cloud-dashboard/test/fixtures/README.md | 11 +- .../test/fixtures/integrations.json | 26 + .../cloud-dashboard/test/screens.test.tsx | 109 ++++- packages/common/package.json | 4 + packages/common/src/github.ts | 297 ++++++++++++ packages/common/src/index.ts | 98 +++- packages/common/test/github.test.ts | 181 +++++++ packages/orchestrator/src/cloud/client.ts | 17 + packages/orchestrator/src/github/service.ts | 167 ++----- packages/orchestrator/src/index.ts | 26 +- .../orchestrator/src/integrations/store.ts | 107 +++++ packages/orchestrator/src/linear/service.ts | 17 +- packages/orchestrator/src/runtime/errors.ts | 10 + .../orchestrator/src/runtime/preflight.ts | 44 +- packages/orchestrator/src/runtime/services.ts | 59 +-- .../orchestrator/test/github/service.test.ts | 207 ++++++++ .../test/integrations/store.test.ts | 137 ++++++ packages/orchestrator/vitest.config.ts | 8 +- tsconfig.base.json | 3 + 46 files changed, 3479 insertions(+), 542 deletions(-) create mode 100644 packages/cloud-api/src/crypto.ts create mode 100644 packages/cloud-api/src/integrations.ts create mode 100644 packages/cloud-api/src/migrations/007_integrations.sql create mode 100644 packages/cloud-api/test/crypto.test.ts create mode 100644 packages/cloud-dashboard/src/components/OrgRail.tsx create mode 100644 packages/cloud-dashboard/src/screens/Integrations.tsx create mode 100644 packages/cloud-dashboard/src/screens/Runners.tsx create mode 100644 packages/cloud-dashboard/test/fixtures/integrations.json create mode 100644 packages/common/src/github.ts create mode 100644 packages/common/test/github.test.ts create mode 100644 packages/orchestrator/src/integrations/store.ts create mode 100644 packages/orchestrator/src/runtime/errors.ts create mode 100644 packages/orchestrator/test/github/service.test.ts create mode 100644 packages/orchestrator/test/integrations/store.test.ts diff --git a/CLAUDE.md b/CLAUDE.md index c5d5d1d..465c3e6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -71,7 +71,9 @@ needs **no local clone** of any repository, and `ProjectConfig` has no `workspac - **`packages/cloud-api`** — the Railway-deployed control plane. Fastify + Postgres. Stores config, the agent registry, and run history; sends Slack notifications. - **`packages/cloud-dashboard`** — the React app over the cloud user API. Vite + - React 19, built on `@16-bits-design/ui`. Named as a sibling of `cloud-api` because + React 19, built on `@16-bits-design/ui`. Organization pages render behind a + **second-level rail** (`components/OrgRail.tsx`, the design's `showOrgRail`), + mounted by `AppShell` only on those paths. Named as a sibling of `cloud-api` because both are hosted; the runner also serves an API, so an unqualified `api` would be ambiguous. Deployed to Railway as its own service; see `docs/dashboard.md`. @@ -100,7 +102,7 @@ harmless where a loss is not. | File | Purpose | |---|---| -| `config.json` | Cloud credentials, Hermes profiles and keys, secrets (v2 schema) | +| `config.json` | Cloud credentials, Hermes profiles and keys, fallback tracker secrets (v2 schema) | | `routes.json` | Last known good routes; the offline fallback, and the whole route table when no cloud is configured | | `running.json` | Pid and port of the running runner | | `sentinel0.db` | SQLite — runs, run events, dispatch ledger, observations, mirror outbox | @@ -150,7 +152,7 @@ Two invariants the dispatcher enforces: agent was reached releases the claim so a fix can run. **Two GitHub identities, and only one of them is Sentinel0's.** The *runner's* own -`gh` does all the polling, labelling and commenting; each *agent* authenticates as +**token** does all the polling, labelling and commenting; each *agent* authenticates as itself inside its Hermes profile, and Sentinel0 passes it no token — `createRun` sends prompt, instructions, session and model, nothing else. `HermesProfileConfig.githubLogin` is therefore the operator's *declaration* of which account a profile uses, verified by @@ -168,6 +170,55 @@ nothing will retry it and the person who pressed the button is owed the reason. records `routeId: 'manual'` and `triggerType: 'manual'`; inventing a plausible route id would be worse than saying there was not one. +### Tracker credentials (`packages/common/src/github.ts`, `orchestrator/src/integrations/`) + +Sentinel0 **does not shell out to `gh`**. It did, and the machine it runs on stopped +being able to install it; the six calls it actually made are plain REST, so +`GitHubRestClient` makes them over `fetch`. It lives in `common` rather than the +orchestrator because the cloud API needs the same three calls to answer the +dashboard's repository and label pickers, and two clients would be two places that +know GitHub's field names. + +The token arrives through a **provider function**, never a constructor string. A PAT +is a constant and would not need one; a GitHub App installation token expires roughly +hourly, and the provider is the shape that serves both — moving to an App changes how +a token is obtained and nothing else in the file. + +Two REST behaviours that are not `gh` behaviours, and both would fail silently: + +- `GET /repos/{o}/{r}/issues` **includes pull requests**; `gh issue list` did not. + Unfiltered, every open PR also raises a `ticket` trigger. +- `gh`'s `reviewRequests` merged users and teams; REST splits them into + `requested_reviewers` and `requested_teams`. Reading only the first makes a route + targeting a review-owning team silently never match. + +`getPullRequestDiff` is **gone**, not ported. It had no callers after the Hermes +pivot: fetching a diff is the agent's job under the boundary, and re-adding it here +would cross that boundary in the wrong direction. What Sentinel0 still needs from +GitHub is five calls — list issues, list pulls, ensure/add/remove labels, and post a +comment. Labels are the non-negotiable half: `sentinel0:in-progress` is the loop +guard, and a runner that cannot write labels re-fires every route on every cycle. +The comment exists for `postFailure`, the one case an agent structurally cannot +report on its own behalf. + +**Credentials are cloud-owned, and the runner still calls GitHub itself.** The cloud +is a *credential broker*, not a data proxy: `GET /v1/runner/integrations` hands the +runner its tokens, and the runner polls `api.github.com` directly. Proxying the data +too would have made trigger collection stop whenever Railway did, which is precisely +what `routes.json`'s offline cache exists to prevent. + +`IntegrationStore` holds them **in memory only**. Routes and projects cache to +`~/.sentinel0/` so an outage cannot stop dispatch, and the same argument applies here +— but a decrypted PAT in a file is a worse trade than a runner that needs its control +plane once at boot. An operator who wants to survive a restart mid-outage sets +`GITHUB_TOKEN` (or `LINEAR_API_KEY`), which is the fallback, not an override: the +cloud wins whenever it has an answer, or a rotated credential would lose to a stale +local one. A failed refresh keeps what was already loaded, for the same reason. + +Precedence is **project override, then organization default**, resolved by +`resolveIntegration` in `common` so the cloud and the dashboard cannot disagree +about it. + ### Route catalog (`packages/common/src/route-catalog.ts`) The supported cases are declared once, as complete routes, and served from @@ -210,6 +261,19 @@ curl, sending no preflight, works perfectly. Migrations are plain `.sql` files applied in filename order, one transaction each. +`SENTINEL0_SECRET_KEY` is **required** once any integration is stored: 32 random +bytes, base64 or hex, encrypting tracker credentials with AES-256-GCM. `crypto.ts` +throws rather than deriving a key from nothing, because a deployment that quietly +encrypted every token under a guessable constant would look exactly like a working +one. `GET /v1/runner/integrations` is the only endpoint that returns a plaintext +secret, and is runner-scoped for precisely that reason — the `snt_usr_` key a browser +holds is rejected before the handler runs. + +`/v1/integrations/slack` stays a static route and so still wins over the parametric +`/v1/integrations/:provider`. Slack is deliberately not in the `integrations` table: +it authenticates nothing and is never handed to a runner, so folding it in would have +meant a column that is null for one provider and required for the others. + `DASHBOARD_URL` is optional and only used to deep-link a run from Slack. Without it the needs-approval message still names the `sentinel0 approve` command — a notification that says a decision is required and no way to make one is the failure this replaced. diff --git a/README.md b/README.md index 4a27bfc..889bc63 100644 --- a/README.md +++ b/README.md @@ -75,7 +75,7 @@ Full walkthrough: **[docs/getting-started.md](./docs/getting-started.md)** npm install -g sentinel0 sentinel0 init # cloud key, Hermes profiles — each key is probed as you enter it -sentinel0 preflight # Node, Hermes, cloud, gh auth +sentinel0 preflight # Node, Hermes, cloud, tracker credentials sentinel0 start sentinel0 runner install # survive reboots (launchd) ``` @@ -115,7 +115,7 @@ sentinel0 run --agent product --prompt "Reply with the word ready." - **Node.js >= 22.5** — the CLI re-executes itself under a compatible interpreter if the active one cannot load `node:sqlite` - **Postgres**, for the control plane -- `gh`, authenticated, if any project pulls from GitHub +- A GitHub token, added under Settings → Integrations, if any project pulls from GitHub ## Repository layout diff --git a/docs/api.md b/docs/api.md index 145d4a2..055996c 100644 --- a/docs/api.md +++ b/docs/api.md @@ -539,6 +539,86 @@ predating the heartbeat still reads as alive on the strength of its long poll al --- +## Integrations + +The tracker credentials the organization authenticates with. One per provider, plus +an optional override per project. + +```http +GET /v1/integrations +PUT /v1/integrations/:provider { "token": "github_pat_…", "projectId": null } +DELETE /v1/integrations/:provider?projectId=acme%2Fplatform +``` + +`:provider` is `github` or `linear`. `projectId` null — or absent — is the +organization default; naming a project stores an override for that project alone. +**A project override wins over the organization default**, so one token can cover +everything while a project that needs a different account carries its own. + +`PUT` **verifies the token against the provider before storing it**, and returns the +account it authenticated as: + +```json +{ "ok": true, "accountLogin": "sentinel0-bot", "scopes": [] } +``` + +A credential saved unverified fails later, on a poll cycle, as a 401 in a log nobody +is reading — while whoever pasted it has already left the screen believing it worked. +A rejected token comes back as a `400` naming the provider's own complaint. + +`GET` never returns a token. It reports a prefix, the account, when it was added and +whether its last use failed: + +```json +{ + "integrations": [ + { + "provider": "github", + "projectId": null, + "tokenPrefix": "github_pat_1", + "accountLogin": "sentinel0-bot", + "scopes": [], + "createdAt": "2026-09-01T10:12:04.221Z", + "updatedAt": "2026-09-01T10:12:04.221Z", + "lastVerifiedAt": "2026-09-01T10:12:04.221Z", + "lastError": null + } + ] +} +``` + +Tokens are encrypted at rest with AES-256-GCM under `SENTINEL0_SECRET_KEY`. That +variable is **required** before any credential can be stored; see +[deploy-cloud.md](./deploy-cloud.md). + +### What GitHub scopes are needed + +A fine-grained PAT with **Issues: read and write** and **Pull requests: read**. +Nothing more. Sentinel0 never pushes code, opens pull requests or leaves reviews — +the agents do all of that under their own accounts inside Hermes. The *write* half of +Issues is not optional: it is what sets the `sentinel0:in-progress` marker, and +without the marker every route re-fires on every cycle. + +`scopes` is empty for a fine-grained token, because GitHub sends no scope header for +one. Empty means "not reported", never "no permissions". + +### Reading GitHub on the dashboard's behalf + +```http +GET /v1/integrations/github/repositories +GET /v1/integrations/github/labels?repo=acme/platform +``` + +Both call GitHub with the stored credential and return only names. They exist because +a browser must never hold a token that grants writing to someone's repositories, so +the API asks on its behalf — which is what lets *Add project* offer a repository +picker and the repository's real labels instead of a free-form JSON field. + +Both answer `409` when no credential is stored, which the dashboard renders as the +prerequisite it is rather than as an error. + +--- + ## Slack ```http @@ -579,6 +659,7 @@ POST /v1/runner/hello register; resets started_at POST /v1/runner/heartbeat periodic health, once per poll cycle PUT /v1/runner/inventory publish discovered agents GET /v1/runner/projects pull ticket sources to watch +GET /v1/runner/integrations pull tracker credentials, decrypted GET /v1/runner/routes pull enabled routes GET /v1/runner/commands?cursor=&wait=&runner= long poll, up to 30s POST /v1/runner/commands/ack { cursor, runner } @@ -587,6 +668,13 @@ PATCH /v1/runner/runs/:id mirror a status change POST /v1/runner/runs/:id/events mirror log events ``` +`GET /v1/runner/integrations` is the only endpoint anywhere that returns a plaintext +secret, and it is runner-scoped for exactly that reason: a `snt_usr_` key — the kind a +browser holds — is rejected before the handler runs. The runner keeps what it receives +in memory only and calls `api.github.com` itself, so the cloud brokers the credential +rather than proxying the data. Trigger collection therefore survives a cloud outage, +the same way cached routes do. + `GET /v1/runner/commands` is held open until something arrives or the window closes. An empty array is the normal, healthy result — not an error. This is how a runner behind NAT receives work without any inbound connection. diff --git a/docs/dashboard.md b/docs/dashboard.md index 2a9d41a..7355ca7 100644 --- a/docs/dashboard.md +++ b/docs/dashboard.md @@ -230,6 +230,28 @@ be saved into a state the runner would reject. - **No renaming an organization.** Its name is set by `org-cli.js` at creation and there is no endpoint to change it. +## The organization rail + +Organization pages — Settings, Integrations, Projects, Access keys, Runners — render +behind a second-level nav between the sidebar and the page. It is the design's +`showOrgRail`, and it exists because that group had outgrown a flat list: five +destinations, each a settings surface rather than a view of work, would have buried +Runs and Routes among them in the primary sidebar. + +`components/OrgRail.tsx` owns both the entries and `isOrgPath`, which `AppShell` uses +to mount it. Each entry carries a hint under its label, because these pages are +visited rarely enough that "Settings" and "Projects" do not by themselves say which +one holds the thing you came for. + +Adding a page to the group means adding one entry to `ORG_RAIL` and one ``; +the rail's visibility follows from the entry rather than from a second list that +could disagree with it. + +The old Settings screen carried the organization, Slack, **and** the runner table. +Slack moved to Integrations, beside the tracker credentials it belongs with, and the +runners to a page of their own — it was the only table on a page of forms, and "is +the Mac Mini still there" is asked far more often than anything else that page held. + ## Design system The UI is built on [`@16-bits-design/ui`](https://github.com/maxigimenez/16-bits-design), diff --git a/docs/deploy-cloud.md b/docs/deploy-cloud.md index fdfd80c..a3eb5bf 100644 --- a/docs/deploy-cloud.md +++ b/docs/deploy-cloud.md @@ -135,7 +135,22 @@ In the service's Variables tab, reference the database: DATABASE_URL = ${{Postgres.DATABASE_URL}} ``` -That is the only required variable. Optional ones: +`SENTINEL0_SECRET_KEY` is required as well, before any tracker credential can be +stored. Thirty-two random bytes, base64 or hex: + +```bash +node -e "console.log(require('crypto').randomBytes(32).toString('base64'))" +``` + +It encrypts GitHub and Linear tokens at rest with AES-256-GCM. The API refuses to +start storing credentials without it rather than deriving a key from nothing — a +deployment that quietly encrypted every token under a guessable constant would look +exactly like a working one. + +**Losing or rotating this key makes every stored credential unreadable.** They are not +recoverable; re-add them under Settings → Integrations. + +Optional variables: | Variable | Default | Purpose | |---|---|---| diff --git a/docs/getting-started.md b/docs/getting-started.md index 341548e..5f9961a 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -222,7 +222,7 @@ profile: a role, a GitHub login (for PR-review routes), and an avatar image URL in Slack). ```bash -sentinel0 preflight # Node, Hermes profiles, cloud, gh auth +sentinel0 preflight # Node, Hermes profiles, cloud, tracker credentials sentinel0 start sentinel0 status ``` diff --git a/docs/routes.md b/docs/routes.md index 73ac411..961e3be 100644 --- a/docs/routes.md +++ b/docs/routes.md @@ -95,14 +95,19 @@ rejects it on a `ticket` route, which could never fire. There are two, and Sentinel0 only owns one of them. -- **The runner's `gh`.** Every poll, every label, every comment Sentinel0 writes goes - through whatever `gh auth login` was run on the runner's machine. This account needs - write access to the repositories you watch, or the `sentinel0:in-progress` marker - cannot be set and routes will fire twice. +- **Sentinel0's own token,** stored under **Settings → Integrations** in the + dashboard. Every poll, every label and every comment Sentinel0 writes goes through + it. A fine-grained PAT with **Issues: read and write** and **Pull requests: read** + is enough — Sentinel0 never pushes code. The write half is not optional: without it + the `sentinel0:in-progress` marker cannot be set and routes fire on every cycle. - **The agent's own `gh`,** inside its Hermes profile. This is who opens the pull request and leaves the review. Sentinel0 passes Hermes **no** GitHub token — the agent's credentials are Hermes' business entirely. +Sentinel0 stopped shelling out to `gh` in favour of the REST API, so the runner's +machine needs no GitHub CLI and no `gh auth login`. The agents still use `gh` inside +their own profiles, which is Hermes' concern rather than Sentinel0's. + `githubLogin` in `~/.sentinel0/config.json` is your *declaration* of which account the second one is. Nothing verifies it. Set it wrong and login-targeted routes silently never match; leave it unset and `sentinel0 preflight` says so. It cannot be set from diff --git a/packages/cli/src/commands/init.ts b/packages/cli/src/commands/init.ts index 38fe016..0d9707e 100644 --- a/packages/cli/src/commands/init.ts +++ b/packages/cli/src/commands/init.ts @@ -237,18 +237,38 @@ export async function runInit(context: CliContext): Promise { return } - // ── Secrets ────────────────────────────────────────────── + /* + * ── Secrets ────────────────────────────────────────────── + * + * Tracker credentials belong in the cloud now, under Settings → + * Integrations: one place for the whole organization, encrypted at rest, and + * the only way the dashboard can offer a repository picker. What stays here + * is a *fallback*, for the two cases the cloud cannot serve -- a runner with + * no control plane, and a runner restarting while the cloud is unreachable. + * + * So this asks once, and only if the operator says they want one. + */ const secrets = { ...existing.secrets } - const needsLinear = assertNotCancel( - await p.confirm({ message: 'Will any project pull from Linear?', initialValue: false }) + const wantsLocalTokens = assertNotCancel( + await p.confirm({ + message: 'Store tracker credentials on this machine as a fallback?', + initialValue: false, + }) ) - if (needsLinear && !secrets.LINEAR_API_KEY) { - secrets.LINEAR_API_KEY = assertNotCancel( - await p.password({ - message: 'Linear API key', - validate: (value) => (value?.trim() ? undefined : 'Required.'), - }) + if (wantsLocalTokens) { + const githubToken = assertNotCancel( + await p.password({ message: 'GitHub token (blank to skip)' }) + ) + if (githubToken?.trim()) { + secrets.GITHUB_TOKEN = githubToken.trim() + } + + const linearKey = assertNotCancel( + await p.password({ message: 'Linear API key (blank to skip)' }) ) + if (linearKey?.trim()) { + secrets.LINEAR_API_KEY = linearKey.trim() + } } const config: StoredConfig = { diff --git a/packages/cli/src/commands/preflight.ts b/packages/cli/src/commands/preflight.ts index 590e4fc..c7594d2 100644 --- a/packages/cli/src/commands/preflight.ts +++ b/packages/cli/src/commands/preflight.ts @@ -1,17 +1,8 @@ -import { spawn } from 'node:child_process' import chalk from 'chalk' import type { CliContext, VerifyCheck } from '../types.js' import { getJson } from '../api.js' import { findCapableNode } from '../node-runtime.js' -async function commandSucceeds(cmd: string, args: string[]): Promise { - return new Promise((resolve) => { - const child = spawn(cmd, args, { stdio: 'ignore' }) - child.on('error', () => resolve(false)) - child.on('close', (code) => resolve(code === 0)) - }) -} - /** * Checks exactly what this runner needs. * @@ -75,29 +66,34 @@ export async function runPreflight(context: CliContext): Promise { checks.push({ name: 'Sentinel0 cloud reachable', ok, required: true, detail }) } - const ghInstalled = await commandSucceeds('gh', ['--version']) + /* + * Credentials, not command-line tools. + * + * This used to check for `gh` and `gh auth status`. Sentinel0 no longer + * shells out to the GitHub CLI, and the machine it runs on may not be able + * to install it -- which is what prompted the change. What matters now is + * whether a token exists, and the authoritative copy of that lives in the + * cloud, so a missing local one is only worth reporting as the fallback it + * is. + */ + const localGitHub = Boolean(process.env.GITHUB_TOKEN || config.secrets.GITHUB_TOKEN) checks.push({ - name: 'GitHub CLI installed', - ok: ghInstalled, + name: 'GITHUB_TOKEN (local fallback)', + ok: localGitHub, required: false, - detail: ghInstalled ? '' : 'Needed only for GitHub projects.', + detail: localGitHub + ? '' + : 'Optional. Settings → Integrations in the dashboard is the usual place.', }) - if (ghInstalled) { - const authed = await commandSucceeds('gh', ['auth', 'status']) - checks.push({ - name: 'GitHub CLI authenticated', - ok: authed, - required: false, - // Only show the remedy when there is something to remedy. - detail: authed ? '' : 'gh auth login', - }) - } + const localLinear = Boolean(process.env.LINEAR_API_KEY || config.secrets.LINEAR_API_KEY) checks.push({ - name: 'LINEAR_API_KEY', - ok: Boolean(process.env.LINEAR_API_KEY || config.secrets.LINEAR_API_KEY), + name: 'LINEAR_API_KEY (local fallback)', + ok: localLinear, required: false, - detail: 'Needed only for Linear projects.', + detail: localLinear + ? '' + : 'Optional. Settings → Integrations in the dashboard is the usual place.', }) console.log('') diff --git a/packages/cli/vitest.config.ts b/packages/cli/vitest.config.ts index bd36c92..97fde25 100644 --- a/packages/cli/vitest.config.ts +++ b/packages/cli/vitest.config.ts @@ -4,8 +4,14 @@ import path from 'node:path' export default defineConfig({ resolve: { alias: { - '@sentinel0/common': path.resolve(__dirname, '../common/src/index.ts'), + /* + * Subpaths first. Vite matches a string alias as a prefix, so a bare + * '@sentinel0/common' listed above these would rewrite + * '@sentinel0/common/github' into 'index.ts/github'. + */ '@sentinel0/common/executor': path.resolve(__dirname, '../common/src/executor.ts'), + '@sentinel0/common/github': path.resolve(__dirname, '../common/src/github.ts'), + '@sentinel0/common': path.resolve(__dirname, '../common/src/index.ts'), }, }, }) diff --git a/packages/cloud-api/src/crypto.ts b/packages/cloud-api/src/crypto.ts new file mode 100644 index 0000000..b8f724d --- /dev/null +++ b/packages/cloud-api/src/crypto.ts @@ -0,0 +1,104 @@ +import { createCipheriv, createDecipheriv, randomBytes } from 'node:crypto' + +/** + * Symmetric encryption for the credentials the cloud stores on an org's behalf. + * + * The Slack webhook that preceded these is stored as plaintext and only ever + * reported as "configured". That was a defensible trade for a URL whose worst + * case is posting into a channel; it is not one for a token that can write to + * every repository an operator can reach. + * + * Two-way, not a hash, because the point is to hand the plaintext back to the + * runner. What encryption buys is that a leaked database dump is not a leaked + * set of GitHub tokens -- the key lives in the environment, not in Postgres. + * + * AES-256-GCM: authenticated, so a tampered ciphertext fails to decrypt rather + * than yielding a plausible wrong token. + */ + +const ALGORITHM = 'aes-256-gcm' +const IV_BYTES = 12 +const KEY_BYTES = 32 + +/** `v1` names the scheme, so a future rotation can be told apart on sight. */ +const VERSION = 'v1' + +let cachedKey: Buffer | undefined + +/** + * The key, read once from `SENTINEL0_SECRET_KEY`. + * + * 32 bytes, base64 or hex. Generate one with: + * node -e "console.log(require('crypto').randomBytes(32).toString('base64'))" + * + * Throws rather than falling back to a default or a derived-from-nothing key: + * a deployment that quietly encrypted every credential under a guessable + * constant would look exactly like a working one. + */ +export function encryptionKey(): Buffer { + if (cachedKey) { + return cachedKey + } + + const raw = process.env.SENTINEL0_SECRET_KEY + if (!raw) { + throw new Error( + 'SENTINEL0_SECRET_KEY is required to store integration credentials. ' + + "Generate one with: node -e \"console.log(require('crypto').randomBytes(32).toString('base64'))\"" + ) + } + + const key = /^[0-9a-f]{64}$/i.test(raw) ? Buffer.from(raw, 'hex') : Buffer.from(raw, 'base64') + if (key.length !== KEY_BYTES) { + throw new Error( + `SENTINEL0_SECRET_KEY must decode to ${KEY_BYTES} bytes; got ${key.length}. ` + + 'Provide 32 random bytes as base64 or hex.' + ) + } + + cachedKey = key + return key +} + +/** Test seam: forget the cached key so a changed env var is picked up. */ +export function resetEncryptionKey(): void { + cachedKey = undefined +} + +/** `v1...`, all base64url. */ +export function encrypt(plaintext: string): string { + const iv = randomBytes(IV_BYTES) + const cipher = createCipheriv(ALGORITHM, encryptionKey(), iv) + const ciphertext = Buffer.concat([cipher.update(plaintext, 'utf8'), cipher.final()]) + return [ + VERSION, + iv.toString('base64url'), + cipher.getAuthTag().toString('base64url'), + ciphertext.toString('base64url'), + ].join('.') +} + +export function decrypt(encoded: string): string { + const [version, iv, authTag, ciphertext] = encoded.split('.') + if (version !== VERSION || !iv || !authTag || !ciphertext) { + throw new Error('Stored credential is not in a format this build can read.') + } + + const decipher = createDecipheriv(ALGORITHM, encryptionKey(), Buffer.from(iv, 'base64url')) + decipher.setAuthTag(Buffer.from(authTag, 'base64url')) + return Buffer.concat([ + decipher.update(Buffer.from(ciphertext, 'base64url')), + decipher.final(), + ]).toString('utf8') +} + +/** + * The visible part of a token. + * + * Enough to recognise which credential is installed, short enough to be + * useless on its own. GitHub's own prefixes (`ghp_`, `github_pat_`) are longer + * than four characters, so the type stays legible. + */ +export function tokenPrefix(token: string): string { + return token.slice(0, 12) +} diff --git a/packages/cloud-api/src/integrations.ts b/packages/cloud-api/src/integrations.ts new file mode 100644 index 0000000..637eabf --- /dev/null +++ b/packages/cloud-api/src/integrations.ts @@ -0,0 +1,235 @@ +import { + INTEGRATION_PROVIDER, + type IntegrationProvider, + type IntegrationSummary, +} from '@sentinel0/common' +import { GitHubRestClient } from '@sentinel0/common/github' +import { decrypt, encrypt, tokenPrefix } from './crypto.js' +import type { Database } from './db.js' + +/** + * Storage and verification for the credentials an organization holds. + * + * Kept out of `routes/user.ts` because both APIs need it: the user API to save + * and list, the runner API to hand the plaintext to a runner. Encryption is + * applied here rather than at either call site, so there is exactly one place + * a plaintext token can enter or leave the database. + */ + +interface IntegrationRow { + id: string + provider: IntegrationProvider + project_id: string | null + token_ciphertext: string + token_prefix: string + account_login: string | null + scopes: string[] + created_at: Date + updated_at: Date + last_verified_at: Date | null + last_error: string | null +} + +export interface VerifiedIdentity { + login: string + scopes: string[] +} + +/** + * Confirms a token works before it is stored. + * + * A credential that is saved unverified fails later, on a poll cycle, as a 401 + * in a log nobody is reading -- and the person who pasted it has already left + * the screen believing it worked. Verifying at save time turns that into an + * error beside the field. + */ +export async function verifyToken( + provider: IntegrationProvider, + token: string +): Promise { + if (provider === INTEGRATION_PROVIDER.GITHUB) { + return new GitHubRestClient(async () => token).identity() + } + + const response = await fetch('https://api.linear.app/graphql', { + method: 'POST', + headers: { 'content-type': 'application/json', authorization: token }, + body: JSON.stringify({ query: '{ viewer { email } }' }), + }) + if (!response.ok) { + throw new Error(`Linear rejected the API key (${response.status}).`) + } + const body = (await response.json()) as { + data?: { viewer?: { email?: string } } + errors?: Array<{ message?: string }> + } + if (body.errors?.length) { + throw new Error(body.errors.map((error) => error.message ?? 'unknown').join('; ')) + } + // Linear has no scope introspection, so the empty list means "not reported" + // here exactly as it does for a fine-grained GitHub token. + return { login: body.data?.viewer?.email ?? 'unknown', scopes: [] } +} + +export async function listIntegrations(db: Database, orgId: string): Promise { + const { rows } = await db.query( + `SELECT id, provider, project_id, token_ciphertext, token_prefix, account_login, + scopes, created_at, updated_at, last_verified_at, last_error + FROM integrations WHERE org_id = $1 + ORDER BY provider, project_id NULLS FIRST`, + [orgId] + ) + return rows.map(toSummary) +} + +export async function saveIntegration( + db: Database, + orgId: string, + input: { + provider: IntegrationProvider + projectId: string | null + token: string + identity: VerifiedIdentity + } +): Promise { + // Two statements, not one, because the uniqueness is two partial indexes: + // Postgres does not treat NULLs as equal, so a single + // `ON CONFLICT (org_id, provider, project_id)` would never fire for the + // organization default and would let an org accumulate any number of them. + // Each branch names the index whose predicate it matches. + if (input.projectId === null) { + await db.query( + `INSERT INTO integrations + (id, org_id, provider, project_id, token_ciphertext, token_prefix, + account_login, scopes, last_verified_at) + VALUES ($1,$2,$3,NULL,$4,$5,$6,$7,now()) + ON CONFLICT (org_id, provider) WHERE project_id IS NULL DO UPDATE SET + token_ciphertext = EXCLUDED.token_ciphertext, + token_prefix = EXCLUDED.token_prefix, + account_login = EXCLUDED.account_login, + scopes = EXCLUDED.scopes, + updated_at = now(), + last_verified_at = now(), + last_error = NULL`, + [ + `int_${input.provider}_${orgId}`, + orgId, + input.provider, + encrypt(input.token), + tokenPrefix(input.token), + input.identity.login, + input.identity.scopes, + ] + ) + return + } + + await db.query( + `INSERT INTO integrations + (id, org_id, provider, project_id, token_ciphertext, token_prefix, + account_login, scopes, last_verified_at) + VALUES ($1,$2,$3,$4,$5,$6,$7,$8,now()) + ON CONFLICT (org_id, provider, project_id) WHERE project_id IS NOT NULL DO UPDATE SET + token_ciphertext = EXCLUDED.token_ciphertext, + token_prefix = EXCLUDED.token_prefix, + account_login = EXCLUDED.account_login, + scopes = EXCLUDED.scopes, + updated_at = now(), + last_verified_at = now(), + last_error = NULL`, + [ + `int_${input.provider}_${orgId}_${input.projectId}`, + orgId, + input.provider, + input.projectId, + encrypt(input.token), + tokenPrefix(input.token), + input.identity.login, + input.identity.scopes, + ] + ) +} + +export async function deleteIntegration( + db: Database, + orgId: string, + provider: IntegrationProvider, + projectId: string | null +): Promise { + await db.query( + projectId === null + ? 'DELETE FROM integrations WHERE org_id = $1 AND provider = $2 AND project_id IS NULL' + : 'DELETE FROM integrations WHERE org_id = $1 AND provider = $2 AND project_id = $3', + projectId === null ? [orgId, provider] : [orgId, provider, projectId] + ) +} + +/** + * The plaintext credential that applies to a project. + * + * Used by the runner endpoint and by the dashboard's repository picker. The + * precedence -- project override, then organization default -- is expressed + * once, in SQL, by ordering the two candidate rows and taking the first. + */ +export async function resolveToken( + db: Database, + orgId: string, + provider: IntegrationProvider, + projectId: string | null +): Promise { + const { rows } = await db.query<{ token_ciphertext: string }>( + `SELECT token_ciphertext FROM integrations + WHERE org_id = $1 AND provider = $2 + AND (project_id IS NULL OR project_id = $3) + ORDER BY project_id NULLS LAST + LIMIT 1`, + [orgId, provider, projectId] + ) + return rows[0] ? decrypt(rows[0].token_ciphertext) : undefined +} + +/** Every credential this org holds, decrypted, for the runner to cache. */ +export async function resolveAllTokens( + db: Database, + orgId: string +): Promise> { + const { rows } = await db.query( + 'SELECT provider, project_id, token_ciphertext FROM integrations WHERE org_id = $1', + [orgId] + ) + return rows.map((row) => ({ + provider: row.provider, + projectId: row.project_id, + token: decrypt(row.token_ciphertext), + })) +} + +/** Records a verification failure without discarding the credential. */ +export async function recordFailure( + db: Database, + orgId: string, + provider: IntegrationProvider, + projectId: string | null, + error: string +): Promise { + await db.query( + projectId === null + ? `UPDATE integrations SET last_error = $3 WHERE org_id = $1 AND provider = $2 AND project_id IS NULL` + : `UPDATE integrations SET last_error = $4 WHERE org_id = $1 AND provider = $2 AND project_id = $3`, + projectId === null ? [orgId, provider, error] : [orgId, provider, projectId, error] + ) +} + +function toSummary(row: IntegrationRow): IntegrationSummary { + return { + provider: row.provider, + projectId: row.project_id, + tokenPrefix: row.token_prefix, + accountLogin: row.account_login, + scopes: row.scopes ?? [], + createdAt: row.created_at.toISOString(), + updatedAt: row.updated_at.toISOString(), + lastVerifiedAt: row.last_verified_at?.toISOString() ?? null, + lastError: row.last_error, + } +} diff --git a/packages/cloud-api/src/migrations/007_integrations.sql b/packages/cloud-api/src/migrations/007_integrations.sql new file mode 100644 index 0000000..bc4917c --- /dev/null +++ b/packages/cloud-api/src/migrations/007_integrations.sql @@ -0,0 +1,44 @@ +-- Tracker credentials, held for the organization rather than on each machine. +-- +-- GitHub access used to be `gh`'s own login on the runner's box and Linear's +-- key a line in `~/.sentinel0/config.json`. Neither survives a machine that +-- cannot install `gh`, and neither lets the dashboard offer a repository +-- picker, because a browser must never hold a token that grants writing to +-- someone's repositories. +-- +-- `project_id` NULL is the organization default; a row naming a project is an +-- override for that project alone. Postgres does not treat NULLs as equal in a +-- unique constraint, so the uniqueness is expressed as two partial indexes -- +-- one over the default row, one over the per-project rows. A single +-- `UNIQUE (org_id, provider, project_id)` would silently permit any number of +-- competing organization defaults. +CREATE TABLE IF NOT EXISTS integrations ( + id TEXT PRIMARY KEY, + org_id TEXT NOT NULL REFERENCES organizations(id) ON DELETE CASCADE, + provider TEXT NOT NULL CHECK (provider IN ('github', 'linear')), + project_id TEXT REFERENCES projects(id) ON DELETE CASCADE, + + -- AES-256-GCM, keyed by SENTINEL0_SECRET_KEY. Never returned by the user API. + token_ciphertext TEXT NOT NULL, + -- The first few characters, so a person can tell which token is installed + -- without the API handing back one that can write to their repositories. + token_prefix TEXT NOT NULL, + + -- Resolved against the provider when the credential was saved, so a route + -- targeting an agent by GitHub identity can be checked against something. + account_login TEXT, + scopes TEXT[] NOT NULL DEFAULT '{}', + + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT now(), + last_verified_at TIMESTAMPTZ, + last_error TEXT +); + +CREATE UNIQUE INDEX IF NOT EXISTS idx_integrations_org_default + ON integrations(org_id, provider) + WHERE project_id IS NULL; + +CREATE UNIQUE INDEX IF NOT EXISTS idx_integrations_org_project + ON integrations(org_id, provider, project_id) + WHERE project_id IS NOT NULL; diff --git a/packages/cloud-api/src/routes/runner.ts b/packages/cloud-api/src/routes/runner.ts index 9f5d6e9..7748adf 100644 --- a/packages/cloud-api/src/routes/runner.ts +++ b/packages/cloud-api/src/routes/runner.ts @@ -2,6 +2,7 @@ import type { FastifyInstance } from 'fastify' import type { AgentDescriptor, RoutingRule, RunLogEntry, RunRecord } from '@sentinel0/common' import { authenticate, newId, parseBearer, type AuthContext } from '../auth.js' import type { Database } from '../db.js' +import { resolveAllTokens } from '../integrations.js' import { notifyRunEvent } from '../notifications/slack.js' /** How long a command long-poll may be held open. */ @@ -222,6 +223,25 @@ export function registerRunnerRoutes(app: FastifyInstance, db: Database): void { return { projects: rows } }) + // ── Integrations ─────────────────────────────────────────── + + /** + * The org's tracker credentials, decrypted. + * + * This is the one endpoint that returns a plaintext secret, and it is + * runner-scoped for that reason: a `snt_usr_` key -- the one a browser holds + * -- is rejected by the auth hook before this handler runs. + * + * Every credential is sent, the organization default and each project + * override alike, rather than one per request keyed by project: the runner + * watches every project in the org anyway, and resolving them one at a time + * would put a round trip in front of each poll cycle. + */ + app.get('/v1/runner/integrations', async (request) => { + const { orgId } = authOf(request) + return { integrations: await resolveAllTokens(db, orgId) } + }) + // ── Routes ───────────────────────────────────────────────── app.get('/v1/runner/routes', async (request) => { diff --git a/packages/cloud-api/src/routes/user.ts b/packages/cloud-api/src/routes/user.ts index e1c8e77..9dbb1c6 100644 --- a/packages/cloud-api/src/routes/user.ts +++ b/packages/cloud-api/src/routes/user.ts @@ -10,8 +10,18 @@ import { validateRoutingRule, type RoutingRule, } from '@sentinel0/common' +import { isIntegrationProvider, INTEGRATION_PROVIDER } from '@sentinel0/common' +import { GitHubRestClient } from '@sentinel0/common/github' import { authenticate, generateKey, newId, parseBearer, type AuthContext } from '../auth.js' import type { Database } from '../db.js' +import { + deleteIntegration, + listIntegrations, + recordFailure, + resolveToken, + saveIntegration, + verifyToken, +} from '../integrations.js' /** * A ceiling on a hand-written prompt. @@ -498,6 +508,171 @@ export function registerUserRoutes(app: FastifyInstance, db: Database): void { return reply.code(202).send({ queued: commandId }) }) + // ── Tracker credentials ──────────────────────────────────── + + /* + * `/v1/integrations/slack` below is a static route and wins over this + * parametric one, which is why Slack keeps its own shape. Slack is a + * webhook rather than a tracker credential -- it authenticates nothing and + * is never handed to a runner -- so folding it into this table would have + * meant a column that is null for one provider and required for the others. + */ + + app.get('/v1/integrations', async (request) => { + const { orgId } = authOf(request) + return { integrations: await listIntegrations(db, orgId) } + }) + + /** + * Saves a credential for the org, or for one project. + * + * The token is verified against the provider before it is stored. A + * credential saved unverified fails later, on a poll cycle, as a 401 in a + * log nobody reads -- while the person who pasted it has already left the + * screen believing it worked. + */ + app.put('/v1/integrations/:provider', async (request, reply) => { + const { orgId } = authOf(request) + const { provider } = request.params as { provider: string } + const body = request.body as { token?: string; projectId?: string | null } + + if (!isIntegrationProvider(provider)) { + return reply.code(400).send({ error: `Unknown integration provider "${provider}".` }) + } + if (typeof body.token !== 'string' || body.token.trim().length === 0) { + return reply.code(400).send({ error: 'A token is required.' }) + } + + const projectId = body.projectId?.trim() || null + if (projectId) { + const { rows } = await db.query('SELECT 1 FROM projects WHERE org_id = $1 AND id = $2', [ + orgId, + projectId, + ]) + if (rows.length === 0) { + return reply.code(404).send({ error: `No project "${projectId}" in this organization.` }) + } + } + + let identity + try { + identity = await verifyToken(provider, body.token.trim()) + } catch (error: unknown) { + return reply.code(400).send({ + error: `${provider} rejected that credential: ${error instanceof Error ? error.message : String(error)}`, + }) + } + + await saveIntegration(db, orgId, { + provider, + projectId, + token: body.token.trim(), + identity, + }) + return { ok: true, accountLogin: identity.login, scopes: identity.scopes } + }) + + app.delete('/v1/integrations/:provider', async (request, reply) => { + const { orgId } = authOf(request) + const { provider } = request.params as { provider: string } + const { projectId } = request.query as { projectId?: string } + + if (!isIntegrationProvider(provider)) { + return reply.code(400).send({ error: `Unknown integration provider "${provider}".` }) + } + await deleteIntegration(db, orgId, provider, projectId?.trim() || null) + return { ok: true } + }) + + /** + * Repositories the stored GitHub credential can reach. + * + * This is why the token lives in the cloud rather than only on the runner: + * the dashboard cannot call GitHub itself without shipping a token that + * grants writing to the operator's repositories into a browser tab. So the + * API asks on its behalf and returns only names. + */ + app.get('/v1/integrations/github/repositories', async (request, reply) => { + const { orgId } = authOf(request) + const { projectId } = request.query as { projectId?: string } + + const token = await resolveToken( + db, + orgId, + INTEGRATION_PROVIDER.GITHUB, + projectId?.trim() || null + ) + if (!token) { + return reply.code(409).send({ error: 'No GitHub credential is configured.' }) + } + + try { + const repositories = await new GitHubRestClient(async () => token).repositories() + return { + repositories: repositories.map((repo) => ({ + slug: repo.full_name, + private: repo.private, + })), + } + } catch (error: unknown) { + // Recorded against the credential, not just returned. A token that has + // been revoked upstream is a fact about the credential, and the + // Integrations screen is where someone would look for it -- this is what + // puts it there rather than only in the response to one request. + const message = error instanceof Error ? error.message : String(error) + await recordFailure( + db, + orgId, + INTEGRATION_PROVIDER.GITHUB, + projectId?.trim() || null, + message + ) + return reply.code(502).send({ error: message }) + } + }) + + /** + * A repository's labels, for the project filter picker. + * + * The slug arrives as a query parameter rather than two path segments: a + * repository name may contain a dot and Fastify's path matching is happier + * with one opaque value than with `:owner/:repo` plus escaping rules. + */ + app.get('/v1/integrations/github/labels', async (request, reply) => { + const { orgId } = authOf(request) + const { repo: slug, projectId } = request.query as { repo?: string; projectId?: string } + + const parts = slug?.split('/') ?? [] + if (parts.length !== 2 || !parts[0] || !parts[1]) { + return reply.code(400).send({ error: 'repo must be "owner/name".' }) + } + + const token = await resolveToken( + db, + orgId, + INTEGRATION_PROVIDER.GITHUB, + projectId?.trim() || null + ) + if (!token) { + return reply.code(409).send({ error: 'No GitHub credential is configured.' }) + } + + try { + const labels = await new GitHubRestClient(async () => token).labels(parts[0], parts[1]) + return { labels: labels.map((label) => label.name) } + } catch (error: unknown) { + const message = error instanceof Error ? error.message : String(error) + await recordFailure( + db, + orgId, + INTEGRATION_PROVIDER.GITHUB, + projectId?.trim() || null, + message + ) + return reply.code(502).send({ error: message }) + } + }) + // ── Slack ────────────────────────────────────────────────── app.get('/v1/integrations/slack', async (request) => { diff --git a/packages/cloud-api/test/crypto.test.ts b/packages/cloud-api/test/crypto.test.ts new file mode 100644 index 0000000..4a03585 --- /dev/null +++ b/packages/cloud-api/test/crypto.test.ts @@ -0,0 +1,90 @@ +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { randomBytes } from 'node:crypto' +import { decrypt, encrypt, encryptionKey, resetEncryptionKey, tokenPrefix } from '../src/crypto.js' + +const KEY = randomBytes(32).toString('base64') + +beforeEach(() => { + process.env.SENTINEL0_SECRET_KEY = KEY + resetEncryptionKey() +}) + +afterEach(() => { + delete process.env.SENTINEL0_SECRET_KEY + resetEncryptionKey() +}) + +describe('encryptionKey', () => { + /* + * The failure this guards is silent, not loud: a deployment that quietly + * derived a key from nothing would encrypt every credential under a + * guessable constant and look exactly like a working one. + */ + it('refuses to invent a key', () => { + delete process.env.SENTINEL0_SECRET_KEY + resetEncryptionKey() + expect(() => encryptionKey()).toThrow(/SENTINEL0_SECRET_KEY is required/) + }) + + it('rejects a key that is not 32 bytes', () => { + process.env.SENTINEL0_SECRET_KEY = Buffer.from('too short').toString('base64') + resetEncryptionKey() + expect(() => encryptionKey()).toThrow(/must decode to 32 bytes/) + }) + + it('accepts hex as well as base64', () => { + const hex = randomBytes(32).toString('hex') + process.env.SENTINEL0_SECRET_KEY = hex + resetEncryptionKey() + expect(encryptionKey().toString('hex')).toBe(hex) + }) +}) + +describe('encrypt / decrypt', () => { + it('round-trips a token', () => { + const token = 'github_pat_11ABCDEF0_abcdefghijklmnopqrstuvwxyz' + expect(decrypt(encrypt(token))).toBe(token) + }) + + /* + * A fresh IV per call. Without it two organizations storing the same token + * would produce identical ciphertext, which leaks that they match. + */ + it('produces different ciphertext for the same input', () => { + expect(encrypt('same')).not.toBe(encrypt('same')) + }) + + it('refuses a tampered ciphertext rather than returning a wrong token', () => { + const encoded = encrypt('github_pat_original') + const [version, iv, tag, body] = encoded.split('.') + const flipped = Buffer.from(body, 'base64url') + flipped[0] ^= 0xff + expect(() => decrypt([version, iv, tag, flipped.toString('base64url')].join('.'))).toThrow() + }) + + it('refuses a value stored under a different key', () => { + const encoded = encrypt('github_pat_original') + process.env.SENTINEL0_SECRET_KEY = randomBytes(32).toString('base64') + resetEncryptionKey() + expect(() => decrypt(encoded)).toThrow() + }) + + it('rejects a payload that is not in this scheme', () => { + expect(() => decrypt('not-encrypted-at-all')).toThrow(/not in a format/) + }) +}) + +describe('tokenPrefix', () => { + /* + * Long enough to tell a fine-grained token from a classic one, which is the + * question someone looking at this field is actually asking. + */ + it('keeps enough to identify the token type', () => { + expect(tokenPrefix('github_pat_11ABCDEF0_secret')).toBe('github_pat_1') + expect(tokenPrefix('ghp_abcdefghijklmnop')).toBe('ghp_abcdefgh') + }) + + it('does not pad a short value into looking longer', () => { + expect(tokenPrefix('short')).toBe('short') + }) +}) diff --git a/packages/cloud-api/vitest.config.ts b/packages/cloud-api/vitest.config.ts index e9c2511..97fde25 100644 --- a/packages/cloud-api/vitest.config.ts +++ b/packages/cloud-api/vitest.config.ts @@ -4,6 +4,13 @@ import path from 'node:path' export default defineConfig({ resolve: { alias: { + /* + * Subpaths first. Vite matches a string alias as a prefix, so a bare + * '@sentinel0/common' listed above these would rewrite + * '@sentinel0/common/github' into 'index.ts/github'. + */ + '@sentinel0/common/executor': path.resolve(__dirname, '../common/src/executor.ts'), + '@sentinel0/common/github': path.resolve(__dirname, '../common/src/github.ts'), '@sentinel0/common': path.resolve(__dirname, '../common/src/index.ts'), }, }, diff --git a/packages/cloud-dashboard/src/App.tsx b/packages/cloud-dashboard/src/App.tsx index 6dc459a..ae4dd3d 100644 --- a/packages/cloud-dashboard/src/App.tsx +++ b/packages/cloud-dashboard/src/App.tsx @@ -7,6 +7,7 @@ import { Spinner } from '@16-bits-design/ui/spinner' import { SessionProvider, useSession } from './lib/session.js' import { AccessKeys } from './screens/AccessKeys.js' import { Agents } from './screens/Agents.js' +import { Integrations } from './screens/Integrations.js' import { KeyNew } from './screens/KeyNew.js' import { Login } from './screens/Login.js' import { Overview } from './screens/Overview.js' @@ -17,6 +18,7 @@ import { RouteList } from './screens/RouteList.js' import { RouteNew } from './screens/RouteNew.js' import { RunDetail } from './screens/RunDetail.js' import { RunList } from './screens/RunList.js' +import { Runners } from './screens/Runners.js' import { Settings } from './screens/Settings.js' function Authenticated(): ReactNode { @@ -47,6 +49,8 @@ function Authenticated(): ReactNode { } /> } /> } /> + } /> + } /> } /> diff --git a/packages/cloud-dashboard/src/api/endpoints.ts b/packages/cloud-dashboard/src/api/endpoints.ts index 24321cc..8ede98d 100644 --- a/packages/cloud-dashboard/src/api/endpoints.ts +++ b/packages/cloud-dashboard/src/api/endpoints.ts @@ -2,6 +2,7 @@ import { request } from './client.js' import type { Agent, ApiKey, + Integration, Project, PromptTemplate, RouteTemplate, @@ -122,6 +123,50 @@ export const api = { revokeKey: (key: string, id: string) => request<{ ok: true }>(key, `/v1/keys/${encodeURIComponent(id)}`, { method: 'DELETE' }), + integrations: (key: string, signal?: AbortSignal) => + request<{ integrations: Integration[] }>(key, '/v1/integrations', { signal }).then( + (r) => r.integrations + ), + + saveIntegration: ( + key: string, + provider: string, + body: { token: string; projectId?: string | null } + ) => + request<{ ok: true; accountLogin: string; scopes: string[] }>( + key, + `/v1/integrations/${encodeURIComponent(provider)}`, + { method: 'PUT', body } + ), + + deleteIntegration: (key: string, provider: string, projectId?: string | null) => { + const query = projectId ? `?projectId=${encodeURIComponent(projectId)}` : '' + return request<{ ok: true }>(key, `/v1/integrations/${encodeURIComponent(provider)}${query}`, { + method: 'DELETE', + }) + }, + + /* + * The repository and label pickers. + * + * Both go through the cloud rather than straight to GitHub: the browser + * holds a `snt_usr_` key, never a GitHub token, so the API asks on its + * behalf and returns only names. + */ + gitHubRepositories: (key: string, signal?: AbortSignal) => + request<{ repositories: Array<{ slug: string; private: boolean }> }>( + key, + '/v1/integrations/github/repositories', + { signal } + ).then((r) => r.repositories), + + gitHubLabels: (key: string, repo: string, signal?: AbortSignal) => + request<{ labels: string[] }>( + key, + `/v1/integrations/github/labels?repo=${encodeURIComponent(repo)}`, + { signal } + ).then((r) => r.labels), + slack: (key: string, signal?: AbortSignal) => request(key, '/v1/integrations/slack', { signal }), diff --git a/packages/cloud-dashboard/src/api/types.ts b/packages/cloud-dashboard/src/api/types.ts index 5d93f97..30d949c 100644 --- a/packages/cloud-dashboard/src/api/types.ts +++ b/packages/cloud-dashboard/src/api/types.ts @@ -143,6 +143,26 @@ export interface ApiKey { revoked_at: string | null } +/** + * A tracker credential, as the API is willing to describe it. + * + * There is no token field and never will be: the API returns a prefix so a + * person can tell which credential is installed, and nothing that could be + * used to act as them. + */ +export interface Integration { + provider: 'github' | 'linear' + /** Null for the organization default; a project id for an override. */ + projectId: string | null + tokenPrefix: string + accountLogin: string | null + scopes: string[] + createdAt: string + updatedAt: string + lastVerifiedAt: string | null + lastError: string | null +} + export interface SlackIntegration { configured: boolean enabled?: boolean diff --git a/packages/cloud-dashboard/src/components/AppShell.tsx b/packages/cloud-dashboard/src/components/AppShell.tsx index 9d1dffe..d7f94a8 100644 --- a/packages/cloud-dashboard/src/components/AppShell.tsx +++ b/packages/cloud-dashboard/src/components/AppShell.tsx @@ -1,7 +1,8 @@ import type { ReactNode } from 'react' -import { Outlet } from 'react-router-dom' +import { Outlet, useLocation } from 'react-router-dom' import { api } from '../api/endpoints.js' import { useResource } from '../lib/useResource.js' +import { isOrgPath, OrgRail } from './OrgRail.js' import { Sidebar } from './Sidebar.js' /** @@ -13,6 +14,7 @@ import { Sidebar } from './Sidebar.js' * still there" is the one fact that goes stale while you watch it. */ export function AppShell(): ReactNode { + const { pathname } = useLocation() const runners = useResource((key, signal) => api.runners(key, signal), [], { pollMs: 30_000 }) // Polled, not loaded once: the counts sit beside a screen that can create and // delete the very things being counted, and a nav that disagrees with the @@ -26,6 +28,13 @@ export function AppShell(): ReactNode { runners={runners.data} counts={{ Routes: routes.data?.length, Agents: agents.data?.length }} /> + {/* + * Mounted only on organization pages, matching the design's + * `showOrgRail`. Rendering it always would put a column of settings + * links beside the run list, which is the surface it was added to keep + * uncluttered. + */} + {isOrgPath(pathname) ? : null}
diff --git a/packages/cloud-dashboard/src/components/OrgRail.tsx b/packages/cloud-dashboard/src/components/OrgRail.tsx new file mode 100644 index 0000000..aec1168 --- /dev/null +++ b/packages/cloud-dashboard/src/components/OrgRail.tsx @@ -0,0 +1,75 @@ +import type { ReactNode } from 'react' +import { NavLink } from 'react-router-dom' +import { useSession } from '../lib/session.js' + +/** + * The organization's second-level navigation. + * + * The design (`sentinel0 App Noir.dc.html`, `showOrgRail`) puts a 190px rail + * between the primary sidebar and the page whenever an organization page is + * open. It exists because the organization group had outgrown a flat list: with + * Integrations added there are five destinations, each a settings surface + * rather than a view of work, and promoting all five into the primary sidebar + * would have buried Runs and Routes among them. + * + * Each entry carries a hint under its label. That is not decoration — these + * pages are visited rarely and their names alone ("Organization", "Projects") + * do not say which one holds the thing you came for. + */ +export interface OrgRailEntry { + to: string + label: string + hint: string +} + +export const ORG_RAIL: OrgRailEntry[] = [ + { to: '/settings', label: 'Settings', hint: 'name, id, control plane' }, + { to: '/settings/integrations', label: 'Integrations', hint: 'github, linear, slack' }, + { to: '/projects', label: 'Projects', hint: 'what the runner watches' }, + { to: '/keys', label: 'Access keys', hint: 'runner and user keys' }, + { to: '/settings/runners', label: 'Runners', hint: 'machines checking in' }, +] + +/** Whether a path is an organization page, and so should show the rail. */ +export function isOrgPath(pathname: string): boolean { + return ORG_RAIL.some((entry) => pathname === entry.to || pathname.startsWith(`${entry.to}/`)) +} + +/** + * Whether an entry should match its path exactly. + * + * Only the entries another entry sits *under* need it: `/settings` would + * otherwise stay lit on Integrations and Runners. Applying `end` to all of + * them instead leaves the rail with nothing lit on a child page like + * `/projects/new` — you are plainly in Projects, and the rail said you were + * nowhere. Derived rather than hand-flagged so adding a nested entry cannot + * reintroduce either half of that. + */ +export function matchesExactly(entry: OrgRailEntry): boolean { + return ORG_RAIL.some((other) => other.to.startsWith(`${entry.to}/`)) +} + +export function OrgRail(): ReactNode { + const { session } = useSession() + + return ( + + ) +} diff --git a/packages/cloud-dashboard/src/screens/Integrations.tsx b/packages/cloud-dashboard/src/screens/Integrations.tsx new file mode 100644 index 0000000..746269c --- /dev/null +++ b/packages/cloud-dashboard/src/screens/Integrations.tsx @@ -0,0 +1,446 @@ +import { useState, type ReactNode } from 'react' +import { Alert } from '@16-bits-design/ui/alert' +import { Badge } from '@16-bits-design/ui/badge' +import { Button } from '@16-bits-design/ui/button' +import { Dialog } from '@16-bits-design/ui/dialog' +import { Input } from '@16-bits-design/ui/input' +import { Select } from '@16-bits-design/ui/select' +import { Spinner } from '@16-bits-design/ui/spinner' +import { Text } from '@16-bits-design/ui/typography' +import { useToast } from '@16-bits-design/ui/toast' +import { api } from '../api/endpoints.js' +import { useKey } from '../lib/session.js' +import { useResource } from '../lib/useResource.js' +import { relativeTime } from '../lib/format.js' +import { ErrorPanel } from '../components/ErrorPanel.js' +import { PageHeader } from '../components/PageHeader.js' +import { Panel, Section } from '../components/Panel.js' +import type { Integration, Project } from '../api/types.js' + +/** What a person needs to know before pasting a token, per provider. */ +const PROVIDERS = { + github: { + label: 'GitHub', + field: 'Personal access token', + placeholder: 'github_pat_… or ghp_…', + hint: 'A fine-grained token with Issues: read and write, and Pull requests: read. Nothing more — Sentinel0 never pushes code; the agents do that under their own accounts.', + docs: 'https://github.com/settings/personal-access-tokens', + }, + linear: { + label: 'Linear', + field: 'API key', + placeholder: 'lin_api_…', + hint: 'A personal API key from Linear’s settings. Used to read issues and to comment and label as Sentinel0.', + docs: 'https://linear.app/settings/api', + }, +} as const + +type ProviderId = keyof typeof PROVIDERS + +/** + * Tracker credentials, and the Slack webhook that sits beside them. + * + * The organization holds one credential per provider, and any project may + * override it. That is what makes a single token cover every repository while + * one project — a different GitHub account, a separate Linear workspace — can + * still carry its own. + * + * Nothing here can show a token back. The API stores them encrypted and + * returns only a prefix, so this reports which credential is installed and + * lets it be replaced, exactly as the Slack webhook already did. + */ +export function Integrations(): ReactNode { + const key = useKey() + const { toast } = useToast() + const integrations = useResource((k, signal) => api.integrations(k, signal), []) + const projects = useResource((k, signal) => api.projects(k, signal), []) + const slack = useResource((k, signal) => api.slack(k, signal), []) + + return ( + <> + + +
+ {integrations.loading ? ( + + ) : integrations.error ? ( + + ) : ( + (Object.keys(PROVIDERS) as ProviderId[]).map((provider) => ( + credential.provider === provider + )} + projects={projects.data ?? []} + onChange={integrations.reload} + /> + )) + )} + + { + await api.saveSlack(key, { webhookUrl, enabled: true }) + toast({ + tone: 'success', + title: 'Slack connected', + message: 'Run notifications will post to that channel.', + }) + slack.reload() + }} + onDisconnect={async () => { + await api.deleteSlack(key) + toast({ + tone: 'info', + title: 'Slack disconnected', + message: 'No further runs will be posted.', + }) + slack.reload() + }} + /> +
+ + + ) +} + +/** + * One provider: the organization default, then any project overrides. + * + * The two are the same form with a different scope, rather than two screens, + * because they are the same decision — which account acts here — asked at two + * levels. + */ +function ProviderSection({ + provider, + credentials, + projects, + onChange, +}: { + provider: ProviderId + credentials: Integration[] + projects: Project[] + onChange: () => void +}): ReactNode { + const key = useKey() + const { toast } = useToast() + const meta = PROVIDERS[provider] + + const [token, setToken] = useState('') + const [scope, setScope] = useState('') + const [saving, setSaving] = useState(false) + const [error, setError] = useState(undefined) + const [removing, setRemoving] = useState(undefined) + + const orgDefault = credentials.find((credential) => credential.projectId === null) + const overrides = credentials.filter((credential) => credential.projectId !== null) + + // Only projects on this provider can override its credential, and a project + // that already has one is offered as a replacement rather than twice. + const eligible = projects.filter((project) => project.provider === provider) + + const save = async (): Promise => { + if (!token.trim()) { + setError(`Paste the ${meta.field.toLowerCase()} to save.`) + return + } + setSaving(true) + setError(undefined) + try { + const result = await api.saveIntegration(key, provider, { + token: token.trim(), + projectId: scope || null, + }) + toast({ + tone: 'success', + title: `${meta.label} connected`, + // Naming the account is the confirmation that matters: pasting the + // wrong one of two tokens is the mistake this catches. + message: `Authenticated as ${result.accountLogin}.`, + }) + setToken('') + setScope('') + onChange() + } catch (cause) { + setError(cause instanceof Error ? cause.message : String(cause)) + } finally { + setSaving(false) + } + } + + const remove = async (credential: Integration): Promise => { + try { + await api.deleteIntegration(key, provider, credential.projectId) + toast({ + tone: 'info', + title: `${meta.label} credential removed`, + message: credential.projectId + ? `${credential.projectId} falls back to the organization credential.` + : 'Projects without their own credential can no longer be polled.', + }) + onChange() + } catch (cause) { + toast({ + tone: 'danger', + title: 'Could not remove the credential', + message: cause instanceof Error ? cause.message : String(cause), + }) + } + } + + return ( + <> +
+
+ setRemoving(orgDefault)} + /> + + {overrides.map((override) => ( + setRemoving(override)} + /> + ))} + +
+ setToken(event.target.value)} + placeholder={meta.placeholder} + hint={meta.hint} + autoComplete="off" + /> + setWebhook(event.target.value)} + placeholder="https://hooks.slack.com/services/…" + hint={ + configured + ? 'Saving replaces the existing webhook. The current one is never shown back.' + : 'Create one in Slack under Incoming Webhooks, then paste it here.' + } + autoComplete="off" + /> + + {failure ? ( + + {failure} + + ) : null} + +
+ + {configured ? ( + + ) : null} +
+
+ )} +
+ + void onDisconnect()} + /> + + ) +} diff --git a/packages/cloud-dashboard/src/screens/ProjectNew.tsx b/packages/cloud-dashboard/src/screens/ProjectNew.tsx index 43dfba7..4733ac7 100644 --- a/packages/cloud-dashboard/src/screens/ProjectNew.tsx +++ b/packages/cloud-dashboard/src/screens/ProjectNew.tsx @@ -1,50 +1,115 @@ -import { useState, type ReactNode } from 'react' +import { useEffect, useState, type ReactNode } from 'react' import { useNavigate } from 'react-router-dom' +import { Alert } from '@16-bits-design/ui/alert' import { Button } from '@16-bits-design/ui/button' import { Input } from '@16-bits-design/ui/input' import { Select } from '@16-bits-design/ui/select' +import { Spinner } from '@16-bits-design/ui/spinner' +import { Text } from '@16-bits-design/ui/typography' import { Textarea } from '@16-bits-design/ui/textarea' import { useToast } from '@16-bits-design/ui/toast' import { api } from '../api/endpoints.js' import { useKey } from '../lib/session.js' -import { Alert } from '@16-bits-design/ui/alert' +import { useResource } from '../lib/useResource.js' import { PageHeader } from '../components/PageHeader.js' import { Panel, Section } from '../components/Panel.js' /** * Registering a tracker for the runner to poll. * - * `filters` is free-form JSON because the shape differs by provider and is - * still moving. It is parsed here, with the cursor still in the field, rather - * than sent for the API to reject. + * `filters` used to be a free-form JSON textarea, hinted with "use {}". That + * asked a person to know the filter schema, to spell a repository slug + * correctly, and to guess which labels a repository actually has — and every + * one of those is a mistake the API discovers hours later, as an empty poll. + * + * With a credential stored in the cloud, the API can answer all three, so this + * asks GitHub instead of asking the operator. The JSON escape hatch is still + * here, under a disclosure, because the filter shape differs by provider and + * is still moving. */ export function ProjectNew(): ReactNode { const key = useKey() const navigate = useNavigate() const { toast } = useToast() - const [id, setId] = useState('') const [provider, setProvider] = useState('github') - const [filters, setFilters] = useState('{}') + const [repo, setRepo] = useState('') + const [state, setState] = useState('open') + const [labels, setLabels] = useState([]) + const [team, setTeam] = useState('') + const [raw, setRaw] = useState('') + const [showRaw, setShowRaw] = useState(false) const [saving, setSaving] = useState(false) const [error, setError] = useState(undefined) + const repositories = useResource( + (k, signal) => + provider === 'github' ? api.gitHubRepositories(k, signal) : Promise.resolve([]), + [provider] + ) + + // Labels depend on the chosen repository, so they load on selection rather + // than up front — there is no useful list before a repository is picked. + const [availableLabels, setAvailableLabels] = useState([]) + const [labelsError, setLabelsError] = useState(undefined) + useEffect(() => { + if (provider !== 'github' || !repo) { + setAvailableLabels([]) + return + } + const controller = new AbortController() + setLabelsError(undefined) + api + .gitHubLabels(key, repo, controller.signal) + .then(setAvailableLabels) + .catch((cause: unknown) => { + if (!controller.signal.aborted) { + setLabelsError(cause instanceof Error ? cause.message : String(cause)) + } + }) + return () => controller.abort() + }, [key, provider, repo]) + + // Changing repository invalidates the label selection: the names came from + // the previous repository and almost certainly do not exist in this one. + useEffect(() => setLabels([]), [repo]) + + const id = provider === 'github' ? repo : team + + const buildFilters = (): Record => { + if (showRaw && raw.trim()) { + return JSON.parse(raw) as Record + } + if (provider === 'linear') { + return team ? { team } : {} + } + const [owner, name] = repo.split('/') + return { + owner, + repo: name, + state, + ...(labels.length > 0 ? { labels } : {}), + } + } + const create = async (): Promise => { if (!id.trim()) { - setError('A project id is required — the repository slug, or the Linear team key.') + setError(provider === 'github' ? 'Choose a repository.' : 'A Linear team key is required.') return } - let parsed: unknown + + let filters: Record try { - parsed = filters.trim() ? JSON.parse(filters) : {} + filters = buildFilters() } catch { - setError('Filters must be valid JSON. Use {} for no filtering.') + setError('The filter override must be valid JSON. Use {} for no filtering.') return } + setSaving(true) setError(undefined) try { - await api.createProject(key, { id: id.trim(), provider, filters: parsed }) + await api.createProject(key, { id: id.trim(), provider, filters }) toast({ tone: 'success', title: 'Project saved', @@ -58,6 +123,13 @@ export function ProjectNew(): ReactNode { } } + /* + * A 409 from the repository list means no GitHub credential is stored. That + * is not an error in this screen — it is a prerequisite, and saying so with + * the link that fixes it beats a red panel repeating the status code. + */ + const needsCredential = repositories.error?.includes('No GitHub credential') + return ( <> @@ -65,37 +137,93 @@ export function ProjectNew(): ReactNode {
-
+ ({ + value: repository.slug, + label: repository.private + ? `${repository.slug} (private)` + : repository.slug, + })), + ]} + /> + + setId(event.target.value)} - hint="owner/repo for GitHub, or the team key for Linear." - placeholder="acme/platform" - autoFocus - /> -