feat(dev): npm run dev:stack runs the full app from source (closes #65)

.github/workflows/dev-stack.yml

@@ -0,0 +1,59 @@
+name: Dev stack
+
+# npm run dev:stack (#65): the whole app from source on a clean runner, without a TTY.
+on:
+  pull_request:
+    paths:
+      - "scripts/dev-stack*.mjs"
+      - "scripts/tests/dev-stack.test.mjs"
+      - "apps/**"
+      - "packages/**"
+      - "deploy/docker/**"
+      - "Dockerfile"
+      - "package.json"
+      - "package-lock.json"
+      - ".github/workflows/dev-stack.yml"
+  push:
+    branches:
+      - main
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: dev-stack-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  dev-stack:
+    name: dev:stack up, owner setup, OpenCode pane, reset
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+
+      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+        with:
+          node-version-file: .nvmrc
+          cache: npm
+
+      - name: Install dependencies
+        env:
+          ELECTRON_SKIP_BINARY_DOWNLOAD: "1"
+        run: npm ci
+
+      - name: Dev-stack unit tests
+        run: node --test scripts/tests/dev-stack.test.mjs
+
+      - name: Run the stack, set up the owner, open an OpenCode pane, reset
+        run: node scripts/dev-stack-ci.mjs < /dev/null
+
+      - name: Nothing of the dev stack is left
+        if: always()
+        run: |
+          test -z "$(docker ps -aq --filter label=com.oll4com.spaceapp.dev-stack)"
+          test -z "$(docker volume ls -q --filter label=com.oll4com.spaceapp.dev-stack)"
+          test ! -e .dev-stack

.gitignore

@@ -9,6 +9,8 @@ coverage/
 playwright-report/
 test-results/
 scratch/
+# npm run dev:stack state: secrets, provider homes, OpenCode install (reset removes it).
+/.dev-stack/
 .npm/
 apps/desktop/release/*-unpacked/
 apps/desktop/release/*.7z

CONTRIBUTING.md

@@ -50,6 +50,96 @@ select a short temporary directory you own and set `TMPDIR` for the command.
 On Windows run this particular smoke check inside WSL or a Linux development
 container. It does not certify the Windows/macOS installer.
 
+## Run the full app from source
+
+`npm run dev:stack` starts a working SpaceApp from this checkout with hot
+reload. It never touches an installed SpaceApp: it has its own compose project
+(`spaceapp-dev-stack`), its own git-ignored state directory (`.dev-stack/`)
+and its own ports, and every port it opens binds to `127.0.0.1` only.
+
+Prerequisites: the Node and npm versions above, `npm ci` done, and Docker with
+the Compose plugin (`docker compose version`). Docker runs only PostgreSQL
+(pgvector) and Temporal, with the images and digests pinned in
+`packages/run-spaceapp/templates/compose.yml` (read from that file at start).
+
+```sh
+npm run dev:stack            # start; prints the URL and the one-time owner setup token
+npm run dev:stack -- down    # stop the processes and containers, keep the data
+npm run dev:stack -- reset   # remove the containers, volumes and .dev-stack/ it created
+npm run dev:stack -- rebuild # rebuild packages/* and restart the API, worker and CLI host
+npm run dev:stack -- status
+```
+
+In order, `dev:stack` starts PostgreSQL and Temporal and waits for their
+health checks, builds the workspace packages, runs the migrations, installs the
+OpenCode version pinned in the `Dockerfile` into `.dev-stack/cli/vendor/` (not
+globally), and then runs from source, each with its own prefix in the output:
+
+| Process | What runs | Reload |
+| --- | --- | --- |
+| `[api]` | `tsx watch apps/api/src/server.ts` | restarts on a change in `apps/api/src` |
+| `[worker-build]`, `[worker]` | `tsc --watch` and `node --watch dist/worker.js` | restarts on a change in `apps/worker/src` |
+| `[web]` | Vite dev server | hot reload in the browser for `apps/web/src` |
+| `[cli-host]` | `packages/cli-host/dist/main.js` on `.dev-stack/run/cli-host.sock` | `rebuild` |
+
+A change in `packages/*` is picked up with `npm run dev:stack -- rebuild` in a
+second terminal: it builds the packages and the running stack restarts the
+processes that load them.
+
+Open the printed URL, enter the setup token, create the owner and sign in
+(the dev-login shortcut is off). The Getting Started room opens; an OpenCode
+pane starts through the from-source CLI host with the same wrapper as the
+image (`deploy/docker/public-cli-wrapper.sh`, pointed at `.dev-stack/cli/`).
+OpenCode's free model needs no provider account. The browser service and the
+optional companions stay disabled.
+
+Ports (environment variables; the defaults are never 4910/4911):
+
+| Variable | Default |
+| --- | --- |
+| `SPACE_DEV_STACK_WEB_PORT` (open this one) | 4921 |
+| `SPACE_DEV_STACK_API_PORT` | 4920 |
+| `SPACE_DEV_STACK_POSTGRES_PORT` | 4922 |
+| `SPACE_DEV_STACK_TEMPORAL_PORT` | 4923 |
+
+`SPACE_DEV_STACK_DIR` moves the state directory and `SPACE_DEV_STACK_PROJECT`
+renames the compose project. The database password, session secret and setup
+token are generated once per state directory in `.dev-stack/secrets/` with
+mode 0600; only the setup token is printed. `SPACE_*` variables of your shell
+are not passed to the processes.
+
+If a process exits, `dev:stack` names it (`dev-stack: worker exited
+unexpectedly ...`) and stops the stack. While it runs it checks `/readyz` and
+prints the dependency that fails (for example `worker=ERROR`).
+
+Troubleshooting:
+
+- *Port ... is in use*: set the matching `SPACE_DEV_STACK_*_PORT`.
+- *The dev stack is already running*: `npm run dev:stack -- down` first.
+- */readyz did not report ok*: the last line names the failing dependency;
+  `docker compose --project-name spaceapp-dev-stack --file .dev-stack/compose.yml logs`
+  shows the PostgreSQL and Temporal side.
+- *was not created by dev:stack*: `SPACE_DEV_STACK_DIR` names a directory that
+  already holds other files; `dev:stack` never uses or removes it.
+- A deep checkout path puts the CLI host socket in a private directory under
+  `TMPDIR` (Unix sockets are limited to about 100 bytes).
+
+Platform notes:
+
+- **Linux**: Docker Engine with the Compose plugin.
+- **macOS**: Docker Desktop, or Colima (`colima start`, then `docker context use colima`).
+  The ports are published on the Mac's `127.0.0.1`.
+- **Windows 11**: run everything inside WSL2 (Ubuntu), with the checkout on the
+  Linux file system (`~/src/spaceapp`, not `/mnt/c/...`, for file watching and
+  speed) and Docker Desktop's WSL integration or Docker Engine inside WSL.
+  Open the URL in a Windows browser; WSL forwards `127.0.0.1`. Native Windows is
+  refused with a message.
+
+The `Dev stack` CI workflow runs `scripts/dev-stack-ci.mjs` on `ubuntu-latest`:
+it starts the stack without a TTY, waits for `/readyz`, creates the owner over
+HTTP, opens an OpenCode pane through the API, checks that it runs, then runs
+`reset` and checks that no container, volume or process is left.
+
 ## Make and verify a small change
 
 Work on a branch, not on `main`, so the change can become a pull request:

apps/web/tests/vite-config.test.ts

@@ -1,7 +1,7 @@
 // @vitest-environment node
 
 import { describe, expect, it } from "vitest";
-import { manualChunkForModule } from "../vite.config.js";
+import { devApiOrigin, manualChunkForModule } from "../vite.config.js";
 
 describe("web bundle chunk policy", () => {
   it("isolates React and xterm while leaving feature modules available for lazy chunks", () => {
@@ -16,3 +16,12 @@ describe("web bundle chunk policy", () => {
     expect(manualChunkForModule("/workspace/src/features/browser-pane/BrowserPane.tsx")).toBeUndefined();
   });
 });
+
+describe("dev server API proxy", () => {
+  it("targets the installed API port by default and the dev-stack API when SPACE_WEB_API_ORIGIN names it", () => {
+    expect(devApiOrigin({})).toBe("http://127.0.0.1:4910");
+    expect(devApiOrigin({ SPACE_WEB_API_ORIGIN: "http://127.0.0.1:4920" })).toBe("http://127.0.0.1:4920");
+    expect(() => devApiOrigin({ SPACE_WEB_API_ORIGIN: "http://0.0.0.0:4920" })).toThrow(/127\.0\.0\.1/);
+    expect(() => devApiOrigin({ SPACE_WEB_API_ORIGIN: "https://example.com" })).toThrow();
+  });
+});

apps/web/vite.config.ts

@@ -17,6 +17,13 @@ export function manualChunkForModule(moduleId: string): string | undefined {
   return undefined;
 }
 
+export function devApiOrigin(env: NodeJS.ProcessEnv): string {
+  const origin = env.SPACE_WEB_API_ORIGIN?.trim();
+  if (!origin) return "http://127.0.0.1:4910";
+  if (!/^http:\/\/(?:127\.0\.0\.1|localhost):\d{2,5}$/.test(origin)) throw new Error("SPACE_WEB_API_ORIGIN must be http://127.0.0.1:<port>.");
+  return origin;
+}
+
 export default defineConfig({
   plugins: [react(), demoBundleBoundaryPlugin(), retainOpenTabAssets()],
   // The icon package ships as source; the alias keeps worktrees (which share live node_modules) resolving this checkout.
@@ -33,13 +40,9 @@ export default defineConfig({
   server: {
     host: process.env.SPACE_WEB_HOST ?? "0.0.0.0",
     port: Number.parseInt(process.env.SPACE_WEB_PORT ?? "4911", 10),
-    proxy: {
-      "/api": "http://127.0.0.1:4910",
-      "/plugins": "http://127.0.0.1:4910",
-      "/healthz": "http://127.0.0.1:4910",
-      "/readyz": "http://127.0.0.1:4910",
-      "/version": "http://127.0.0.1:4910"
-    }
+    // npm run dev:stack runs its API on its own port (SPACE_WEB_API_ORIGIN) so it never reaches an installed SpaceApp on 4910.
+    // /api also carries the pane terminal WebSockets.
+    proxy: Object.fromEntries(["/api", "/plugins", "/healthz", "/readyz", "/version"].map((path) => [path, { target: devApiOrigin(process.env), ws: path === "/api" }]))
   },
   test: {
     environment: "jsdom",

deploy/docker/opencode-first-run.mjs

@@ -40,7 +40,8 @@ export function toolCycleSucceeded(result,token) {
   const answer=list.findIndex((e,i)=>i>tool&&e.type==='text'&&e.part?.text?.includes(token));
   return answer>tool;
 }
-const MEMORY_DIRECTORY='/var/lib/spaceapp/memory';
+// npm run dev:stack runs the wrapper outside the container and names its own memory directory.
+const MEMORY_DIRECTORY=process.env.SPACE_MEMORY_ROOT||'/var/lib/spaceapp/memory';
 // The 1.1.x default. Configs that still carry exactly this value get the new
 // default; any other value is an owner policy and is preserved.
 const LEGACY_EXTERNAL_DIRECTORY={'*':'ask',[MEMORY_DIRECTORY]:'allow',[`${MEMORY_DIRECTORY}/*`]:'allow'};

package.json

@@ -46,6 +46,7 @@
     "dev:api": "npm run dev -w @space/api",
     "dev:web": "npm run dev -w @space/web",
     "dev:worker": "npm run dev -w @space/worker",
+    "dev:stack": "node scripts/dev-stack.mjs",
     "db:migrate": "npm run migrate -w @space/db",
     "build": "npm run build -w @space/contracts && npm run build -w @space/icons && npm run build -w @space/streaming && npm run build -w @space/opencode-control && npm run build -w @space/memory-graph && npm run build -w @space/browser-host && npm run build -w @space/cli-host && npm run build -w @space/codex-app-server && npm run build -w @space/runtime && npm run build -w @space/db && npm run build -w @space/gateway && node scripts/build-demo-projects.mjs && npm run build -w @space/api && npm run build -w @space/worker && npm run build -w @space/web && node scripts/fix-build-permissions.mjs",
     "public:export:verify": "node scripts/public-export.mjs --verify",
@@ -58,7 +59,7 @@
     "build:demos": "node scripts/build-demo-projects.mjs",
     "test:architecture": "node --test scripts/tests/domain-boundaries.test.mjs scripts/tests/composition-root-size.test.mjs scripts/tests/quality-known-limitations.test.mjs && node scripts/domain-boundaries.mjs",
     "smoke:offline": "npm run build -w @space/cli-host && node scripts/contributor-smoke.mjs",
-    "test:contributor": "node --test scripts/tests/contributor-smoke.test.mjs",
+    "test:contributor": "node --test scripts/tests/contributor-smoke.test.mjs scripts/tests/dev-stack.test.mjs",
     "lint": "eslint --max-warnings=-1 apps packages scripts"
   },
   "devDependencies": {

scripts/dev-stack-ci.mjs

@@ -0,0 +1,98 @@
+#!/usr/bin/env node
+// CI check of npm run dev:stack (#65), without a TTY: start the stack, wait for /readyz, create the owner over HTTP with
+// the setup token, open an OpenCode pane through the API and check that its CLI session runs, then reset and check that
+// no container, volume or process of the dev stack is left.
+import { spawn, spawnSync } from "node:child_process";
+import { existsSync } from "node:fs";
+import { readFile } from "node:fs/promises";
+import { join } from "node:path";
+import process from "node:process";
+import { repoRoot, resolveConfig, statePaths } from "./dev-stack.mjs";
+
+const config = resolveConfig(process.env);
+const paths = statePaths(config);
+const api = `http://${config.host}:${config.ports.api}`;
+const web = `http://${config.host}:${config.ports.web}`;
+const label = `com.oll4com.spaceapp.dev-stack=${config.project}`;
+const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
+const step = (text) => process.stdout.write(`dev-stack-ci: ${text}\n`);
+
+async function until(what, check, timeoutMs, isAlive = () => true) {
+  const deadline = Date.now() + timeoutMs;
+  let last;
+  while (Date.now() < deadline) {
+    if (!isAlive()) throw new Error(`dev:stack exited while waiting for ${what}.`);
+    try { const value = await check(); if (value) return value; } catch (error) { last = error; }
+    await delay(2_000);
+  }
+  throw new Error(`Timed out waiting for ${what}${last ? ` (${last.message})` : ""}.`);
+}
+
+function leftovers() {
+  const containers = spawnSync("docker", ["ps", "-a", "-q", "--filter", `label=${label}`], { encoding: "utf8" }).stdout?.trim() ?? "";
+  const volumes = spawnSync("docker", ["volume", "ls", "-q", "--filter", `label=${label}`], { encoding: "utf8" }).stdout?.trim() ?? "";
+  const processes = (spawnSync("ps", ["-eo", "pid=,args="], { encoding: "utf8" }).stdout ?? "").split("\n")
+    .filter((line) => line.includes(config.stateDir) || line.includes(paths.socket) || /scripts\/dev-stack\.mjs up/.test(line))
+    .filter((line) => !line.includes("dev-stack-ci.mjs") && !line.includes(" ps -eo"));
+  return { containers, volumes, processes, stateDir: existsSync(config.stateDir) };
+}
+
+async function main() {
+  step("starting npm run dev:stack");
+  const stack = spawn(process.execPath, [join(repoRoot, "scripts", "dev-stack.mjs"), "up"], { cwd: repoRoot, stdio: ["ignore", "inherit", "inherit"] });
+  let stackExited = false;
+  stack.once("exit", () => { stackExited = true; });
+  const alive = () => !stackExited;
+  let failure;
+  try {
+    await until("/readyz ok: true", async () => (await (await fetch(`${api}/readyz`)).json()).ok === true, 20 * 60_000, alive);
+    step("/readyz reports ok: true");
+    const webIndex = await fetch(web);
+    if (!webIndex.ok) throw new Error(`The Vite dev server answered ${webIndex.status}.`);
+    const token = (await readFile(join(paths.secrets, "setup-token"), "utf8")).trim();
+    // Through the Vite dev server, as a browser would, so the proxy is part of the check.
+    const claim = await fetch(`${web}/api/setup/claim`, {
+      method: "POST", headers: { "content-type": "application/json" },
+      body: JSON.stringify({ token, email: "owner@dev-stack.invalid", password: `dev-stack-${token.slice(0, 16)}` })
+    });
+    if (!claim.ok) throw new Error(`Owner setup failed: HTTP ${claim.status}.`);
+    const { starterRoomId } = await claim.json();
+    const cookie = claim.headers.getSetCookie().map((value) => value.split(";")[0]).join("; ");
+    step("owner created with the setup token");
+    const csrf = await (await fetch(`${web}/api/auth/csrf`, { headers: { cookie } })).json();
+    const headers = { cookie, "content-type": "application/json", [csrf.headerName]: csrf.csrfToken };
+    const created = await fetch(`${web}/api/rooms/${encodeURIComponent(starterRoomId)}/panes`, {
+      method: "POST", headers, body: JSON.stringify({ panes: [{ mode: "TERMINAL", terminalRuntimeId: "cli:opencode" }] })
+    });
+    if (!created.ok) throw new Error(`Opening the OpenCode pane failed: HTTP ${created.status} ${await created.text()}`);
+    const paneId = (await created.json()).data[0].id;
+    const session = await fetch(`${web}/api/panes/${encodeURIComponent(paneId)}/cli/session`, { method: "POST", headers, body: JSON.stringify({ runtimeId: "cli:opencode" }) });
+    if (!session.ok) throw new Error(`Starting the OpenCode CLI session failed: HTTP ${session.status} ${await session.text()}`);
+    const running = await until("the OpenCode pane to run", async () => {
+      const current = await (await fetch(`${web}/api/panes/${encodeURIComponent(paneId)}/cli/session`, { headers: { cookie } })).json();
+      if (current?.status === "ERROR" || current?.status === "EXITED") throw new Error(`OpenCode pane ${current.status}: ${current.statusReason}`);
+      return current?.status === "RUNNING" ? current : null;
+    }, 3 * 60_000, alive);
+    step(`OpenCode pane ${paneId} is ${running.status}`);
+  } catch (error) {
+    failure = error;
+  } finally {
+    step("npm run dev:stack -- reset");
+    const reset = spawnSync(process.execPath, [join(repoRoot, "scripts", "dev-stack.mjs"), "reset"], { cwd: repoRoot, stdio: "inherit" });
+    await until("dev:stack to stop", () => stackExited, 60_000).catch((error) => { failure ??= error; stack.kill("SIGKILL"); });
+    if (reset.status !== 0) failure ??= new Error(`reset failed with exit ${reset.status}.`);
+  }
+  const left = leftovers();
+  step(`after reset: containers=[${left.containers}] volumes=[${left.volumes}] processes=${left.processes.length} stateDir=${left.stateDir}`);
+  if (left.containers || left.volumes || left.processes.length || left.stateDir) {
+    for (const line of left.processes) step(`left: ${line}`);
+    failure ??= new Error("reset left containers, volumes, processes or the state directory behind.");
+  }
+  if (failure) throw failure;
+  step("passed");
+}
+
+main().catch((error) => {
+  process.stderr.write(`dev-stack-ci: ${error.message}\n`);
+  process.exitCode = 1;
+});