Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -500,6 +500,9 @@ Without `--effort`, `--model` uses the model's default level. Both flags require
`--prompt` (or `-p`) and leave the saved default for new sessions unchanged; the
run's model and effort are recorded on that session.

For automation and long-running `-p` jobs, see [headless execution](./docs/HEADLESS.md)
for output completion, cancellation, background execution and validation limits.

ZCode reads MCP servers, hooks, plugins, permissions, network, storage and display
settings from `~/.zcode/cli/setting.json` (or
`%USERPROFILE%\.zcode\cli\setting.json` on Windows). Project settings use
Expand Down
61 changes: 61 additions & 0 deletions docs/HEADLESS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Headless execution

`zcode -p 'task'` runs without a TUI. It is a foreground command, not a daemon,
and headless mode is separate from the browser's `--browser-use=headless` setting.

## Long-running jobs

- The prompt lifecycle keeps the Node event loop alive until prompt work and
cleanup finish, including waits that otherwise have only unreferenced timers.
This keepalive is not a task timeout or progress heartbeat.
- The launcher adds no overall job deadline. Model, tool, network, workflow and
cleanup policies still apply. Use a host supervisor for an overall deadline;
model retry settings are not a job timeout.
- Text and JSON output normally arrive at completion. For observable progress,
use `--output-format stream-json` and consume the JSONL stream incrementally.
A quiet interval during a tool call or retry is not by itself a failure.
- Consume stdout **and** stderr. Completion waits for queued writes to reach
their consumers rather than allowing the runtime's exit watchdog to discard
them. If a consumer stops reading, completion can wait until it resumes or
the job is cancelled. This does not guarantee the consumer has persisted data.
- With `stream-json`, require both exit status 0 and a terminal `result` record.
A missing result is incomplete even if the process reports status 0.

On POSIX systems, `SIGINT`, `SIGTERM` and `SIGHUP` are forwarded to the runtime
until it actually exits, including repeated signals during cleanup. Cancellation
must not be interpreted as success; conventional exit statuses are 130, 143
and 129 respectively. `SIGKILL` cannot run cleanup.

Headless mode does not make terminal logout safe automatically. Use a service
manager or explicitly detach a job if it must survive the shell/SSH connection:

```sh
nohup zcode -p 'your task' --output-format stream-json \
>run.jsonl 2>run.stderr </dev/null &
```

Record the PID and exit status in your supervisor if you need reliable
cancellation and completion tracking. Configure permissions deliberately;
headless execution does not provide the TUI's interactive approval experience.

## Regression coverage and limits

Local HTTP fixtures and real Node child processes exercise:

- Connection-reset recovery, incomplete SSE recovery and exhausted recovery.
- HTTP 503 retry backoff without a test-only keepalive preload.
- A 512 KiB answer in text, JSON and stream-JSON, including a consumer that
delays reading for 10 seconds; stream-JSON also has a normal-reader control.
- `-p` cancellation during retry backoff and repeated launcher termination signals.
- Unreferenced pending work, success/failure handle cleanup and output-write errors.

The slow-reader case previously reproduced exit status 0 without the final result.
The repeated-signal case previously left the runtime alive after its launcher exited.
The lifecycle wrapper and launcher signal forwarding address those failures.

These are bounded regression tests, not an hours-long soak test. They do not
establish a fixed memory ceiling or cover every plugin, MCP server, background
workflow or real provider outage. Streaming output does not imply constant-memory
execution: the native runtime still retains turn events and the final answer.
For unattended production jobs, monitor process memory, disk space and progress,
and keep resumable session state rather than relying only on captured stdout.
4 changes: 4 additions & 0 deletions docs/HOST_INTEGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ to replace the CLI package without changing its own process model.
- For non-interactive commands, preserve the inherited standard-stream
contract; the command's documented output is the only supported machine
interface.
- Consume both stdout and stderr throughout a headless run. A successful run
waits for pending output writes before exiting; a blocked consumer can delay
completion. With `--output-format stream-json`, check both exit status and the
terminal `result` record. See [headless execution](./HEADLESS.md).
- Set the child's working directory to the workspace the user selected. The
launcher passes it to the runtime.

Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
"docs/PROVIDER_CONFIG.md",
"docs/PROVIDER_CONFIG.zh-CN.md",
"docs/HOST_INTEGRATION.md",
"docs/HEADLESS.md",
"docs/THIRD_PARTY_CONTENT.md",
"LICENSES/Apache-2.0.txt",
"LICENSES/README.md",
Expand Down
1 change: 1 addition & 0 deletions scripts/check-package.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ export async function validatePackageTree(base = root): Promise<void> {
"docs/CONFIGURATION.zh-CN.md",
"docs/PROVIDER_CONFIG.md",
"docs/PROVIDER_CONFIG.zh-CN.md",
"docs/HEADLESS.md",
"package.json",
"src/app-server-client.ts",
"src/command.ts",
Expand Down
4 changes: 4 additions & 0 deletions scripts/check-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import {
hasRuntimeCliSettingsFile,
hasRuntimeCliHelpContract,
hasRuntimeHttpNoContentGuard,
hasRuntimeHeadlessLifecycle,
hasRuntimeNetworkRetryGuard,
hasRuntimeResolvedContextWindow,
hasRuntimeRecursiveWorkspacePaths,
Expand All @@ -31,6 +32,7 @@ import {
hasRuntimeTrackedDetachedAgentLifecycle,
patchRuntimeGoalFailurePause,
patchRuntimeHttpNoContent,
patchRuntimeHeadlessLifecycle,
patchRuntimeLoginModelDefaults,
patchRuntimeNetworkRetryClassification,
patchRuntimeOfficialMcpAvailability,
Expand Down Expand Up @@ -92,6 +94,8 @@ if (patchRuntimeLoginModelDefaults(runtimeSource) !== runtimeSource
|| (patchEnabled("goal-failure-pause") && patchRuntimeGoalFailurePause(runtimeSource) !== runtimeSource)
|| patchRuntimeHttpNoContent(runtimeSource) !== runtimeSource
|| !hasRuntimeHttpNoContentGuard(runtimeSource)
|| patchRuntimeHeadlessLifecycle(runtimeSource) !== runtimeSource
|| !hasRuntimeHeadlessLifecycle(runtimeSource)
|| patchRuntimeNetworkRetryClassification(runtimeSource) !== runtimeSource
|| !hasRuntimeNetworkRetryGuard(runtimeSource)
|| patchRuntimeSqliteBusyTimeout(runtimeSource) !== runtimeSource
Expand Down
2 changes: 2 additions & 0 deletions scripts/pack-release.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ const requiredFiles = [
"docs/PROVIDER_CONFIG.md",
"docs/PROVIDER_CONFIG.zh-CN.md",
"docs/HOST_INTEGRATION.md",
"docs/HEADLESS.md",
"package.json",
"vendor/extraction.json",
"vendor/node_modules/@zcode/tui/dist/index.js",
Expand All @@ -65,6 +66,7 @@ const allowedRoots = new Set([
"docs/PROVIDER_CONFIG.md",
"docs/PROVIDER_CONFIG.zh-CN.md",
"docs/HOST_INTEGRATION.md",
"docs/HEADLESS.md",
"package.json",
"zcode-runtime.lock.json"
]);
Expand Down
1 change: 1 addition & 0 deletions scripts/package-contents.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ export const publishedFiles = [
"docs/PROVIDER_CONFIG.md",
"docs/PROVIDER_CONFIG.zh-CN.md",
"docs/HOST_INTEGRATION.md",
"docs/HEADLESS.md",
...attributionFiles,
"zcode-runtime.lock.json",
"README.md",
Expand Down
21 changes: 21 additions & 0 deletions scripts/runtime-headless-patches.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
const marker = ".withHeadlessPromptLifecycle(";

export function hasRuntimeHeadlessLifecycle(runtime: string): boolean {
return runtime.includes(marker);
}

export function patchRuntimeHeadlessLifecycle(runtime: string): string {
if (hasRuntimeHeadlessLifecycle(runtime)) return runtime;
const start = /[A-Za-z_$][\w$]*=[A-Za-z_$][\w$]*\(async\([^)]*\)=>\{if\([A-Za-z_$][\w$]*\.trim\(\)\.length===0\)return ([A-Za-z_$][\w$]*)\.stderr\.write/u.exec(runtime);
const end = start ? runtime.indexOf('},"runPrompt")', start.index) : -1;
if (!start || end < 0) {
throw new Error("ZCode runtime is incompatible with the headless lifecycle patch (runPrompt anchors missing).");
}
const bodyStart = start.index + start[0].indexOf("=>{") + 3;
const helper = 'require(require("node:path").join(__dirname,"cli-config.cjs"))';
return runtime.slice(0, bodyStart)
+ `return await ${helper}.withHeadlessPromptLifecycle(${start[1]},async()=>{`
+ runtime.slice(bodyStart, end)
+ "})"
+ runtime.slice(end);
}
8 changes: 8 additions & 0 deletions scripts/sync-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ import {
} from "../src/runtime-capabilities.ts";
import { parseReleaseVersion, syncedReleaseVersion } from "./release-version.ts";
import { markRuntimeModified } from "./runtime-attribution.ts";
import { hasRuntimeHeadlessLifecycle, patchRuntimeHeadlessLifecycle } from "./runtime-headless-patches.ts";
export { hasRuntimeHeadlessLifecycle, patchRuntimeHeadlessLifecycle } from "./runtime-headless-patches.ts";
import { hasRuntimeSessionLifecycle, patchRuntimeSessionLifecycle } from "./runtime-session-lifecycle-patches.ts";
export { hasRuntimeSessionLifecycle, patchRuntimeSessionLifecycle } from "./runtime-session-lifecycle-patches.ts";
import { hasRuntimeBigmodelOAuth, patchRuntimeBigmodelOAuth } from "./runtime-bigmodel-oauth-patches.ts";
Expand Down Expand Up @@ -1907,6 +1909,12 @@ export const runtimePatchPlan: readonly RuntimePatchDefinition[] = [
apply: patchRuntimeStreamEofFinishGuard,
verify: hasRuntimeStreamEofFinishGuard
},
{
id: "headless-lifecycle",
requirement: "required",
apply: patchRuntimeHeadlessLifecycle,
verify: hasRuntimeHeadlessLifecycle
},
{
id: "sqlite-busy-timeout",
requirement: "required",
Expand Down
8 changes: 4 additions & 4 deletions src/launcher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -420,14 +420,14 @@ async function runRuntime(
let forwardedSignal = false;
const forwardSignal = (signal: NodeJS.Signals) => {
forwardedSignal = true;
if (!child.killed) child.kill(signal);
if (child.exitCode === null && child.signalCode === null) child.kill(signal);
};
const onSigint = () => forwardSignal("SIGINT");
const onSigterm = () => forwardSignal("SIGTERM");
const onSighup = () => forwardSignal("SIGHUP");
process.once("SIGINT", onSigint);
process.once("SIGTERM", onSigterm);
if (process.platform !== "win32") process.once("SIGHUP", onSighup);
process.on("SIGINT", onSigint);
process.on("SIGTERM", onSigterm);
if (process.platform !== "win32") process.on("SIGHUP", onSighup);
try {
const code = await waitForChild(
child,
Expand Down
1 change: 1 addition & 0 deletions src/runtime-config-bridge.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export { exchangeBigmodelOAuthCode } from "./bigmodel-oauth.ts";
export { browserOpenCommand } from "./browser-open.ts";
export { sharedOAuthCredentialReader, zaiBusinessAccessToken } from "./shared-oauth.ts";
export { inspectRuntimeConfiguration, writeConfigurationDoctor } from "./config-diagnostics.ts";
export { withHeadlessPromptLifecycle } from "./runtime-headless-lifecycle.ts";

function record(value: unknown): Record<string, unknown> | undefined {
return value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : undefined;
Expand Down
23 changes: 23 additions & 0 deletions src/runtime-headless-lifecycle.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import type { Writable } from "node:stream";

function flushOutput(stream: Writable): Promise<void> {
return new Promise((resolve, reject) => {
stream.write("", (error) => error ? reject(error) : resolve());
});
}

export async function withHeadlessPromptLifecycle<Result>(
output: { stdout: Writable; stderr: Writable },
run: () => Promise<Result>
): Promise<Result> {
const keepAlive = setInterval(() => {}, 1_000);
try {
return await run();
} finally {
try {
await Promise.all([flushOutput(output.stdout), flushOutput(output.stderr)]);
} finally {
clearInterval(keepAlive);
}
}
}
5 changes: 3 additions & 2 deletions test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@ Counts describe this reset, not an assertion that future suites must have a fixe

| File | Failure this must prevent |
| --- | --- |
| `launcher.test.ts` | Keyless prompts run, missing sessions leak diagnostics, or a local plugin cannot be installed and discovered. |
| `launcher.test.ts` | Keyless prompts run, missing sessions leak diagnostics, repeated cancellation leaves the runtime alive, or a local plugin cannot be installed and discovered. |
| `app-server-auth.test.ts` | Real protocol turns use the wrong credential owner, fail to rotate keys, or fall back from missing credentials to host authentication. |
| `settings-file.test.ts` | Runtime writes and trust changes escape a host-selected settings store or change native MCP precedence. |
| `config-migration.test.ts` | Migration loses providers, overwrites conflicting/invalid configuration, or runs twice concurrently. |
| `shared-oauth.test.ts` | Native login stores plaintext/wrong tokens, ignores Desktop account changes, or damages shared login state on an HTTP failure. |
| `network-retry.test.ts` | A reset/unfinished stream silently truncates output, duplicates partial output on retry, or reports success after exhaustion. |
| `network-retry.test.ts` | Retry/slow pipes lose a headless result, partial output is duplicated, or exhaustion/cancellation reports success. |
| `session-lifecycle.test.ts` | An untouched session is saved, rename/resume loses data, or failed work becomes unresumable. |
| `session-model-recovery.test.ts` | An invalid saved model sends a request, cannot be repaired and persisted, or falls back from an unavailable explicit model. |
| `guarded-permission-tui.test.ts` | A destructive command executes before single-use approval. |
Expand All @@ -70,6 +70,7 @@ Counts describe this reset, not an assertion that future suites must have a fixe

| File | Failure this must prevent |
| --- | --- |
| `headless-lifecycle.test.cjs` | Unreferenced work exits early, output errors disappear, or lifecycle handles keep a finished process alive. |
| `session-lifecycle.test.cjs` | Concurrent first writes create duplicate sessions or failed title writes claim success. |
| `sqlite-session-store.test.cjs` | Migration races corrupt the database, lock exhaustion commits data, killed writes survive rollback, or native timeout policy disappears. |
| `sqlite-recovery.test.cjs` | Retrying duplicates records/counters, enters a caller's transaction, blocks the event loop, loses update order, or persists work after close. |
Expand Down
9 changes: 8 additions & 1 deletion test/fixtures/network-reset-server.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,21 @@ const server = createServer(async (request, response) => {

streamingRequests += 1;
process.stdout.write(`REQUEST ${streamingRequests}\n`);
if (streamingRequests === 1 && process.env.ZCODE_TEST_HTTP_RETRY === "1") {
response.writeHead(503, { "content-type": "application/json", connection: "close" });
response.end(JSON.stringify({ error: { message: "Temporary fixture overload" } }));
return;
}
response.writeHead(200, { "content-type": "text/event-stream" });
if (streamingRequests === 1) {
response.write(streamChunk("chatcmpl-reset", "PARTIAL_SHOULD_BE_DISCARDED"));
setTimeout(() => response.socket?.destroy(), 25);
return;
}

response.write(streamChunk("chatcmpl-recovered", "RECOVERED_FINAL"));
response.write(streamChunk("chatcmpl-recovered", process.env.ZCODE_TEST_RESPONSE_BYTES
? "R".repeat(Number(process.env.ZCODE_TEST_RESPONSE_BYTES))
: "RECOVERED_FINAL"));
response.write(streamChunk("chatcmpl-recovered", "", "stop"));
response.end("data: [DONE]\n\n");
});
Expand Down
3 changes: 0 additions & 3 deletions test/fixtures/runtime-keepalive.mjs

This file was deleted.

46 changes: 46 additions & 0 deletions test/node/headless-lifecycle.test.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
const { test } = require("node:test");
const assert = require("node:assert/strict");
const { spawnSync } = require("node:child_process");
const path = require("node:path");

const root = path.resolve(__dirname, "../..");

test("headless lifecycle keeps unreferenced work alive and releases its handle on success and failure", () => {
const child = spawnSync(process.execPath, ["-e", `
const { withHeadlessPromptLifecycle } = require("./vendor/cli-config.cjs");
const run = async () => {
const result = await withHeadlessPromptLifecycle(process, () => new Promise(resolve => {
setTimeout(() => resolve("settled"), 50).unref();
}));
process.stdout.write(result + "\\n");
try {
await withHeadlessPromptLifecycle(process, () => new Promise((resolve, reject) => {
setTimeout(() => reject(new Error("fixture failure")), 50).unref();
}));
} catch (error) {
process.stderr.write(error.message + "\\n");
}
};
run().catch(() => { process.exitCode = 1; });
`], { cwd: root, encoding: "utf8", timeout: 3_000 });

assert.ifError(child.error);
assert.equal(child.status, 0, child.stderr);
assert.equal(child.stdout, "settled\n");
assert.equal(child.stderr, "fixture failure\n");
});

test("headless output failures reject and do not leak the keepalive handle", () => {
const child = spawnSync(process.execPath, ["-e", `
const { Writable } = require("node:stream");
const { withHeadlessPromptLifecycle } = require("./vendor/cli-config.cjs");
const broken = new Writable({ write(chunk, encoding, callback) { callback(new Error("output failed")); } });
broken.on("error", () => {});
withHeadlessPromptLifecycle({ stdout: broken, stderr: process.stderr }, async () => 0)
.then(() => { process.exitCode = 1; }, error => { process.stderr.write(error.message); });
`], { cwd: root, encoding: "utf8", timeout: 3_000 });

assert.ifError(child.error);
assert.equal(child.status, 0, child.stderr);
assert.equal(child.stderr, "output failed");
});
29 changes: 29 additions & 0 deletions test/release-contract.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import { expect, test } from "bun:test";
import { parseRuntimeLock, selectRuntimeLock } from "../scripts/sync-runtime.ts";
import { parseReleaseVersion, syncedReleaseVersion } from "../scripts/release-version.ts";
import { validatePackResult, type PackResult } from "../scripts/pack-release.ts";
import { pluginLicenseFiles, publishedFiles } from "../scripts/package-contents.ts";

const manifest = await Bun.file(new URL("../package.json", import.meta.url)).json();
const lock = parseRuntimeLock(await Bun.file(new URL("../zcode-runtime.lock.json", import.meta.url)).json());
Expand All @@ -22,3 +24,30 @@ test("a lagging update manifest cannot downgrade the pinned runtime", () => {
test("a runtime upgrade preserves the CLI build number", () => {
expect(syncedReleaseVersion("9.0.0", "8.0.0-41")).toBe("9.0.0-41");
});

test("the tarball contract accepts published documentation and rejects missing or unreviewed files", () => {
const paths = [
...publishedFiles.filter((path) => path !== "vendor"),
...pluginLicenseFiles,
"package.json",
"vendor/extraction.json",
"vendor/node_modules/@zcode/tui/dist/index.js",
"vendor/node_modules/@zcode/tui/package.json",
"vendor/zcode.cjs",
"vendor/cli-config.cjs",
"vendor/provider/zcode-builtin.json"
];
const result: PackResult = {
name: manifest.name, version: manifest.version, filename: "fixture.tgz",
size: 0, unpackedSize: 0, integrity: "fixture", shasum: "fixture",
files: paths.map((path) => ({ path, mode: path === "bin/zcode.js" ? 0o755 : 0o644, size: 0 }))
};

expect(() => validatePackResult(result, manifest)).not.toThrow();
expect(() => validatePackResult({
...result, files: result.files.filter((file) => file.path !== "docs/HEADLESS.md")
}, manifest)).toThrow("npm tarball is missing: docs/HEADLESS.md");
expect(() => validatePackResult({
...result, files: [...result.files, { path: "docs/unreviewed.md", mode: 0o644, size: 0 }]
}, manifest)).toThrow("npm tarball contains an unreviewed path: docs/unreviewed.md");
});
Loading
Loading