diff --git a/README.md b/README.md index 45bd060..7c671f5 100644 --- a/README.md +++ b/README.md @@ -496,7 +496,7 @@ saved default: zcode --model account:zai-individual-coding-plan/GLM-5.3-Flash --effort high --prompt 'Reply with exactly: ok' ``` -`--model` requires a full provider/model ID and `--prompt` (or `-p`). It also +`--model` normally requires a full provider/model ID and `--prompt` (or `-p`). It also overrides a resumed session's model for that run. Unknown or unavailable models fail before a prompt is sent, rather than falling back to the default. @@ -511,6 +511,26 @@ run's model and effort are recorded on that session. For automation and long-running `-p` jobs, see [headless execution](./docs/HEADLESS.md) for output completion, cancellation, background execution and validation limits. +For CI with a custom model endpoint, configure the invocation through environment +variables; no Desktop installation, browser login or GitHub App is required: + +```bash +ZCODE_MODEL_BASE_URL=https://gateway.example.com/v1 \ +ZCODE_API_KEY="$CI_MODEL_API_KEY" \ +ZCODE_MODEL=organization/model-name \ +ZCODE_API_TYPE=openai-chat-completions \ +ZCODE_EFFORT=high \ +zcode -p "$(cat .github/prompts/review.md)" --output-format stream-json +``` + +In this mode `ZCODE_MODEL` and `--model` are raw server model IDs, including IDs +containing `/`. Explicit `--model` and `--effort` override their environment +counterparts. The provider is temporary and does not replace saved settings. +`ZCODE_MODEL_BASE_URL` is the inference endpoint; the existing `ZCODE_BASE_URL` +still controls the ZCode account service. See the [CI setup and security +guidance](./docs/HEADLESS.md#environment-configuration-for-ci) before running this +against repository-controlled code. + ZCode reads MCP servers, hooks, plugins, permissions, network, storage and display settings from `~/.zcode/cli/setting.json` (or `%USERPROFILE%\.zcode\cli\setting.json` on Windows). Project settings use diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 85b9c3d..04cba06 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -384,6 +384,13 @@ flow; ciphertext is not an API key. ## Custom provider +For unattended `--prompt` runs, `ZCODE_MODEL_BASE_URL`, `ZCODE_API_KEY`, +`ZCODE_MODEL`, `ZCODE_API_TYPE` and `ZCODE_EFFORT` provide a temporary custom +provider without writing personal settings. See [CI environment +configuration](HEADLESS.md#environment-configuration-for-ci) for precedence, +protocol mappings, GitHub Actions setup and security limits. `ZCODE_BASE_URL` +remains the account service URL, not the inference URL. + Use `provider.example.json` as a reference for a new provider file. Its enabled model inherits the upstream catalog; its disabled reference models demonstrate all smart-override and manual fields. Fill the empty API key, replace the diff --git a/docs/CONFIGURATION.zh-CN.md b/docs/CONFIGURATION.zh-CN.md index 9e65e71..0f72192 100644 --- a/docs/CONFIGURATION.zh-CN.md +++ b/docs/CONFIGURATION.zh-CN.md @@ -8,6 +8,17 @@ CLI 跟随当前 ZCode runtime 的 provider registry schema。Provider 和模型 ## 配置文件 +CI 等无人值守场景可在 `--prompt` / `-p` 运行时设置 `ZCODE_MODEL_BASE_URL`、 +`ZCODE_API_KEY`、`ZCODE_MODEL`、`ZCODE_API_TYPE` 和 `ZCODE_EFFORT`，无需预先登录 +或手写 provider 文件。自定义端点模式使用临时 provider，不覆盖个人配置；此时模型值是 +服务端原始 ID（可包含 `/`），`--model` / `--effort` 优先于对应环境变量。 +不配置自定义端点时，`ZCODE_MODEL` 与 `--model` 一样使用 `provider/model` 格式。 +`ZCODE_BASE_URL` 仍为账号服务地址，模型地址必须使用 `ZCODE_MODEL_BASE_URL`。 +协议默认 `openai-chat-completions`，也支持 `openai-responses` 和 +`anthropic-messages`。这些变量仅作用于 headless prompt，不改变 TUI 或 app-server。 +具体协议映射、GitHub Actions 示例和密钥隔离限制见 +[CI 环境变量配置](HEADLESS.md#environment-configuration-for-ci)。 + | 文件 | 用途 | | --- | --- | | `~/.zcode/cli/setting.json` | MCP、hooks、插件、权限、网络、存储和 CLI 显示设置 | diff --git a/docs/HEADLESS.md b/docs/HEADLESS.md index a41e511..c719e1f 100644 --- a/docs/HEADLESS.md +++ b/docs/HEADLESS.md @@ -3,6 +3,119 @@ `zcode -p 'task'` runs without a TUI. It is a foreground command, not a daemon, and headless mode is separate from the browser's `--browser-use=headless` setting. +## Environment configuration for CI + +Headless `--prompt` / `-p` invocations accept the following environment variables. +They do not configure the TUI, login, doctor, plugin commands or app-server. + +| Variable | Meaning | +| --- | --- | +| `ZCODE_MODEL_BASE_URL` | Custom inference API root, including its API prefix, e.g. `https://gateway.example.com/v1` | +| `ZCODE_API_KEY` | API key for that custom endpoint | +| `ZCODE_API_TYPE` | `openai-chat-completions` (default), `openai-responses`, or `anthropic-messages` | +| `ZCODE_MODEL` | Raw server model ID with a custom endpoint; otherwise a saved `provider/model` selection | +| `ZCODE_EFFORT` | Reasoning effort for this run; explicit `--effort` takes precedence | + +Blank variables are treated as unset. Specifying an endpoint, key or API type +activates custom-provider mode and requires an endpoint, key and model. `--model` +may supply the model instead of `ZCODE_MODEL`. In custom-provider mode it is a +raw server ID, not a saved provider selection, and `/` is preserved verbatim. +Without custom-provider mode, `--model` and `ZCODE_MODEL` require `provider/model`. +Explicit CLI model/effort flags take precedence over environment variables, which +take precedence over saved or resumed selections. Reapply the custom endpoint +environment when resuming a session created with it. + +`ZCODE_BASE_URL` retains its existing ZCode account/control-service meaning. It is +not an alias for `ZCODE_MODEL_BASE_URL`. A custom model endpoint must be HTTP(S) +without embedded credentials, query parameters or fragments. Use HTTPS for remote +services. The endpoint must implement the selected protocol, including streaming +and tool calling for agent tasks; a compatible-looking URL alone is insufficient. + +Custom-provider mode creates a private temporary provider file (directory mode +0700 and file mode 0600 on POSIX), containing only the supplied provider. It uses +the internal provider ID `cli-env` and does not overwrite the personal provider +file, saved default model or account credentials. The launcher removes its +temporary directory after success, failure or handled cancellation. `SIGKILL`, +runner failure or power loss cannot guarantee cleanup; prefer disposable runners +and do not archive temporary/runtime directories containing secrets. Normal +runtime session data may still be persisted; this is not an ephemeral-session or +secret-isolation feature. + +With a custom endpoint, omitted effort or `auto` sends no explicit reasoning +parameters. Explicit effort uses the selected protocol's mapping: + +- Chat Completions: `reasoning_effort`. +- Responses: `reasoning.effort`. +- Anthropic Messages: adaptive `thinking` with `output_config.effort`; + `none` / `disabled` disables thinking instead. + +The custom endpoint validates whether the model accepts that effort and mapping. +These are protocol presets, not automatic model capability discovery. Older +Anthropic models requiring token-budget thinking, gateway-specific mappings, +custom headers and model capability overrides should use the full +[provider configuration](PROVIDER_CONFIG.md) instead. Without custom-provider +mode, effort continues to use the saved provider's catalog and validation. +Invalid local configuration fails before starting the runtime. Provider errors +fail the job rather than falling back to an account or saved provider. + +### GitHub Actions example + +Create repository variables `AI_BASE_URL`, `AI_MODEL`, and optionally `AI_EFFORT`, +plus the secret `AI_API_KEY`. Add a trusted prompt at `.github/prompts/review.md`. +This minimal example runs only on manual dispatch, not arbitrary external PRs: + +```yaml +name: ZCode task +on: + workflow_dispatch: +permissions: + contents: read +jobs: + task: + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@v6 + with: + persist-credentials: false + - uses: actions/setup-node@v6 + with: + node-version: '24' + - name: Install CLI + run: npm install -g zcode-app-cli + - name: Run trusted task + env: + ZCODE_MODEL_BASE_URL: ${{ vars.AI_BASE_URL }} + ZCODE_API_KEY: ${{ secrets.AI_API_KEY }} + ZCODE_MODEL: ${{ vars.AI_MODEL }} + ZCODE_EFFORT: ${{ vars.AI_EFFORT }} + ZCODE_API_TYPE: openai-chat-completions + shell: bash + run: | + set -euo pipefail + zcode -p "$(cat .github/prompts/review.md)" \ + --mode build --disallowed-tools 'Bash Edit Write' \ + --output-format stream-json > "$RUNNER_TEMP/zcode-result.jsonl" + jq -s -e 'any(.[]; .type == "result")' "$RUNNER_TEMP/zcode-result.jsonl" > /dev/null +``` + +Pin CLI versions and action commit SHAs according to your deployment policy. +This example restricts common shell/edit tools for an inspection task; it is not +an OS sandbox or a comprehensive allowlist. Approval-requiring operations cannot +rely on the TUI in CI. Use `yolo` only for trusted workloads on isolated runners. +Successful execution is not a quality gate by itself; parse the task's outcome +separately if findings should fail CI. + +Pass keys only to the invocation/step that needs them, not a whole job containing +untrusted install/build scripts. Even step-scoped keys and private files can be +read by tools or repository-controlled code executed by the agent. Environment +configuration does not isolate credentials. Do not expose secrets to untrusted +forks or use `pull_request_target` to execute untrusted PR code. Review prompt +injection risks, project hooks, plugins and MCP configuration. Run GitHub writes +(comments/PR creation) separately with narrowly scoped GitHub credentials; those +credentials are distinct from the model API key. A GitHub App is not required +for this CLI-based integration. + ## Long-running jobs - The prompt lifecycle keeps the Node event loop alive until prompt work and diff --git a/src/launcher.ts b/src/launcher.ts index c8b0586..0d3018a 100644 --- a/src/launcher.ts +++ b/src/launcher.ts @@ -35,6 +35,7 @@ import { requestAppServer } from "./app-server-client.ts"; import { runPluginCommand } from "./plugin-cli.ts"; import { missingCodingPlanKey } from "./prompt-preflight.ts"; import { extractPromptOverrides } from "./prompt-model.ts"; +import { preparePromptEnvironment, resolvePromptEnvironment, type PromptEnvironment } from "./prompt-environment.ts"; import { capabilitiesFromExtractionMetadata, type RuntimeCliOptionType @@ -488,13 +489,11 @@ export async function main(args: string[]): Promise { return 0; } - let promptModel: string | undefined; - let promptEffort: string | undefined; + let promptSelection: PromptEnvironment; try { - const parsed = extractPromptOverrides(args, readRuntimeCliOptionTypes()); + const parsed = extractPromptOverrides(args, readRuntimeCliOptionTypes(), Boolean(process.env.ZCODE_MODEL_BASE_URL?.trim())); args = parsed.args; - promptModel = parsed.model; - promptEffort = parsed.effort; + promptSelection = resolvePromptEnvironment(parsed); } catch (error) { console.error(`Error: ${error instanceof Error ? error.message : String(error)}`); return 1; @@ -520,7 +519,7 @@ export async function main(args: string[]): Promise { } const node = resolveNodeExecutable(); - if (providerMigrationNeeded()) { + if (!promptSelection.provider && providerMigrationNeeded()) { try { await new Promise((resolve, reject) => { const child = spawnChild(node, [runtimePath, "--version"], { @@ -633,8 +632,11 @@ export async function main(args: string[]): Promise { } } + let preparedPrompt: Awaited> | undefined; try { - const diagnostic = await readConfiguredModelAccess() ? undefined : await promptPreflight(login.args, process.env, promptModel); + preparedPrompt = await preparePromptEnvironment(promptSelection); + const promptEnv = { ...process.env, ...preparedPrompt.env }; + const diagnostic = await readConfiguredModelAccess(promptEnv) ? undefined : await promptPreflight(login.args, promptEnv, promptSelection.model); if (diagnostic) { console.error(diagnostic); return 1; @@ -642,16 +644,18 @@ export async function main(args: string[]): Promise { const runtimeArgs = withDefaultBrowserUse(login.args); const code = await runRuntime(node, runtimeArgs, { ...firstRunSetupEnv(setupPending, runtimeArgs), - ...(promptModel ? { ZCODE_CLI_PROMPT_MODEL: promptModel } : {}), - ...(promptEffort ? { ZCODE_CLI_PROMPT_EFFORT: promptEffort } : {}) + ...preparedPrompt.env }); if (code === 0 && (args.includes("--help") || args.includes("-h"))) { - console.log("\n --model Select a model for this --prompt run without changing the saved default."); + console.log("\n --model Select a model for this --prompt run; with ZCODE_MODEL_BASE_URL, use a raw server model ID."); console.log(" --effort Select the model reasoning effort for this --prompt run; auto keeps the default."); + console.log("\n Headless environment: ZCODE_MODEL, ZCODE_EFFORT; custom endpoint: ZCODE_MODEL_BASE_URL, ZCODE_API_KEY, ZCODE_API_TYPE."); } return code; } catch (error) { console.error(`Error: ${error instanceof Error ? error.message : String(error)}`); return 1; + } finally { + await preparedPrompt?.dispose(); } } diff --git a/src/prompt-environment.ts b/src/prompt-environment.ts new file mode 100644 index 0000000..4befa5a --- /dev/null +++ b/src/prompt-environment.ts @@ -0,0 +1,104 @@ +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import type { PromptOverrides } from "./prompt-model.ts"; + +const environmentProviderId = "cli-env"; +const apiTypes = ["openai-chat-completions", "openai-responses", "anthropic-messages"] as const; +type ApiType = typeof apiTypes[number]; + +export interface PromptEnvironment { + model?: string; + effort?: string; + provider?: { baseUrl: string; apiKey: string; modelId: string; apiType: ApiType }; +} + +export function resolvePromptEnvironment( + overrides: PromptOverrides, env: NodeJS.ProcessEnv = process.env +): PromptEnvironment { + if (!overrides.headless) return { model: overrides.model, effort: overrides.effort }; + const model = overrides.model ?? (env.ZCODE_MODEL?.trim() || undefined); + const effort = overrides.effort ?? (env.ZCODE_EFFORT?.trim() || undefined); + if (effort && (!/^[a-z][a-z0-9_-]*$/iu.test(effort))) { + throw new Error("ZCODE_EFFORT must be a single reasoning effort level."); + } + const baseUrl = env.ZCODE_MODEL_BASE_URL?.trim(); + const apiKey = env.ZCODE_API_KEY?.trim(); + const apiType = env.ZCODE_API_TYPE?.trim() || "openai-chat-completions"; + const customProvider = Boolean(baseUrl || apiKey || env.ZCODE_API_TYPE?.trim()); + if (!customProvider) { + if (model && !/^[^/\s]+\/\S+$/u.test(model)) { + throw new Error("ZCODE_MODEL must be a provider/model ID unless ZCODE_MODEL_BASE_URL is configured."); + } + return { model, effort }; + } + if (!baseUrl || !apiKey || !model) { + throw new Error("Custom model access requires ZCODE_MODEL_BASE_URL, ZCODE_API_KEY and ZCODE_MODEL (or --model)."); + } + let url: URL; + try { url = new URL(baseUrl); } catch { + throw new Error("ZCODE_MODEL_BASE_URL must be an absolute HTTP(S) URL."); + } + if (!["https:", "http:"].includes(url.protocol) || url.username || url.password || url.search || url.hash) { + throw new Error("ZCODE_MODEL_BASE_URL must be an HTTP(S) URL without credentials, query or fragment."); + } + if (!apiTypes.includes(apiType as ApiType)) { + throw new Error(`ZCODE_API_TYPE must be one of: ${apiTypes.join(", ")}.`); + } + if (/\s/u.test(model) || model.startsWith("-")) throw new Error("Custom model ID must not contain whitespace or start with '-'."); + if (/[\r\n]/u.test(apiKey)) throw new Error("ZCODE_API_KEY must not contain line breaks."); + return { + model: `${environmentProviderId}/${model}`, effort, + provider: { baseUrl, apiKey, modelId: model, apiType: apiType as ApiType } + }; +} + +export async function preparePromptEnvironment(selection: PromptEnvironment): Promise<{ + env: NodeJS.ProcessEnv; + dispose(): Promise; +}> { + const env: NodeJS.ProcessEnv = { + ...(selection.model ? { ZCODE_CLI_PROMPT_MODEL: selection.model } : {}), + ...(selection.effort ? { ZCODE_CLI_PROMPT_EFFORT: selection.effort } : {}) + }; + if (!selection.provider) return { env, dispose: async () => {} }; + const { baseUrl, apiKey, modelId, apiType } = selection.provider; + const effort = selection.effort?.toLowerCase(); + const explicitEffort = effort && effort !== "auto" ? effort : undefined; + const reasoningMap = !explicitEffort ? "{}" : apiType === "openai-chat-completions" + ? '{"reasoning_effort": reasoningLevel}' + : apiType === "openai-responses" ? '{"reasoning": {"effort": reasoningLevel}}' + : 'reasoningLevel == "disabled" || reasoningLevel == "none" ? {"thinking": {"type": "disabled"}} : {"thinking": {"type": "adaptive"}, "output_config": {"effort": reasoningLevel}}'; + const providerConfig = { + schemaVersion: 1, + config: { + providerConfigRules: { providerRules: [{ + providerId: environmentProviderId, providerName: "Environment model", enabled: true, + config: { + group: "standard-personal", access: { type: "api-key", apiKey }, + api: { type: apiType, baseUrl }, personalModelIds: [modelId] + } + }] }, + modelConfigRules: { + providerModelRules: [{ providerId: environmentProviderId, modelId, config: { + enabled: true, optionSpecs: { reasoningLevel: { + values: [explicitEffort ?? "default"], map: reasoningMap + } } + } }], + manualProviderModelRules: [] + }, + defaultModelSelection: { providerId: environmentProviderId, modelId } + } + }; + const directory = await mkdtemp(join(tmpdir(), "zcode-prompt-provider-")); + const dispose = async () => { await rm(directory, { recursive: true, force: true }); }; + try { + const path = join(directory, "provider_config.json"); + await writeFile(path, JSON.stringify(providerConfig), { mode: 0o600 }); + env.ZCODE_PERSONAL_PROVIDER_CONFIG_FILE = path; + return { env, dispose }; + } catch (error) { + await dispose(); + throw error; + } +} diff --git a/src/prompt-model.ts b/src/prompt-model.ts index 205b4c6..4edde7b 100644 --- a/src/prompt-model.ts +++ b/src/prompt-model.ts @@ -5,6 +5,7 @@ export interface PromptOverrides { args: string[]; effort?: string; model?: string; + headless: boolean; } function removalIndexes(token: { index: number; inlineValue?: boolean }): number[] { @@ -12,7 +13,7 @@ function removalIndexes(token: { index: number; inlineValue?: boolean }): number } export function extractPromptOverrides( - args: string[], optionTypes: Readonly> + args: string[], optionTypes: Readonly>, allowBareModel = false ): PromptOverrides { const options = Object.fromEntries(Object.entries(optionTypes).map(([name, type]) => [name, { type }])); const parsed = parseArgs({ @@ -22,6 +23,8 @@ export function extractPromptOverrides( tokens: true, options: { ...options, + help: { type: "boolean", short: "h" }, + version: { type: "boolean", short: "v" }, prompt: { type: "string", short: "p" }, model: { type: "string" }, effort: { type: "string" } @@ -29,7 +32,8 @@ export function extractPromptOverrides( }); const modelTokens = parsed.tokens.filter(token => token.kind === "option" && token.name === "model"); const effortTokens = parsed.tokens.filter(token => token.kind === "option" && token.name === "effort"); - if (modelTokens.length === 0 && effortTokens.length === 0) return { args }; + const headless = typeof parsed.values.prompt === "string" && !parsed.values.help && !parsed.values.version; + if (modelTokens.length === 0 && effortTokens.length === 0) return { args, headless: Boolean(headless) }; if (modelTokens.length !== 0 && modelTokens.length !== 1) throw new Error("Specify --model only once."); if (effortTokens.length !== 0 && effortTokens.length !== 1) throw new Error("Specify --effort only once."); if (typeof parsed.values.prompt !== "string") { @@ -42,7 +46,9 @@ export function extractPromptOverrides( if (modelTokens.length === 1) { if (typeof parsed.values.model !== "string") throw new Error("--model must be a provider/model ID."); model = parsed.values.model.trim(); - if (!/^[^/\s]+\/\S+$/u.test(model)) throw new Error("--model must be a provider/model ID."); + if (allowBareModel ? !model || /\s/u.test(model) || model.startsWith("-") : !/^[^/\s]+\/\S+$/u.test(model)) { + throw new Error(allowBareModel ? "--model must be a model ID." : "--model must be a provider/model ID."); + } for (const index of removalIndexes(modelTokens[0]!)) removed.add(index); } if (effortTokens.length === 1) { @@ -53,5 +59,5 @@ export function extractPromptOverrides( } for (const index of removalIndexes(effortTokens[0]!)) removed.add(index); } - return { args: args.filter((_, index) => !removed.has(index)), model, effort }; + return { args: args.filter((_, index) => !removed.has(index)), model, effort, headless: Boolean(headless) }; } diff --git a/test/prompt-environment.test.ts b/test/prompt-environment.test.ts new file mode 100644 index 0000000..e1c52ca --- /dev/null +++ b/test/prompt-environment.test.ts @@ -0,0 +1,117 @@ +import { expect, test } from "bun:test"; +import { readFile, stat } from "node:fs/promises"; +import { extractPromptOverrides } from "../src/prompt-model.ts"; +import { preparePromptEnvironment, resolvePromptEnvironment } from "../src/prompt-environment.ts"; + +const customEnvironment = { + ZCODE_MODEL_BASE_URL: "https://gateway.example.test/v1", + ZCODE_API_KEY: "fixture-secret", + ZCODE_MODEL: "organization/custom-model", + ZCODE_EFFORT: "high" +}; + +function resolve(args: string[], env: NodeJS.ProcessEnv = {}) { + return resolvePromptEnvironment(extractPromptOverrides(args, {}, Boolean(env.ZCODE_MODEL_BASE_URL?.trim())), env); +} + +test("uses environment model and effort only for headless prompts", () => { + expect(resolve(["-p", "task"], { ZCODE_MODEL: "custom/model", ZCODE_EFFORT: "high" })) + .toEqual({ model: "custom/model", effort: "high" }); + for (const args of [[], ["tui"], ["app-server"], ["doctor"], ["--help"], ["-p", "task", "-h"], ["-p", "task", "--help"]]) { + expect(resolve(args, { ...customEnvironment, ZCODE_API_TYPE: "invalid" }).provider).toBeUndefined(); + expect(resolve(args, customEnvironment).model).toBeUndefined(); + } +}); + +test("explicit arguments override environment model and effort", () => { + const selection = resolve(["-p", "task", "--model", "different/model", "--effort", "auto"], customEnvironment); + expect(selection.model).toBe("cli-env/different/model"); + expect(selection.provider?.modelId).toBe("different/model"); + expect(selection.effort).toBe("auto"); + expect(resolve(["-p", "task", "--model=custom/other", "--effort=low"], { + ZCODE_MODEL: "invalid model", ZCODE_EFFORT: "invalid effort" + })).toEqual({ model: "custom/other", effort: "low" }); + expect(resolve(["-p", "task", "--model", "bare-model"], customEnvironment).provider?.modelId).toBe("bare-model"); +}); + +test("leaves blank environment settings unset and preserves the account base URL", () => { + expect(resolve(["-p", "task"], { ZCODE_MODEL: " ", ZCODE_EFFORT: " ", ZCODE_API_KEY: " ", ZCODE_API_TYPE: " " })) + .toEqual({ model: undefined, effort: undefined }); + expect(resolve(["-p", "task"], { ZCODE_BASE_URL: "https://account.example.test" }).provider).toBeUndefined(); + expect(resolve(["-p", "task"], { ...customEnvironment, ZCODE_BASE_URL: "https://account.example.test" }).provider?.baseUrl) + .toBe(customEnvironment.ZCODE_MODEL_BASE_URL); +}); + +test("rejects partial or invalid configuration without including credentials in errors", () => { + for (const env of [ + { ZCODE_API_KEY: "secret" }, { ZCODE_MODEL_BASE_URL: "https://example.test" }, + { ...customEnvironment, ZCODE_MODEL: "" }, { ...customEnvironment, ZCODE_API_KEY: "" }, + { ...customEnvironment, ZCODE_API_TYPE: "invalid" }, { ...customEnvironment, ZCODE_MODEL: "bad model" }, + { ...customEnvironment, ZCODE_API_KEY: "secret\nheader" }, { ZCODE_MODEL: "bare-model" }, + { ZCODE_EFFORT: "high effort" }, { ...customEnvironment, ZCODE_EFFORT: "--high" } + ]) expect(() => resolve(["-p", "task"], env)).toThrow(); + for (const baseUrl of ["invalid", "file:///secret", "https://user:secret@example.test", "https://example.test?key=secret", "https://example.test/#secret"]) { + try { + resolve(["-p", "task"], { ...customEnvironment, ZCODE_MODEL_BASE_URL: baseUrl }); + throw new Error("accepted invalid URL"); + } catch (error) { + expect(String(error)).toContain("ZCODE_MODEL_BASE_URL"); + expect(String(error)).not.toContain("secret"); + } + } +}); + +test.each([ + ["openai-chat-completions", '{"reasoning_effort": reasoningLevel}'], + ["openai-responses", '{"reasoning": {"effort": reasoningLevel}}'], + ["anthropic-messages", 'reasoningLevel == "disabled" || reasoningLevel == "none" ? {"thinking": {"type": "disabled"}} : {"thinking": {"type": "adaptive"}, "output_config": {"effort": reasoningLevel}}'] +])("creates a private disposable provider for %s", async (apiType, mapping) => { + const selection = resolve(["-p", "task"], { ...customEnvironment, ZCODE_API_TYPE: apiType }); + const prepared = await preparePromptEnvironment(selection); + const path = prepared.env.ZCODE_PERSONAL_PROVIDER_CONFIG_FILE!; + try { + const config = JSON.parse(await readFile(path, "utf8")).config; + const provider = config.providerConfigRules.providerRules[0]; + expect(provider.providerId).toBe("cli-env"); + expect(provider.config.api).toEqual({ type: apiType, baseUrl: customEnvironment.ZCODE_MODEL_BASE_URL }); + expect(provider.config.access).toEqual({ type: "api-key", apiKey: "fixture-secret" }); + expect(provider.config.personalModelIds).toEqual(["organization/custom-model"]); + expect(config.modelConfigRules.providerModelRules[0].config.optionSpecs.reasoningLevel) + .toEqual({ values: ["high"], map: mapping }); + expect(prepared.env.ZCODE_CLI_PROMPT_MODEL).toBe("cli-env/organization/custom-model"); + expect(prepared.env.ZCODE_CLI_PROMPT_EFFORT).toBe("high"); + if (process.platform !== "win32") expect((await stat(path)).mode & 0o777).toBe(0o600); + } finally { await prepared.dispose(); } + expect(await Bun.file(path).exists()).toBe(false); +}); + +test.each([undefined, "auto"])("does not send reasoning parameters when effort is %s", async effort => { + const selection = resolve(["-p", "task"], { ...customEnvironment, ZCODE_EFFORT: effort }); + const prepared = await preparePromptEnvironment(selection); + try { + const config = JSON.parse(await readFile(prepared.env.ZCODE_PERSONAL_PROVIDER_CONFIG_FILE!, "utf8")); + expect(config.config.modelConfigRules.providerModelRules[0].config.optionSpecs.reasoningLevel) + .toEqual({ values: ["default"], map: "{}" }); + } finally { await prepared.dispose(); } +}); + +test("concurrent runs use independent configuration files", async () => { + const selection = resolve(["-p", "task"], customEnvironment); + const first = await preparePromptEnvironment(selection); + const second = await preparePromptEnvironment(selection); + try { + expect(first.env.ZCODE_PERSONAL_PROVIDER_CONFIG_FILE).not.toBe(second.env.ZCODE_PERSONAL_PROVIDER_CONFIG_FILE); + await first.dispose(); + expect(await Bun.file(second.env.ZCODE_PERSONAL_PROVIDER_CONFIG_FILE!).exists()).toBe(true); + } finally { await Promise.all([first.dispose(), second.dispose()]); } +}); + +test("existing CLI argument validation remains strict outside custom endpoint mode", () => { + for (const args of [ + ["--model", "custom/model"], ["-p", "task", "--model", "bare-model"], + ["-p", "task", "--model", "one/model", "--model", "two/model"], + ["-p", "task", "--effort", "high", "--effort", "low"], ["-p", "task", "--effort", "bad level"] + ]) expect(() => resolve(args)).toThrow(); + expect(resolve(["-p", "task", "--effort", "low"], { ZCODE_MODEL: "saved/model" })) + .toEqual({ model: "saved/model", effort: "low" }); +}); diff --git a/test/runtime/prompt-environment.test.ts b/test/runtime/prompt-environment.test.ts new file mode 100644 index 0000000..a4d1498 --- /dev/null +++ b/test/runtime/prompt-environment.test.ts @@ -0,0 +1,223 @@ +import { expect, test } from "bun:test"; +import { chmod, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { dirname, join, resolve } from "node:path"; +import { ensureCliSettings } from "../../src/model-access.ts"; +import { writeProviderFixture } from "../fixtures/provider-config.ts"; +import { runtimeTestEnv } from "../fixtures/runtime-env.ts"; + +const root = resolve(import.meta.dir, "../.."); + +async function runFixture(options: { + apiType?: string; effort?: string; args?: string[]; fail?: boolean; noSavedProvider?: boolean; + savedSelection?: boolean; +} = {}) { + const home = await mkdtemp(join(tmpdir(), "zcode-environment-runtime-")); + const requests: { path: string; authorization: string | null; apiKey: string | null; body: Record }[] = []; + const apiType = options.apiType ?? "openai-chat-completions"; + const server = Bun.serve({ hostname: "127.0.0.1", port: 0, async fetch(request) { + const body = await request.json() as Record; + requests.push({ path: new URL(request.url).pathname, authorization: request.headers.get("authorization"), apiKey: request.headers.get("x-api-key"), body }); + if (options.fail) return Response.json({ error: { message: "ENV_PROVIDER_REJECTED" } }, { status: 401 }); + if (!body.stream) return Response.json({ choices: [{ message: { role: "assistant", content: "ENV_REPLY" }, finish_reason: "stop" }] }); + if (apiType === "openai-responses") { + const item = { id: "msg_fixture", type: "message", role: "assistant", status: "completed", + content: [{ type: "output_text", text: "ENV_REPLY", annotations: [] }] }; + const response = { id: "resp_fixture", object: "response", created_at: 1, model: body.model, + status: "completed", output: [item], usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 } }; + const events = [ + { type: "response.created", response: { ...response, status: "in_progress", output: [] } }, + { type: "response.output_item.added", output_index: 0, item: { ...item, status: "in_progress", content: [] } }, + { type: "response.content_part.added", item_id: item.id, output_index: 0, content_index: 0, part: { type: "output_text", text: "", annotations: [] } }, + { type: "response.output_text.delta", item_id: item.id, output_index: 0, content_index: 0, delta: "ENV_REPLY" }, + { type: "response.output_text.done", item_id: item.id, output_index: 0, content_index: 0, text: "ENV_REPLY" }, + { type: "response.output_item.done", output_index: 0, item }, + { type: "response.completed", response } + ]; + return new Response(events.map(event => `event: ${event.type}\ndata: ${JSON.stringify(event)}\n\n`).join(""), { headers: { "content-type": "text/event-stream" } }); + } + if (apiType === "anthropic-messages") { + const events = [ + { type: "message_start", message: { id: "msg_fixture", type: "message", role: "assistant", model: body.model, content: [], stop_reason: null, usage: { input_tokens: 1, output_tokens: 0 } } }, + { type: "content_block_start", index: 0, content_block: { type: "text", text: "" } }, + { type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "ENV_REPLY" } }, + { type: "content_block_stop", index: 0 }, + { type: "message_delta", delta: { stop_reason: "end_turn", stop_sequence: null }, usage: { output_tokens: 1 } }, + { type: "message_stop" } + ]; + return new Response(events.map(event => `event: ${event.type}\ndata: ${JSON.stringify(event)}\n\n`).join(""), { headers: { "content-type": "text/event-stream" } }); + } + const chunks = [{ role: "assistant", content: "ENV_REPLY" }, {}].map((delta, index) => + `data: ${JSON.stringify({ id: "fixture", object: "chat.completion.chunk", model: body.model, + choices: [{ index: 0, delta, finish_reason: index ? "stop" : null }] })}\n\n`).join(""); + return new Response(`${chunks}data: [DONE]\n\n`, { headers: { "content-type": "text/event-stream" } }); + } }); + try { + const env = runtimeTestEnv(home); + const saved = options.noSavedProvider ? undefined : await writeProviderFixture(env, { + providerId: "saved", modelId: "saved-model", models: ["saved-model", "environment-selected-model"], apiKey: "saved-key", + baseUrl: options.savedSelection ? `http://127.0.0.1:${server.port}/v1` : "http://127.0.0.1:1/v1" + }); + const original = saved && await readFile(saved.path, "utf8"); + const bootstrap = await ensureCliSettings(env); + const settings = JSON.parse(await readFile(bootstrap.configPath, "utf8")); + settings.storage = { dir: join(home, ".zcode"), sessionDbPath: join(home, "sessions.sqlite") }; + settings.memory = { use: false, write: false, autoConsolidate: false }; + settings.features = { compact: false, rewind: false, subagent: false, skill: false, mcp: false }; + settings.plugins = { enabled: false, dirs: [], enabledPlugins: {}, options: {}, suppressedBuiltins: [] }; + await writeFile(bootstrap.configPath, JSON.stringify(settings)); + if (!options.savedSelection) await writeFile(join(dirname(bootstrap.configPath), "config.json"), JSON.stringify({ + provider: { legacy: { options: { apiKey: "legacy-key", baseURL: "http://127.0.0.1:1/v1" } } } + })); + const child = Bun.spawn([Bun.which("node")!, "bin/zcode.js", "-p", "Reply with ENV_REPLY.", "--cwd", home, + "--mode", "build", "--output-format", "stream-json", ...(options.args ?? [])], { + cwd: root, env: { + ...env, TMPDIR: home, TMP: home, TEMP: home, + ZCODE_MODEL_BASE_URL: options.savedSelection ? undefined : `http://127.0.0.1:${server.port}/v1`, + ZCODE_API_KEY: options.savedSelection ? undefined : "environment-key", + ZCODE_API_TYPE: options.savedSelection ? undefined : apiType, + ZCODE_MODEL: options.savedSelection ? "saved/environment-selected-model" : "organization/custom-model", + ZCODE_EFFORT: options.effort, + ZCODE_MODEL_RETRY_MAX_RETRIES: "0" + }, stdin: "ignore", stdout: "pipe", stderr: "pipe" + }); + const timeout = setTimeout(() => child.kill("SIGKILL"), 20_000); + let code: number, stdout: string, stderr: string; + try { [code, stdout, stderr] = await Promise.all([child.exited, new Response(child.stdout).text(), new Response(child.stderr).text()]); } + finally { clearTimeout(timeout); } + if (saved) expect(await readFile(saved.path, "utf8")).toBe(original!); + else expect(await Bun.file(env.ZCODE_PERSONAL_PROVIDER_CONFIG_FILE!).exists()).toBe(false); + expect((await readdir(home)).filter(name => name.startsWith("zcode-prompt-provider-"))).toEqual([]); + expect(stdout).not.toContain("environment-key"); + expect(stderr).not.toContain("environment-key"); + return { code, stdout, stderr, requests }; + } finally { + server.stop(true); + await rm(home, { recursive: true, force: true }); + } +} + +test("headless environment credentials run without a saved provider or browser login", async () => { + const result = await runFixture({ noSavedProvider: true, effort: "high" }); + expect(result.code, result.stderr).toBe(0); + expect(result.stdout).toContain('"type":"result"'); + expect(result.stdout).toContain("ENV_REPLY"); + expect(result.requests.some(request => request.body.stream)).toBe(true); + for (const request of result.requests.filter(request => request.body.stream)) { + expect(request.path).toBe("/v1/chat/completions"); + expect(request.authorization).toBe("Bearer environment-key"); + expect(request.body.model).toBe("organization/custom-model"); + expect(request.body.reasoning_effort).toBe("high"); + expect(request.body.thinking).toBeUndefined(); + } +}, 30_000); + +test("explicit raw model and effort override environment without changing saved providers", async () => { + const result = await runFixture({ effort: "high", args: ["--model", "different/model", "--effort", "low"] }); + expect(result.code, result.stderr).toBe(0); + const request = result.requests.find(request => request.body.stream)!; + expect(request.body.model).toBe("different/model"); + expect(request.body.reasoning_effort).toBe("low"); +}, 30_000); + +test.each([undefined, "auto"])("omits reasoning parameters with effort %s", async effort => { + const result = await runFixture({ effort }); + expect(result.code, result.stderr).toBe(0); + const request = result.requests.find(request => request.body.stream)!; + expect(request.body.reasoning_effort).toBeUndefined(); + expect(request.body.reasoning).toBeUndefined(); + expect(request.body.thinking).toBeUndefined(); +}, 30_000); + +test("Anthropic-compatible environment credentials and effort reach the custom endpoint", async () => { + const result = await runFixture({ apiType: "anthropic-messages", effort: "high" }); + expect(result.code, result.stderr).toBe(0); + const request = result.requests.find(request => request.body.stream)!; + expect(request.path).toBe("/v1/messages"); + expect(request.apiKey).toBe("environment-key"); + expect(request.body.model).toBe("organization/custom-model"); + expect(request.body.thinking).toEqual({ type: "adaptive" }); + expect(request.body.output_config).toEqual({ effort: "high" }); +}, 30_000); + +test("Responses-compatible environment credentials and effort reach the custom endpoint", async () => { + const result = await runFixture({ apiType: "openai-responses", effort: "high" }); + expect(result.code, result.stderr).toBe(0); + expect(result.stdout).toContain("ENV_REPLY"); + const request = result.requests.find(request => request.body.stream)!; + expect(request.path).toBe("/v1/responses"); + expect(request.authorization).toBe("Bearer environment-key"); + expect(request.body.model).toBe("organization/custom-model"); + expect(request.body.reasoning).toEqual({ effort: "high" }); + expect(request.body.reasoning_effort).toBeUndefined(); +}, 30_000); + +test("provider rejection fails without falling back and cleans up temporary credentials", async () => { + const result = await runFixture({ fail: true }); + expect(result.code).not.toBe(0); + expect(result.requests.length).toBeGreaterThan(0); + expect(`${result.stdout}\n${result.stderr}`).toContain("ENV_PROVIDER_REJECTED"); +}, 30_000); + +test("environment model selects a saved provider when no custom endpoint is configured", async () => { + const result = await runFixture({ savedSelection: true }); + expect(result.code, result.stderr).toBe(0); + const request = result.requests.find(request => request.body.stream)!; + expect(request.body.model).toBe("environment-selected-model"); + expect(request.authorization).toBe("Bearer saved-key"); +}, 30_000); + +test.skipIf(process.platform === "win32")("handled cancellation removes temporary provider credentials", async () => { + const home = await mkdtemp(join(tmpdir(), "zcode-environment-cancel-")); + const fakeNode = join(home, "fake-node"); + await writeFile(fakeNode, [ + `#!${Bun.which("node")!}`, + "process.on('SIGTERM', () => process.exit(143));", + "setInterval(() => {}, 1000);", + "process.stdout.write(process.env.ZCODE_PERSONAL_PROVIDER_CONFIG_FILE + '\\n');", + "" + ].join("\n")); + await chmod(fakeNode, 0o755); + const child = Bun.spawn([Bun.which("node")!, "bin/zcode.js", "-p", "Wait for cancellation.", "--cwd", home], { + cwd: root, env: { + ...runtimeTestEnv(home), ZCODE_NODE: fakeNode, TMPDIR: home, + ZCODE_MODEL_BASE_URL: "https://example.test/v1", ZCODE_API_KEY: "cancellation-secret", ZCODE_MODEL: "model" + }, stdin: "ignore", stdout: "pipe", stderr: "pipe" + }); + const timeout = setTimeout(() => child.kill("SIGKILL"), 10_000); + try { + const reader = child.stdout.getReader(); + const first = await reader.read(); + const path = new TextDecoder().decode(first.value).trim(); + expect(path).toContain("zcode-prompt-provider-"); + expect(await Bun.file(path).exists()).toBe(true); + child.kill("SIGTERM"); + expect(await child.exited).toBe(143); + expect(await Bun.file(path).exists()).toBe(false); + reader.releaseLock(); + } finally { + clearTimeout(timeout); + child.kill("SIGKILL"); + await child.exited; + await rm(home, { recursive: true, force: true }); + } +}, 15_000); + +test("incomplete custom environment fails before creating settings or launching the runtime", async () => { + const home = await mkdtemp(join(tmpdir(), "zcode-environment-invalid-")); + try { + const child = Bun.spawn([Bun.which("node")!, "bin/zcode.js", "-p", "Must not run."], { + cwd: root, env: { + ...runtimeTestEnv(home), ZCODE_NODE: join(home, "nonexistent-node"), + ZCODE_API_KEY: "incomplete-secret", ZCODE_MODEL: "raw-model" + }, stdin: "ignore", stdout: "pipe", stderr: "pipe" + }); + const [code, stdout, stderr] = await Promise.all([child.exited, new Response(child.stdout).text(), new Response(child.stderr).text()]); + expect(code).toBe(1); + expect(stdout).toBe(""); + expect(stderr).toContain("Custom model access requires"); + expect(stderr).not.toContain("incomplete-secret"); + expect(stderr).not.toContain("ENOENT"); + expect(await readdir(home)).toEqual([]); + } finally { await rm(home, { recursive: true, force: true }); } +});