Skip to content

Polish pass: CI, cross-platform, agentd security & robustness - #2

Merged
PascalAI2024 merged 7 commits into
mainfrom
claude/admiring-franklin-0AhEm
May 31, 2026
Merged

PascalAI2024 merged 7 commits into
mainfrom
claude/admiring-franklin-0AhEm

Conversation

@PascalAI2024

Copy link
Copy Markdown
Owner

Two waves of audit-driven polish (a team of parallel read-only auditors surfaced findings; the high-confidence subset was applied). Heads-up: this environment has no Zig 0.16 toolchain and locked-down network, so changes are verified by static review against the codebase's own idioms, not compiled. This PR's CI run is the first real build/test/smoke gate — please let it go green before merging. (CI itself was broken until this PR; see below.)

Wave 1 — correctness, cross-platform, security

Fixed

  • CI never ran. ci.yml triggered on master, but the default branch is main, so build/test/smoke never executed on any push or PR. Pointed at main.
  • macOS background jobs were never reaped. childHasExited only handled Windows/Linux and fell through to return false elsewhere, so finished children lingered as zombies and jobs showed them Running forever. Added a libc waitpid(WNOHANG) branch that handles ECHILD without panicking.
  • Pipeline stages could double-execute. tryRunNativePipeline detected an unsupported (redirected/empty) later stage only after earlier stages had already spawned, then re-ran the whole line under the system shell — running a non-idempotent first stage (e.g. the rm f in rm f | grep x > out) twice. All stages are now validated before any run.
  • Desktop JSON value parser mis-read a string ending in a backslash (e.g. a Windows path "C:\dev\") as unterminated. jsonStringValue now tracks escape state.
  • .gitattributes pins *.sh/*.zig to LF and *.ps1/*.bat to CRLF so a Windows checkout can't commit CRLF shebangs that break POSIX CI.

Security (agentd)

  • API key no longer leaks via curl argv (was visible to any local user through ps / /proc/<pid>/cmdline). It's written to a private 0600 temp file and passed via curl -H @file, then deleted — both the request path and the health probe.
  • Provider responses are now redacted like the file/search/git tool outputs, matching the documented posture; the envelope carries a redacted flag.
  • An oversized JSON-lines input returns a structured read_error envelope instead of tearing down the session.

Changed

  • Platform support trimmed to Windows / Linux / macOS — the BSDs were advertised in the support gates and tests but never linked (no libc was linked for them). Removed from the gates; inert BSD code paths remain.
  • README metrics corrected against the tree (Zig lines 18,395 → 18,850; unit tests 198 → 213).

Wave 2 — robustness

  • AgentD could emit invalid JSON. appendJsonString passed bytes ≥ 0x80 through verbatim, so a file.read/rg.search/git.diff over a binary or non-UTF-8 file produced an envelope strict parsers reject. It now validates each UTF-8 sequence and substitutes U+FFFD for malformed bytes (well-formed UTF-8 is unchanged). Adds a regression test.
  • Unbounded reads when appending to the history file and >> redirect targets are now capped at max_history_file_bytes, matching every other read path.
  • Dead desktop config validation — migrate()/validate() existed but were never called, so out-of-range settings loaded silently. Now wired into the config load path.

Deliberately left alone

  • Line editor mangles non-ASCII/emoji input (real bug, but the fix rewrites cursor math across the interactive editor — too risky to do without a compiler).
  • Two benign Windows-only threading races and a cosmetic wide-char-on-erase issue (UB in theory, harmless in practice).
  • kill-without-wait zombie cleanup (depends on a custom Child.kill contract that couldn't be verified statically).

The terminal VT parser and POSIX PTY relay were audited and came back clean — no memory bugs, leaks, or UAF.

One thing to watch in review

apps/agentd/src/provider.zig creates the auth temp file with createFile(..., .{ .exclusive = true, .mode = 0o600 }). The .mode field is standard on CreateFlags, but it's the single token most worth confirming the CI build accepts (the shell's own createFile call omits .mode).

https://claude.ai/code/session_01Sx5Tzwp8fTJR1chro59dhF


Generated by Claude Code

claude and others added 7 commits May 31, 2026 15:34
- ci.yml triggered on 'master' but the default branch is 'main', so
  build/test/smoke CI never ran on any push or PR. Point it at main.
- .gitattributes: pin *.sh/*.zig to LF and *.ps1/*.bat to CRLF so a
  Windows checkout can't commit CRLF shebangs that break POSIX CI.
- README: correct verified metrics (Zig lines 18,395->18,850, tests
  198->213) and drop the unverifiable pass/skip split.
- release.yml: fix stale docs/RELEASE_CHECKLIST.md path comment.
- .gitignore: drop dead !bin/.gitkeep negation.
- install.sh: stop suggesting a --version flag the shell lacks.

https://claude.ai/code/session_01Sx5Tzwp8fTJR1chro59dhF
…N parse

Cross-platform / correctness fixes surfaced by a code audit (verified by
static review; this environment has no Zig toolchain to compile-check):

shell:
- childHasExited only handled Linux/Windows, so macOS and BSD fell through
  to 'return false'. Background jobs were never marked done: finished
  children lingered as zombies and 'jobs' showed them Running forever. Add
  a libc waitpid(WNOHANG) branch that handles ECHILD without panicking.
- tryRunNativePipeline detected an unsupported (redirected/empty) stage only
  mid-run, after earlier stages had already spawned, then fell back to the
  system shell and re-ran the WHOLE line — double-executing non-idempotent
  stages like the 'rm f' in 'rm f | grep x > out'. Pre-validate every stage
  before running any. (Safe: /backticks are rejected upstream.)
- Collapse a dead duplicated runViaSystemShell branch.

agentd:
- providerAgentRun returned provider stdout/stderr verbatim while the README
  promises redacted output. Run redactSecretsAlloc over both (made pub) and
  add a 'redacted' flag, matching the file.read/rg.search/git.diff paths.

desktop:
- jsonStringValue used a single-byte backslash lookback to find a string's
  closing quote, so a value ending in a backslash (e.g. a Windows path
  "C:\\dev\\") was read as an unterminated string. Track escape state.

https://claude.ai/code/session_01Sx5Tzwp8fTJR1chro59dhF
agentd (security):
- Stop leaking the provider API key through curl argv (world-readable via
  ps / /proc/<pid>/cmdline). Write the Authorization header to a private
  0600 temp file and pass it via 'curl -H @file', then delete it. Applied
  to both the request path and the health probe.
- Return a structured read_error envelope instead of tearing down the
  JSON-lines session when an input line exceeds the read buffer.

platform support:
- Trim FreeBSD/NetBSD/OpenBSD from the support gates (isPosixPtyTarget,
  currentBackend, posix_pty.supported, the posix_app import guards) and
  their tests. They never linked (no libc was linked for them) and weren't
  tested. Inert BSD switch arms remain but are no longer advertised. The
  macOS job-reaping branch added earlier is now macOS-only.

docs:
- CHANGELOG: document this polish pass (Fixed / Security / Changed).
- docs/index.html: sync LOC figure to ~18.8k.

Note: this environment has no Zig toolchain, so these changes are verified
by static review against the surrounding code's idioms; CI on push is the
first compile/test gate.

https://claude.ai/code/session_01Sx5Tzwp8fTJR1chro59dhF
…lidation

agentd:
- appendJsonString now validates UTF-8 and substitutes U+FFFD for malformed
  bytes. Previously bytes >= 0x80 were emitted verbatim, so a tool reading a
  binary or non-UTF-8 file produced a JSON envelope strict parsers reject.
  Well-formed UTF-8 still passes through unchanged. Adds a regression test.

shell:
- Bound the existing-file reads in writeHistoryFile and the >> redirect append
  path to max_history_file_bytes, matching every other read path (they used
  .unlimited and could exhaust memory on a huge pre-existing file).

desktop:
- Wire the previously-dead migrate()/validate() into loadDesktopConfig so an
  out-of-range setting is rejected (or upgraded) instead of loading silently.

Static review only (no Zig toolchain in this environment); CI on push is the
first compile/test gate.

https://claude.ai/code/session_01Sx5Tzwp8fTJR1chro59dhF
- docs/index.html: correct stale metrics (18,395->18,850 lines, 198->213
  tests; drop the unverifiable pass/skip split).
- SECURITY.md: document that the provider API key is never on the curl
  command line (0600 temp file + -H @file) and that provider responses are
  redacted.
- docs/reference/agentd-protocol.md: document the provider response envelope
  (redaction + 'redacted' flag), the read_error-on-oversized-line behavior,
  the UTF-8/U+FFFD JSON-string guarantee, and snake_case error codes.
- apps/agentd/README.md: fix CamelCase error code in example (UnknownTool
  -> unknown_tool) to match the actual snake_case output.
- docs/reference/data-map.md: config migrate()/validate() are now enforced
  on load.
- docs/vision/next-20-features.md: drop BSD from the cross-platform-jobs
  target list (platform support is Windows/Linux/macOS).

https://claude.ai/code/session_01Sx5Tzwp8fTJR1chro59dhF
Two regressions in this PR broke CI on all platforms:

1. provider.zig: Io.Dir.CreateFileOptions has no `mode` field in Zig
   0.16.0, so createFile(.., .{ .exclusive = true, .mode = 0o600 })
   failed to compile. Create the auth-header file exclusively, then set
   0600 explicitly via file.setPermissions(io, Permissions.fromMode(0o600))
   on POSIX (guarded out on Windows, where the field is meaningless and
   ACL defaults already restrict access). Preserves the intended
   owner-only permission on the bearer-token temp file.

2. pty.zig: the "posix backend stubs return NotImplemented" test asserted
   spawnNativePosix returns NotImplemented, but the PR made the POSIX
   path a real fork/execve implementation, so on Linux/macOS it now hits
   the empty-argv guard (error.EmptyArgv) instead. Guard that one
   assertion to Windows (where the function genuinely returns
   NotImplemented); keep asserting the two real stubs on all platforms.

Verified locally with the pinned zig 0.16.0 toolchain: zig build,
zig build test (214 tests), and scripts/smoke.sh all pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> | Model: claude-opus-4 | Mode: ultrathink | 🤖 J.A.R.V.I.S.
The shell created its stdout/stderr writers via `std.Io.File.stdout()
.writer(self.io, ...)`, which defaults to POSITIONAL writes (pwrite at a
per-writer `pos` starting at 0). Each command path constructs a fresh
writer (run loop, emitCommandOutput, pipelines, jobs), so when stdout is
a regular file every new writer seeks back to offset 0 and overwrites the
previously written bytes — turning three echoed lines into garbage like
`line t$ ee`. Pipes hid the bug (not seekable -> sequential write), which
is why only the POSIX smoke step (`... > out.txt 2>&1`) failed while the
interactive/TTY path looked fine.

Switch all 12 shell stdout/stderr writer constructions to
`writerStreaming`, which writes sequentially against the shared OS file
offset. Independent writer objects — and inherited-stdout child processes
spawned via /bin/sh — now all append in order. Matches the existing
`readerStreaming` / `writeStreamingAll` usage style.

Verified with Zig 0.16.0: `zig build`, `zig build test` (all green),
and `sh ./scripts/smoke.sh` now prints "ZiggyZag smoke passed".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> | Model: claude-opus-4 | Mode: ultrathink | 🤖 J.A.R.V.I.S.
@PascalAI2024
PascalAI2024 merged commit 43d20fc into main May 31, 2026
3 checks passed
@PascalAI2024
PascalAI2024 deleted the claude/admiring-franklin-0AhEm branch September 22, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants