Run the hook eval on Linux #4

Activity

1. sabind

   added

   priority: lowWhen convenientWhen convenient

   effort: low

   area: quality

   on Oct 11, 2026
2. sabind

   added this to the Listing on Claude marketplace milestone on Oct 11, 2026
3. sabind

   added

   qualityThe behaviour of the skills, agents and rules, and the evals and checks that guard itThe behaviour of the skills, agents and rules, and the evals and checks that guard it

   and removed

   effort: low

   area: quality

   on Oct 12, 2026
4. sabind

   added the

   Task

   issue type on Oct 12, 2026
5. sabind commented on Oct 13, 2026

   

   sabind

   on Oct 13, 2026

   ContributorAuthor

   More actions

   Run on 2026-10-13 from a Mac (Docker 29.5.2, linux/arm64) in a scratch container; nothing was added to the repository for it.

   Image: node:22-bookworm plus apt-get install make git python3 and npm install -g @anthropic-ai/claude-code, with the 3-eval-checks-and-cadence checkout copied to /work (no .git, re-initialised with one snapshot commit), run as the node user.
   Claude Code: 2.1.296.
   Command: built with docker build -t handbook-hook-eval:scratch . from a context holding the Dockerfile above and the checkout, then

   docker run --rm -e CLAUDE_CODE_OAUTH_TOKEN handbook-hook-eval:scratch

   whose entrypoint is claude --version && make eval-hook EVAL_ARGS="--runs 1". The token came from claude setup-token, exported in the shell and passed only as that environment variable.

   Result:

   Plugin under test: "freethinking" version "0.6.0" at "/work"
     rules-loaded run 1/1: score 1.00  $0.04
       ✓ names-code-changes (weight 1): matched ^#?\s*Code changes\s*$
       ✓ names-workflow (weight 1): matched ^#?\s*Workflow\s*$
       ✓ not-no-rules (weight 1): pattern absent as expected
   ✓ rules-loaded  score 1.00  (1 run)  $0.04
     subagent-rules-loaded run 1/1: score 1.00  $0.07
       ✓ names-code-changes (weight 1): matched ^#?\s*Code changes\s*$
       ✓ names-workflow (weight 1): matched ^#?\s*Workflow\s*$
       ✓ not-no-rules (weight 1): pattern absent as expected
   ✓ subagent-rules-loaded  score 1.00  (1 run)  $0.07
   
   CASE                   SCORE PASS% RUNS COST    NOTES
   rules-loaded           1.00  100%  1    $0.04
   subagent-rules-loaded  1.00  100%  1    $0.07
   

   The SubagentStart hook fires inside a sub-agent under Linux and both rule files reach it. Closed by the 3-eval-checks-and-cadence PR.
6. sabind commented on Oct 14, 2026

   

   sabind

   on Oct 14, 2026

   ContributorAuthor

   More actions

   Closed by PR #22 (merged 2026-10-14); the Linux run is recorded in the comment above. The PR body said "Closes #3, #13 and #4" but GitHub only auto-closes the first reference after a keyword, so closing by hand.
7. sabind

   closed this as completedon Oct 14, 2026