Skip to content

docs: routing and harness anti-patterns and reword-first evidence (#13) - #27

Merged
lukemaj merged 2 commits into
mainfrom
docs/13-anti-patterns
Sep 29, 2026
Merged

lukemaj merged 2 commits into
mainfrom
docs/13-anti-patterns

Conversation

@lukemaj

@lukemaj lukemaj commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

What: The anti-pattern catalog gains nine routing and harness failure modes measured in the AgentsMD benchmarks, and the mechanism page gains two AgentsMD cases for measuring and rewording before building a hook.
Why: The catalog covered only karpathy-wiki failures, so authors of routing tables and trigger harnesses had no checklist entry for the failures that cost AgentsMD invalid benchmarks and missed procedures.
So what: Reviewed and approved on the exact head; the user merges.

Closes #13
Refs #7

Changes

docs/10-anti-patterns.md: nine entries, each with Definition, Evidence and a Counter linking /docs/05-authoring/triggers, /docs/06-testing/trigger-benchmarks or the case study. The unsourced "OpenCode ... less affected" flag (:53) is replaced by the claim boundary: the reported case is on Claude; no other host has been measured.

docs/07-mechanism-vs-decoration.md: new section "Measure, reword, mechanism last: two AgentsMD cases"; the paths: trade-off now names Claude Code and Grok Build with primary sources.

public/llms.txt: regenerated.

Numbers and sources

Entry Number Source
Trigger that does not name the action verification before gh pr create 0/5, then 5/5 toolboxmd/agentsmd#181, #184
Bare conjunct SKILL-MECHANICS.md 15/15, prose.md 4/15 attempts; both 5/5 with own before-clause 174-routing-injection/records-prefix.jsonl (first batch, arms A/B/C, recounted), #179
Abstract task-start rule 0/40 naive; 20/20 with pointer; fix1 verification 5/5 to 3/5, research on question 2/5 #164 results, #174 results, #184; fix1 3/5 regenerated with --summary on 182-routing-sweep/records.jsonl
Read in the same call diagnosis 8/10 on fix2, 5/5 on fix3 #182 results (regenerated with --summary), #184
Always-loaded text tuned on one host OpenCode research on question 5/5 on fix2 (0/5 clean), 5/5 clean after exclusion case study, #184
Refused read scored as a read 17/20 with every read refused; 10 of 17 never edited records-prefix.jsonl, #179
No-action run scored as pass or miss 10 of 17 never edited; 12 verdicts changed on #182 rescoring, 10 no-action #179, case study, #184
Green harness tests 349 tests, self-checks on four hosts, APPROVE #168, case study, #179
Unconfined harness shell pilot wrote to the canonical checkout; Bash open; write-only sandbox #164 results, #181, #183
Mechanism page 9 of 11, then 5 of 13 Issue-creating sessions skipped the Elon method; hook added toolboxmd/agentsmd#138, #161
Mechanism page verification hook proposed, conditional, not built after 5/5 #181, #182, #184
paths: hosts Claude Code, Grok Build code.claude.com/docs/en/skills; xai-org/grok-build conditional.rs at 97f190f

Deviation from the ticket: entry 2 uses 4/15 against 15/15 (as published in triggers.md, recounted from the records) instead of the ticket's 7/22 against 22/22. #161 extended the existing PreToolUse hook rather than registering a new one; the page says so. No Haiku numbers.

Elon record

Proof

  • npm ci && npm test under Node 22.23.1: all checks pass (links, llms freshness and coverage, absolute paths, bare-path gate).
  • npm run build:llms regenerated public/llms.txt.
  • Independent review: Luna (gpt-6-luna) max, REQUEST_CHANGES on da85d36 (bare-conjunct figures blurred attempts and reads; fixed), then APPROVE on 79d0591.

🤖 Generated with Claude Code

lukemaj and others added 2 commits September 29, 2026 23:46
…ence (#13)

The catalog gains nine measured failure modes from the AgentsMD routing
benchmarks; the mechanism page gains two AgentsMD cases for measuring and
rewording before building a hook, and names the hosts that honor paths:.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add the routing and harness failure modes to the anti-pattern catalog and the mechanism page

1 participant