Skip to content

docs: activation and budget pages match measured behavior (#12) - #28

Merged
lukemaj merged 2 commits into
mainfrom
docs/12-activation-budget
Sep 29, 2026
Merged

lukemaj merged 2 commits into
mainfrom
docs/12-activation-budget

Conversation

@lukemaj

@lukemaj lukemaj commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

What: The quickstart, mental model, three-questions and token-economics pages now state the measured activation behavior and each host's listing and hook-output limits.
Why: These first-read pages said the agent matches descriptions on every turn and that session-start injection is only for bootstraps; AgentsMD measured the opposite on Claude Code (0/40 without a pointer, 3/20 to 20/20 with one).
So what: Needs Luna max review of the exact head, then the user merges.

Closes #12
Refs #7

Changes

  • 03-three-questions.md: Question 1 separates topic-shaped requests from moment-shaped needs, links the routing case study and /docs/05-authoring/triggers; Question 3 budgets are per-host; the karpathy-wiki "zero when the skill does not fire" becomes "the description in the listing".
  • 04-token-economics.md: section 2 is a per-host listing table; section 4 is "full body or pointer" with which hosts needed the pointer and the hook-output limits; the calculator gains a pointer mode; compaction numbers re-verified with a date; the Codex and Gemini CLI statement (old :131) now matches Correct the harness facts for Claude Code, Codex, Grok Build, OpenCode and Gemini CLI #8.
  • 02-mental-model.md: model: opus alias; the zero-per-session-cost claim corrected; a "must be read before a specific action" shortcut.
  • 01-quickstart.md: hello-world path unchanged; "What just happened" and "Why the description matters" call the trigger phrase the easy case and link Triggers; the workflow-summary warning states its evidence (one reported superpowers case).
  • public/llms.txt: regenerated with npm run build:llms; no change (no first paragraph changed).

Number sources

Number Source
0/40 naive entry loads, Opus 5.5 (Claude Code) trigger_test.py --summary 164-trigger-audit/records.jsonl at 06b7af6: 0/20 main, 0/20 branch
3/20 to 20/20 target reads; 4/20 to 20/20 entry loads; mean tokens 150,264 and 182,323 --summary 174-routing-injection/records.jsonl at 06b7af6, base and B2
OpenCode 16/20 to 20/20; Codex and Grok 20/20 without pointer case study #164 and #174 tables, --summary above
Pointer 417 chars (about 105 tokens), widened to 482 case study, Routing fixes (PRs toolboxmd/agentsmd#179, #184)
17,800 tokens, 57 hours, 13 firings, 1,370 per firing obra/superpowers#1220 body, read 2026-09-29 (26 h + 31 h sessions, 9 + 4 firings)
25,000 / 5,000 compaction code.claude.com/docs/en/skills, read 2026-09-29
Claude Code 1,536 chars, 1% listing code.claude.com/docs/en/skills, read 2026-09-29; #8 page
Claude Code 10,000-char hook cap, 2,000-char preview code.claude.com/docs/en/hooks, read 2026-09-29; #8 page
Codex 2% or 8,000 chars, 1,024 per description; 2,500-token hook output openai/codex 687a119 render.rs, output_spill.rs, fetched 2026-09-29; #8 page
Grok Build 400 bytes per entry, 50% listing xai-org/grok-build 97f190f listing.rs, fetched 2026-09-29; #8 page
OpenCode lists every Skill in the skill tool #8 others.md (OpenCode v1.18.33)

Elon record

Proof

🤖 Generated with Claude Code

@lukemaj

lukemaj commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

REQUEST_CHANGES

  1. docs/03-three-questions.md:21 says that when a request names the topic, "the agent sees the words the description lists and loads the skill." This presents description matching as a reliable outcome, but the cited benchmarks measure moment-shaped prompts and do not test this topic-shaped claim; Correct the harness facts for Claude Code, Codex, Grok Build, OpenCode and Gemini CLI #8 also describes implicit invocation as something the model may do. Please qualify this as a possible match/load rather than a guaranteed load, and keep the #164/#174 measured boundary explicit.

  2. docs/03-three-questions.md:65-66 says every host lists each Skill description and every description is paid every session. Correct the harness facts for Claude Code, Codex, Grok Build, OpenCode and Gemini CLI #8 documents exceptions: Claude Code omits descriptions for disable-model-invocation: true Skills, Grok Build holds paths: Skills out until a matching file is touched, and OpenCode omits Skills denied by permission.skill; budgets can also shorten or drop entries. Please scope these claims to descriptions actually present in the host listing (and account for conditional listings). The same universal wording appears in docs/04-token-economics.md:2.

lukemaj and others added 2 commits September 29, 2026 23:54
Three questions separates topic-shaped from moment-shaped needs; token
economics replaces "bootstrap only" injection with the full-body versus
pointer choice and gives per-host listing and hook-output limits; mental
model and quickstart drop the zero-cost and every-turn claims.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…#12)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@lukemaj

lukemaj commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

APPROVE

Re-reviewed the full PR at a5e58ef6c01242914a6795727204aef9e0245365, including the fixes to topic-shaped matching and per-host listing exceptions. Both prior findings are resolved. The acceptance criteria, #8 host facts, glossary terms, trigger guidance, evidence boundaries, and allowed file scope are consistent; the #164/#174 summaries regenerate to the cited results. No actionable findings.

@lukemaj
lukemaj merged commit 53f1864 into main Sep 29, 2026
3 checks passed
@lukemaj
lukemaj deleted the docs/12-activation-budget branch September 29, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

State the measured activation behavior in the quickstart, mental model, three questions and token economics

1 participant