Add: skill-activation-as-precision-recall

[uipreliga]

PR intent

• Adds a reusable, generalized pattern (not a product announcement).
• One pattern per PR.

patterns/skill-activation-as-precision-recall.md — Reliability & Eval, 155 lines.

A skill is normally tested like a function: run it, check the output. But before it runs, the model must choose it from among every other skill's description — and that routing decision is what fails in production, in two directions, both silently. Under-triggering is a recall problem that users report; over-triggering is a precision problem that gets absorbed, because the output still looks like an answer. The decision is also coupled across the whole catalog: editing one description re-routes prompts belonging to skills you did not touch.

The pattern is to measure it as the classification problem it is — a labelled dataset including a negative region no skill should own, one agent trace per row scored against every skill, per-skill precision/recall/F1, and thresholds that gate CI.

Reviewer checklist

• Problem, Solution, and Trade-offs are clearly documented.
• No marketing language or calls to action in the PR description or diff content.
• The pattern is not tied to a single brand as its primary value proposition.
• Public references are included; if self-referencing, additional context from third-party or non-affiliated sources is provided when available.
• Conflict-of-interest disclosures are included where applicable.

Conflict of interest

I maintain Coder Eval, which is one implementation of this pattern, and three of the references are its blog. So, per the template's note on self-referencing PRs:

• source: and based_on: both disclose the origin in the front matter.
• The body is tool-neutral. The Solution section is pseudocode and a JSONL shape, not an API; nothing in Problem, Solution, How to use it, or Trade-offs names a product. The only mention is one References bullet, worded as "one open-source implementation," which is how the existing workflow-evals-with-mocked-tools pattern cites LangSmith and Promptfoo.
• Non-affiliated corroboration is included: mongodb/agent-skills independently ships pairwise skill-selection cases in testing/skills-boundaries/ (3 pairs at the time of writing) alongside a qa-eval workflow. The measured pilot is also hosted third-party, as a public issue on neondatabase/agent-skills#76, including the rows we excluded and why.
• Happy to cut the Coder Eval reference entirely if you would rather the pattern carried none.

AI disclosure (per AI_POLICY.md)

• Tool/model: Claude Opus 5 via Claude Code. The commit carries an Assisted-by: trailer.
• AI-assisted parts: drafting the prose, the Mermaid diagram, and schema/format compliance against SCHEMA.md, TEMPLATE.md and the existing patterns.
• Human-owned: the pattern itself and the underlying measurements are mine; I directed and reviewed the draft and can defend any part of it in review.
• How it was verified: every cited URL was opened and returns 200; the two Related: links resolve to existing files; the MongoDB and Neon claims were checked against the live repositories rather than recalled. Two claims from an earlier draft were removed because they could not be verified publicly — a "required check" characterisation of MongoDB's CI (workflow existence is public, branch protection is not), and an unsourced generalisation about "shipped public catalogs," now replaced by the specific, published collision in the Neon issue. Numbers in Evidence are quoted from the linked pages, and the Evidence section states its own grade as medium with an explicit "Unverified / Unclear" list.

Generated artifacts

bun run build:data was run and its output is committed in the same commit, as the CI error message asks. That adds the README catalog line (inside the AUTO-GENERATED markers — not hand-edited) and the artifacts under apps/web/public. apps/web/src/data/github-stars.json is deliberately not included, matching the git restore in the CI check's own logic.

Verified locally: bun run validate:patterns clean, the CI drift check reproduces green, and bun run build succeeds (203 pages).

One note in case it shows up in triage: bun run validate:patterns:content reports 15 warnings and 1 error (cross-agent-lesson-sharing.md: missing references) on pre-existing patterns. None are from this PR, and I have not touched them. My file is not flagged — including for heading order, since ## Evidence sits between Solution and How-to-use exactly as TEMPLATE.md places it.

[vercel]

@uipreliga is attempting to deploy a commit to the nkkko's projects Team on Vercel.

A member of the Team first needs to authorize it.

[uipreliga]

The review-pattern-proposal check is red here, but it looks unrelated to this PR — it has failed on all 12 runs since 2026-08-21, on your own branches too. Since I had the log open, the cause may be useful:

The run ends as error_max_turns, and the action then fails with --json-schema was provided but Claude did not return structured_output. So the surfaced message points at the schema, but the schema is fine — the reviewer just hits --max-turns 6 before it emits anything.

That start date lines up with #140, which added --json-schema. Before it, a run that ran out of turns could still publish whatever text it had; after it, exhausting the budget became a hard failure with no review posted. So the effect is the opposite of what the commit intended.

Two cheap things, if useful:

• Raise --max-turns. Six has to cover gh pr diff, gh pr view, reading the pattern, any Grep/Glob, and composing a review of up to 60k chars.
• Scope the diff to gh pr diff -- patterns/. Since the drift gate landed on 08-25, every pattern PR also carries ~11 generated files — this one is 12 files and +514 lines, of which llms-full.txt alone is +128. The reviewer is spending its budget on generated output.

Happy to open a separate PR for either if you'd rather not spend the time. The run I looked at is here.