Rewrite two skill descriptions so owid-catalog stops losing to search-charts

AGENTS.md

@@ -5,6 +5,7 @@ This repository publishes **agent skills for working with Our World in Data** (s
 ## Structure
 
 ```
+FAQ.md                          # common user and contributor questions
 Makefile                        # entry points: make validate / test / triggers
 skills/<skill-name>/SKILL.md    # one directory per skill, and nothing else
 .claude-plugin/marketplace.json # marketplace + plugin definition

FAQ.md

@@ -0,0 +1,169 @@
+# FAQ
+
+Common questions about using and contributing to these skills. For repo
+conventions see [AGENTS.md](AGENTS.md); for how the skills are evaluated see
+[evals/README.md](evals/README.md).
+
+## Using the skills
+
+### My agent isn't using the skills at all
+
+Work through these in order — the first two are the most common.
+
+**1. Are the files where your agent looks?** Every agent reads a different
+directory, and installing to the wrong one fails silently. The
+[`skills` CLI](https://github.com/vercel-labs/skills) knows the paths for 76
+agents and picks the right one:
+
+```bash
+npx skills add owid/skills            # into the current project
+npx skills add owid/skills --global   # user-level, all projects
+npx skills list                       # what is installed where
+```
+
+Nineteen of those agents — including Codex, Cursor and Gemini CLI — share the
+project-level `.agents/skills/` convention. Claude Code is the notable exception,
+using `.claude/skills/` per project and `~/.claude/skills/` globally. If you
+installed by hand, check with `ls .agents/skills/ .claude/skills/ 2>/dev/null`.
+
+**2. Is your agent's reasoning effort turned down?** Skills are invoked through a
+tool call, and lowering reasoning effort makes agents make fewer tool calls — so
+a setting you turned down for cost can stop skills firing at all.
+
+We measured this on Claude Code while building the trigger evals: at
+`--effort low`, five queries that reliably invoke a skill at normal effort were
+answered directly instead, with no skill consulted. Nothing about the skills
+changed; only the effort did.
+
+The same class of setting exists elsewhere — Codex has `model_reasoning_effort`
+(`minimal`/`low`/`medium`/`high`/`xhigh`) in `~/.codex/config.toml`, and most
+agents expose something similar. **We have only measured the effect on Claude
+Code**, so treat the others as a plausible first thing to check rather than a
+known cause. If your agent ignores skills, raise the effort and try again before
+assuming the skill is at fault.
+
+**3. Is the task substantial enough?** Agents skip skills for work they can do
+unaided. "What's the URL for OWID's CO2 chart?" may not trigger anything, while
+"find the best OWID chart on CO2 per capita and pull the data for the G7" will.
+This is by design — it isn't a bug you need to report.
+
+**4. Ask for it by name.** `use the search-charts skill to find…` bypasses
+routing entirely, and is the quickest way to tell "the skill is missing" apart
+from "the skill wasn't selected".
+
+### The wrong skill triggered
+
+These four skills cover deliberately adjacent ground, so this happens. The rough
+division of labour:
+
+| You have… | You want… | Skill |
+|---|---|---|
+| a topic | to find a chart | `search-charts` |
+| a chart URL | its data or metadata | `fetch-chart-data` |
+| your own dataset | it joined to OWID data | `joining-data` |
+| Python, or a need for units/metadata/indicators | a DataFrame | `owid-catalog` |
+
+Naming the skill explicitly always wins. If a realistic request routes to the
+wrong skill repeatedly, that is a bug in our `description` fields and worth
+[opening an issue](https://github.com/owid/skills/issues) — please include the
+prompt you used, since that becomes a trigger-eval case.
+
+### Does installing this put files in my repo?
+
+Yes, if you install per-project: the skill directories are copied into your
+agent's skills directory inside the project. Each skill is a single `SKILL.md`
+and nothing else — about 32 KB in total — and this is deliberate. Our test
+fixtures and eval scripts live in a top-level `evals/` directory precisely so
+they are never copied into your repository, where a fixture CSV could be mistaken
+for your own data. See
+[evals/README.md](evals/README.md#why-evals-live-here-and-not-inside-the-skill-directory).
+
+To keep them out of version control, add your agent's skills directory to
+`.gitignore`, or install with `--global` instead.
+
+### Which tools do I need installed?
+
+Per skill, so you only need what you use:
+
+| Skill | Needs |
+|---|---|
+| `search-charts` | `curl`, `jq` |
+| `fetch-chart-data` | `curl`, `jq` |
+| `joining-data` | `duckdb` |
+| `owid-catalog` | `uv` (or `pip`) |
+
+On macOS, `./install-prerequisites-macos.sh` installs all four. Skills only use
+public OWID endpoints — there are no credentials to configure.
+
+### A skill gave me data I think is wrong
+
+Check whether the skill or the data is at fault. The skills are documentation
+over OWID's public API; they don't transform values. Fetch the same numbers
+directly:
+
+```bash
+curl -s "https://ourworldindata.org/grapher/life-expectancy.csv?csvType=filtered&country=USA&time=2020"
+```
+
+If that matches what the agent told you, the skill worked and any concern belongs
+with the underlying data — see the chart's own page on
+[ourworldindata.org](https://ourworldindata.org). If it doesn't match, that's our
+bug. Two known traps worth ruling out first:
+
+- **`csvType=filtered` applies the chart's own default entity selection**, not
+  "all countries". `population.csv?csvType=filtered&time=2020` returns seven rows
+  — continents and World — with no individual countries. Pass an explicit
+  `country=` filter.
+- **A no-match search still returns results.** OWID's search falls back to
+  low-relevance hits rather than returning nothing, so `nbHits` is never a
+  reliable signal that a topic is missing. Judge the titles.
+
+## Contributing
+
+### `make test` fails and I didn't change anything
+
+That is the contract tests doing their job. They check the OWID endpoints and
+response shapes each `SKILL.md` documents against what the API actually returns,
+so they can break when OWID ships a change and nobody has touched this repo.
+They also run nightly for exactly that reason.
+
+Read the failure before assuming it's a flake — it names the endpoint and the
+mismatch, and the downloaded responses are kept under `evals/results/contract/`
+so you can inspect one without re-running. Network outages also surface here,
+which is intentional.
+
+### `make triggers` costs a lot. How do I make it cheaper?
+
+It runs `queries x RUNS x skills` full agent sessions — 120 for the default
+invocation. While iterating on a description, narrow it:
+
+```bash
+make triggers SKILL=owid-catalog RUNS=1
+```
+
+Don't reach for a lower effort level to save money: as above, effort changes
+whether skills fire at all, so a cheap run measures something other than what
+your users experience. Same caution for `MODEL=` — routing is model-dependent, so
+a cheaper model measures that model's routing. Both are fine for fast iteration
+on wording, then confirm on the real model and effort before believing a number.
+
+### Why do the skills never mention their own evals?
+
+Because the `description` and `SKILL.md` body are loaded into the user's context
+when a skill triggers, and eval prose would be pure overhead there. It's enforced
+by `make validate`, not left to discipline.
+
+### `make triggers` exits non-zero. Is that a failure?
+
+Only if runs errored. Trigger accuracy is a measurement, not a pass/fail gate —
+100% routing accuracy isn't a realistic bar. The runner exits non-zero when runs
+actually failed (meaning the numbers can't be trusted) or when you set a floor
+with `--min-accuracy`. `make test` is the gate.
+
+### How do I add a skill?
+
+See [AGENTS.md](AGENTS.md). The short version: create
+`skills/<name>/SKILL.md` with `name` matching the directory, register it in
+`.claude-plugin/marketplace.json`, then run `make validate`. Keep the directory
+to just `SKILL.md` unless you genuinely need bundled `scripts/`, `references/` or
+`assets/` — everything in there ships to every user.

Makefile

@@ -40,6 +40,13 @@ validate: ## Check spec conformance, the plugin manifest and marketplace registr
 	  echo "  x a SKILL.md references eval files - drop the reference or inline the content"; \
 	  exit 1; \
 	else echo "  ok  no SKILL.md references eval files"; fi
+	@# Eval JSON is hand-authored and hand-reviewed, so it must stay readable. A
+	@# python json.dumps without ensure_ascii=False silently rewrites every em dash
+	@# and accent as a \uXXXX escape, which is unreviewable prose.
+	@if grep -rln '\\u[0-9a-fA-F]\{4\}' evals/skills/*/*.json 2>/dev/null; then \
+	  echo "  x the file(s) above contain escaped unicode - rewrite with ensure_ascii=False"; \
+	  exit 1; \
+	else echo "  ok  eval json has no escaped unicode"; fi
 	@# Registration: neither validator above knows about marketplace.json, and an
 	@# unregistered skill is installable by neither route.
 	@fail=0; \

README.md

@@ -65,6 +65,13 @@ Then put `owid-skills/skills/*` where your agent looks for skills:
 - **Per project** — `./.agents/skills/` (the shared convention read by Codex, Cursor, OpenCode, …)
 - **Per user** — your agent's own skills directory, e.g. `~/.codex/skills/`, `~/.gemini/skills/`, or `~/.claude/skills/`
 
+### Not working?
+
+If your agent doesn't seem to be using the skills, see the
+[FAQ](FAQ.md#my-agent-isnt-using-the-skills-at-all). The two usual causes are the
+files being in a directory your agent doesn't read, and your agent's reasoning
+effort being turned down far enough that it stops making tool calls.
+
 ### Prerequisites
 
 The skills use a few common command-line tools: `curl`, `jq`, `duckdb`, and `uv`. Install them with your package manager (e.g. `brew install jq duckdb uv`), or on macOS run:
@@ -79,7 +86,7 @@ Data published by Our World in Data is open: it is available under the [Creative
 
 ## Development
 
-Want to add or improve a skill? See [AGENTS.md](AGENTS.md) for repo conventions and [evals/README.md](evals/README.md) for how the skills are evaluated.
+Want to add or improve a skill? See [AGENTS.md](AGENTS.md) for repo conventions, [evals/README.md](evals/README.md) for how the skills are evaluated, and the [FAQ](FAQ.md) for questions that come up often.
 
 ```bash
 make            # list targets

evals/run-trigger-eval.py

@@ -73,12 +73,9 @@ def build_command(query: str, model: str | None, effort: str, max_budget: float
         "--verbose",
         "--allowed-tools",
         ALLOWED_TOOLS,
-        # Routing is a shallow decision the model makes before doing any work, so
-        # thinking tokens are pure waste here. Raise this for a fidelity run that
-        # should match a real session's effort.
-        "--effort",
-        effort,
     ]
+    if effort:
+        cmd += ["--effort", effort]
     if model:
         cmd += ["--model", model]
     if max_budget is not None:
@@ -106,7 +103,10 @@ def skills_in_line(line: str, candidates: list[str]) -> set[str]:
         obj = None
 
     if isinstance(obj, dict):
-        blocks = (obj.get("message") or {}).get("content")
+        message = obj.get("message")
+        # Some stream events carry `message` as a plain string, so this cannot
+        # assume a dict — `(x or {}).get(...)` blows up on a non-empty string.
+        blocks = message.get("content") if isinstance(message, dict) else None
         if isinstance(blocks, list):
             for block in blocks:
                 if not isinstance(block, dict) or block.get("type") != "tool_use":
@@ -234,12 +234,20 @@ def main() -> int:
     target.add_argument("--all", action="store_true", help="every skill with a triggers.json")
     parser.add_argument("--runs", type=int, default=3, help="runs per query (default: 3)")
     parser.add_argument("--workers", type=int, default=4, help="parallel runs (default: 4)")
-    parser.add_argument("--timeout", type=int, default=90, help="seconds per run (default: 90)")
+    parser.add_argument(
+        "--timeout", type=int, default=180,
+        help="seconds per run (default: 180). A run that fires a skill is killed immediately, "
+             "but a true negative has no decision to observe, so it only ends when the model "
+             "finishes answering — which at higher effort can take a while.",
+    )
     parser.add_argument("--threshold", type=float, default=0.5, help="fire rate counted as a trigger (default: 0.5)")
     parser.add_argument("--model", default=None, help="model for claude -p (default: your configured model)")
     parser.add_argument(
-        "--effort", default="low", choices=["low", "medium", "high", "xhigh", "max"],
-        help="reasoning effort per run (default: low — routing needs no deep thinking)",
+        "--effort", default=None, choices=["low", "medium", "high", "xhigh", "max"],
+        help="reasoning effort per run (default: inherit your session's effort). Measured, "
+             "not assumed: at low effort the model answers more queries directly instead of "
+             "reaching for a skill, which shows up as misses and understates real triggering. "
+             "Use it to cut cost only when comparing two descriptions at the same effort.",
     )
     parser.add_argument(
         "--max-budget-usd", type=float, default=None,

evals/skills/owid-catalog/triggers.json

@@ -12,7 +12,7 @@
   {
     "query": "does owid have population broken down by sex and age group? the grapher charts only seem to give me totals and i need the extra dimensions for a cohort model",
     "should_trigger": true,
-    "note": "the garden vs grapher channel distinction is the answer"
+    "note": "the garden vs grapher channel distinction is the answer. Deliberately hard: it opens with 'does owid have', which reads as chart discovery, and the owid-catalog tells (extra dimensions, cohort model) only arrive later. It misrouted to search-charts 3/3 before the descriptions were rewritten."
   },
   {
     "query": "write a uv script that pulls owid renewable energy share, resamples to 5-year means per country, and writes the result to renewables_5yr.parquet",

skills/owid-catalog/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: "owid-catalog"
-description: "Access Our World In Data's published datasets using the owid-catalog Python library. Provides a unified Python API for searching and fetching chart data, catalog tables, and indicators — returning enhanced pandas DataFrames with metadata. Use this as a Python-native alternative to the HTTP-based search-charts and fetch-chart-data skills."
+description: "Access Our World In Data from Python with the owid-catalog library: load chart data, catalog tables or individual indicators as pandas DataFrames that carry their own units, descriptions, sources and citations. Use this skill whenever the work happens in Python or a notebook (pandas, a uv script, matplotlib, parquet); whenever you need an indicator's metadata, units or codebook; whenever you need dimensions that published charts flatten away, such as sex, age group or projection variant; or whenever you need to search OWID's full catalog of indicators and tables, including semantic search by meaning, rather than only its published charts. Prefer it over the HTTP-based search-charts and fetch-chart-data skills for any Python-based analysis."
 allowed-tools:
 - "Bash(uv:*)"
 - "Bash(pip:*)"

skills/search-charts/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: "search-charts"
-description: "Our World In Data offers thousands of charts and related data on many important topics - from global population data, energy and electricity, economic data like GDP or poverty, health data like causes of death or prevalence of diseases, to data on democracy, violence and war. This skill describes how to effectively search for charts to either show visually or download the data for."
+description: "Search Our World In Data's published charts by keyword to find the chart you need, across topics like population, energy and electricity, CO2 and climate, poverty and GDP, health and causes of death, education, democracy, violence and war. Use this whenever someone wants to find, browse, link or embed an OWID chart and does not already have its URL, or asks what OWID publishes on a topic. Returns each chart's title, subtitle and URL, plus which visualisations it supports so you can build a ?tab= link. Not for: fetching the data behind a URL you already have (use fetch-chart-data); Python or pandas work, indicator and column metadata, or searching the full catalog of indicators and tables beyond published charts (use owid-catalog); combining OWID data with your own (use joining-data)."
 allowed-tools:
 - "Bash(curl:*)"
 - "Bash(cat:*)"