docs: render every published figure from its source

CHANGELOG.md

@@ -12,6 +12,21 @@ Releases before `0.2.5` predate the public launch; their notes live in the
 
 ## [Unreleased]
 
+### Changed
+
+- **Published figures are rendered from their sources, never typed.**
+  `scripts/render_figures.py` rewrites every `<!-- fig:NAME -->` marker in the
+  README and docs (test counts, action counts, story-set sizes, recorded pass
+  rates), and `scripts/check_evidence_claims.py` now refuses a figure outside a
+  marker. Figures that routine PRs move are rounded floors ("2,000+"); recorded
+  story pass rates stay exact. `UPDATE_TEST_BASELINE=1 scripts/test_baseline.sh`
+  records the evidence and re-renders the docs in one step, and
+  `scripts/check_public_claims.sh` runs `render_figures.py --check`. The README
+  shows the exact Rust and frontend test counts as live badges read from
+  `tests/evidence/workspace-tests.json`. The README line that said the Ubuntu
+  suite "has since grown to 96" was already stale (it holds 99) and is now a
+  derived figure.
+
 ### Security
 
 - **`request_hash` is now keyed: an audit export is no longer an offline

CONTRIBUTING.md

@@ -65,10 +65,10 @@ table below.
 
 | Area | Why it matters | Difficulty |
 |---|---|---|
-| **Ubuntu LTS support** | The current suite is 99 Ubuntu stories. Committed live-VM evidence covers 79 of those Ubuntu stories on each LTS release, with a committed replay twin that reproduces each run: 22.04, 24.04 and 26.04 all at 79/79. The twenty additional stories are not yet included in a committed live-VM run. `ubuntu-vm.sh` accepts `UBUNTU_RELEASE=jammy\|noble\|resolute`. Remaining: story coverage for the cross-family actions, and every Debian-only action has a story. | medium |
+| **Ubuntu LTS support** | The current suite is <!-- fig:stories_ubuntu floor=10 -->90+<!-- /fig --> Ubuntu stories. Committed live-VM evidence covers <!-- fig:lts_story_total -->79<!-- /fig --> of those Ubuntu stories on each LTS release, with a committed replay twin that reproduces each run: 22.04, 24.04 and 26.04 all at <!-- fig:lts_story_rate -->79/79<!-- /fig -->. The <!-- fig:stories_ubuntu_unrecorded floor=10 -->20+<!-- /fig --> additional stories are not yet included in a committed live-VM run. `ubuntu-vm.sh` accepts `UBUNTU_RELEASE=jammy\|noble\|resolute`. Remaining: story coverage for the cross-family actions, and every Debian-only action has a story. | medium |
 | **Distro detection coverage** | Robust `/etc/os-release` parsing for every release we claim to support. Pure-function tests against real fixture files, no integration mocks. The existing fixtures at the bottom of `crates/sysknife-core/src/distro.rs` show the shape. | easy |
 | **Action catalogue gaps** | Add a typed action (for example `EnableFirewallZone`). Small and isolated, and every PR carries the policy entry, the risk level and the tests. | easy |
-| **E2E story coverage** | Real prompts, real LLM, real daemon. The suite is 153 stories: 54 atomic + 99 Ubuntu. What is left is the cross-family middle: of the action names available on both families, 46 are still untouched by any story, plus 10 Fedora-only and 0 Ubuntu-only ones. See #233 for the clustered map. | medium |
+| **E2E story coverage** | Real prompts, real LLM, real daemon. The suite is <!-- fig:stories_total floor=10 -->150+<!-- /fig --> stories: <!-- fig:stories_atomic floor=10 -->50+<!-- /fig --> atomic + <!-- fig:stories_ubuntu floor=10 -->90+<!-- /fig --> Ubuntu. What is left is the cross-family middle: of the action names available on both families, <!-- fig:uncovered_all -->46<!-- /fig --> are still untouched by any story, plus <!-- fig:uncovered_fedora -->10<!-- /fig --> Fedora-only and <!-- fig:uncovered_ubuntu -->0<!-- /fig --> Ubuntu-only ones. See #233 for the clustered map. | medium |
 | **Fedora Atomic validation** | The action families exist and `DistroId::is_supported()` returns true for Atomic 41 and up. Nobody has run `tests/e2e/atomic-vm.sh` against a current release. Needs Fedora Atomic hardware or a VM host. | tedious |
 | **Demo recording on real hardware** | Replace the bundled demo GIF with a 30-second recording on real Ubuntu 26.04 with rollback visible. | easy |
 
@@ -168,16 +168,21 @@ manifests. A per-crate run like
 `cargo test -p sysknife-types` never touches those, so a green crate is not a
 green board.
 
-**Adding or removing a Rust test moves a published figure.** The suite size lives
-in `tests/evidence/workspace-tests.json`, and `README.md`,
-`docs/introduction.md` and `docs/distro-support.md` state it in prose. Regenerate
-the artifact and update those three files in the same commit:
+**Never type a figure.** Test counts, the action count, story-set sizes and
+story pass rates are rendered by `scripts/render_figures.py` from the artifact or
+source that produces them, and a number the claim checks match outside a marker
+fails the build. Add your tests, run the command below, and commit the evidence
+file together with whatever it re-rendered:
 
 ```sh
 UPDATE_TEST_BASELINE=1 scripts/test_baseline.sh
-grep -rn 'Rust tests' README.md docs/introduction.md docs/distro-support.md
 ```
 
+Figures that routine pull requests move are rounded floors ("2,000+"), so most
+PRs re-render nothing. The exact Rust and frontend counts are the README badges,
+read from the evidence file. The marker syntax is explained once, in
+[docs/contributing/testing.md](docs/contributing/testing.md#published-figures).
+
 `workspace-tests.json` stores the expected count for each suite and the canonical
 command that measures it. Verification re-runs the suite on the current
 checkout and requires the observed count to match. It makes no historical claim

HACKING.md

@@ -631,12 +631,18 @@ chained to the previous one.
   short reference version of this file.
 - [`docs/testing/user-stories.md`](docs/testing/user-stories.md) — the
   20 worked Atomic scenarios and their pass criteria (the atomic family itself
-  is 54 stories; the Ubuntu family is a further 99).
+  is <!-- fig:stories_atomic floor=10 -->50+<!-- /fig --> stories; the Ubuntu family is a further <!-- fig:stories_ubuntu floor=10 -->90+<!-- /fig -->).
 - [`tests/e2e/atomic-vm.sh help`](tests/e2e/atomic-vm.sh) — the
   subcommand reference, kept in sync with the script itself.
 
 ## 17. Adding a new action
 
+Published figures (the action count, test counts, story-set sizes) are never typed:
+they are `fig` markers rendered by `scripts/render_figures.py`. After adding an
+action or a story, run `UPDATE_TEST_BASELINE=1 scripts/test_baseline.sh` and commit
+the evidence file and whatever it re-rendered. Syntax:
+[docs/contributing/testing.md](docs/contributing/testing.md#published-figures).
+
 An "action" is a named operation the daemon can perform — `GetDiskUsage`,
 `AddLayeredPackage`, `CreateUser`, etc. Action names are string literals
 duplicated across several independent match expressions and lists. The

README.md

@@ -18,6 +18,8 @@
   <a href="https://github.com/lacs-project/sysknife/discussions"><img src="https://img.shields.io/github/discussions/lacs-project/sysknife?style=flat-square&label=discuss" alt="Discussions"></a>
   <a href="https://www.npmjs.com/package/sysknife-setup"><img src="https://img.shields.io/npm/v/sysknife-setup?style=flat-square&logo=npm&label=npx%20setup" alt="npm version"></a>
   <a href="https://glama.ai/mcp/servers/lacs-project/sysknife"><img src="https://glama.ai/mcp/servers/lacs-project/sysknife/badges/score.svg" alt="Glama MCP server quality score"></a>
+  <a href="https://github.com/lacs-project/sysknife/blob/main/tests/evidence/workspace-tests.json"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2Flacs-project%2Fsysknife%2Fmain%2Ftests%2Fevidence%2Fworkspace-tests.json&query=$.tests&label=Rust%20tests&style=flat-square&color=2f855a" alt="Rust test count, read from the recorded baseline"></a>
+  <a href="https://github.com/lacs-project/sysknife/blob/main/tests/evidence/workspace-tests.json"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2Flacs-project%2Fsysknife%2Fmain%2Ftests%2Fevidence%2Fworkspace-tests.json&query=$.frontend_tests&label=frontend%20tests&style=flat-square&color=2f855a" alt="Frontend test count, read from the recorded baseline"></a>
 </p>
 
 <p align="center">
@@ -234,9 +236,9 @@ privilege boundary is worse than none:
 sudo make uninstall
 ```
 
-All three Ubuntu LTS releases record a live-VM run of 79 stories from the Ubuntu
-suite (which has since grown to 96), and each run has a replay twin that reproduces it: 22.04, 24.04 and 26.04
-all at 79/79, every twin serving every call with zero misses. The runs are in
+All three Ubuntu LTS releases record a live-VM run of <!-- fig:lts_story_total -->79<!-- /fig --> stories from the Ubuntu
+suite (which has since grown to <!-- fig:stories_ubuntu floor=10 -->90+<!-- /fig -->), and each run has a replay twin that reproduces it: 22.04, 24.04 and 26.04
+all at <!-- fig:lts_story_rate -->79/79<!-- /fig -->, every twin serving every call with zero misses. The runs are in
 `tests/evidence/story-runs/`. The suite grew from 50 when every Debian-only
 action got a story, `GetHostState` first.
 Fedora Atomic is the rpm-ostree target; `make install` does not complete on
@@ -342,18 +344,18 @@ milestone.
 | Component | State |
 |---|---|
 | `sysknife-brain` — LLM planner, tool loop, safety fence | ✅ |
-| `sysknife-daemon` — 192 typed actions, auth, preview, transactions | ✅ |
+| `sysknife-daemon` — <!-- fig:typed_actions floor=10 -->190+<!-- /fig --> typed actions, auth, preview, transactions | ✅ |
 | Live IPC + streaming + atomic-host rollback (rpm-ostree) | ✅ |
 | Terminal approval gate — one-time, TTL-bounded receipts | ✅ |
 | MCP server (Claude Code / Cursor / any MCP client) | ✅ |
 | Tamper-evident Ed25519-signed audit chain | ✅ |
 | RFC 5424 syslog forwarding (Splunk / Sentinel / QRadar) | ✅ |
 | Postgres backend (RDS / Cloud SQL / Neon / Supabase) | ✅ |
-| **Ubuntu support** — 79/79 stories on a live 22.04 VM, recorded in `tests/evidence/story-runs/` | ✅ |
-| **Every Ubuntu LTS validated** — 22.04, 24.04 and 26.04 all at 79/79, each with a replay twin that reproduces it | ✅ |
+| **Ubuntu support** — <!-- fig:story_run:ubuntu-22.04-gpt-oss-120b -->79/79<!-- /fig --> stories on a live 22.04 VM, recorded in `tests/evidence/story-runs/` | ✅ |
+| **Every Ubuntu LTS validated** — 22.04, 24.04 and 26.04 all at <!-- fig:lts_story_rate -->79/79<!-- /fig -->, each with a replay twin that reproduces it | ✅ |
 | Telegram approval interface | 📋 roadmap |
 
-**2,011 Rust tests and 72 frontend tests** form the current deterministic
+**<!-- fig:rust_tests floor=1000 -->2,000+<!-- /fig --> Rust tests and <!-- fig:frontend_tests floor=10 -->70+<!-- /fig --> frontend tests** form the current deterministic
 release baseline.
 
 ## Configure your LLM
@@ -399,8 +401,8 @@ All config files that may contain API keys are created with `chmod 0600`.
 
 See [ROADMAP.md](ROADMAP.md) for the full milestone breakdown.
 
-- ✅ **Ubuntu 22.04** — 79/79 stories on a live VM (recorded in `tests/evidence/story-runs/`)
-- ✅ **Ubuntu 24.04 and 26.04** — 79/79 and 79/79 on live VMs; every LTS run has a replay twin that reproduces it
+- ✅ **Ubuntu 22.04** — <!-- fig:story_run:ubuntu-22.04-gpt-oss-120b -->79/79<!-- /fig --> stories on a live VM (recorded in `tests/evidence/story-runs/`)
+- ✅ **Ubuntu 24.04 and 26.04** — <!-- fig:story_run:ubuntu-24.04-gpt-oss-120b -->79/79<!-- /fig --> and <!-- fig:story_run:ubuntu-26.04-gpt-oss-120b -->79/79<!-- /fig --> on live VMs; every LTS run has a replay twin that reproduces it
 - ✅ `sysknife audit export` — stored signed chain rows as JSON with `--since` / `--limit`
 - 📋 Telegram inline-button approvals
 - 📋 CEF / NDJSON output modes for SIEM ingest

docs/architecture.md

@@ -72,7 +72,7 @@ component uses the same resolution order.
 
 Privileged. The only component that touches the system. Provides:
 
-- 192 typed actions (rpm-ostree, systemd, firewall, users, containers,
+- <!-- fig:typed_actions floor=10 -->190+<!-- /fig --> typed actions (rpm-ostree, systemd, firewall, users, containers,
   flatpak, toolbox, SSH, kernel args, …)
 - Role-based authorization (`Observer` → `Dev` → `Admin`, plus `Boot`)
 - Policy enforcement: stale-approval detection, request hash validation

docs/contributing/testing.md

@@ -13,7 +13,7 @@ validation before a release.
 | Integration (Rust) | Daemon IPC, safety fence, policy | <10s | Every commit, every CI run |
 | **Dev stories (local, no VM)** | LLM plan structure for read-only stories; runs on any Linux host | 1-3 min | After brain/prompt changes |
 | CI smoke (container) | Daemon + Ollama + read-only stories in a Linux runner | 5-10 min | Opt-in (manual `workflow_dispatch` only) |
-| E2E Atomic VM | **Real Silverblue** in QEMU/KVM, full stack, all 54 stories | 15-30 min first boot; 2-3 min subsequent | Local / pre-release |
+| E2E Atomic VM | **Real Silverblue** in QEMU/KVM, full stack, all <!-- fig:stories_atomic floor=10 -->50+<!-- /fig --> stories | 15-30 min first boot; 2-3 min subsequent | Local / pre-release |
 | **E2E Ubuntu VM** | **Ubuntu 24.04** cloud image, full story suite (pass rate in `tests/evidence/story-runs/`) | ~15 min first boot; ~2 min subsequent | After Ubuntu action changes |
 | Manual QA | Real Silverblue/Kinoite hardware, destructive actions, GUI | 30-60 min | Before releases + demo video |
 
@@ -275,7 +275,7 @@ sudo chmod +r /boot/vmlinuz-*
 # Run the read-only stories (non-destructive default)
 ./tests/e2e/atomic-vm.sh run
 
-# Run ALL 54 stories including destructive — restore the baseline afterwards.
+# Run ALL stories including destructive — restore the baseline afterwards.
 SYSKNIFE_ALLOW_DESTRUCTIVE=1 ./tests/e2e/atomic-vm.sh run
 
 # Roll back to the clean baseline so the next run is fast
@@ -376,12 +376,43 @@ sudo -E tests/e2e/run-stories.sh 1 4 7
 
 Per-story logs are written to `tests/e2e/logs/story-N.log`.
 
+## Published figures
+
+Never type a figure. A count the README or the docs state about the project
+(tests, actions, story-set sizes, recorded pass rates) is rendered from its
+source by `scripts/render_figures.py`, and `scripts/check_evidence_claims.py`
+refuses one that sits outside a marker. After you add tests, run
+`UPDATE_TEST_BASELINE=1 scripts/test_baseline.sh` (add `--frontend` for the
+vitest suite), then commit the evidence file and whatever it re-rendered.
+
+A marker is an HTML comment pair around the value, invisible on GitHub and in
+mdBook:
+
+- `<!-- fig:NAME -->VALUE<!-- /fig -->` renders the exact value with thousands
+  separators. Story pass rates read from a recorded run are exact.
+- `<!-- fig:NAME floor=K -->VALUE<!-- /fig -->` renders the value rounded down to
+  a multiple of `K` with a trailing plus. A figure a routine PR moves is a
+  floor, so most PRs change no prose. `render_figures.py` refuses a floor above
+  the value.
+
+`NAME` is a key in the registry at the top of `scripts/render_figures.py`; each
+one reads its source through a function from `check_evidence_claims.py`. A
+marker sits on one line, cannot nest, and cannot sit inside a fenced code
+block. Inline code spans are ignored, which is how this section shows the
+syntax. `scripts/render_figures.py --check` names the file and line of every
+stale, unknown or malformed marker, and `--write` fixes the stale ones.
+
+The exact Rust and frontend test counts are the two README badges, which read
+`tests/evidence/workspace-tests.json` from `main`.
+
 ## Before opening a PR
 
 1. `cargo nextest run --workspace && npm test --prefix apps/sysknife-shell` — required, fast
 2. `cargo clippy --workspace --all-features --all-targets --locked -- -D warnings`
 3. `cargo fmt --all --check`
-4. For changes to the brain, daemon, IPC, or action catalogue:
+4. If you added or removed tests, `UPDATE_TEST_BASELINE=1 scripts/test_baseline.sh`
+   and commit what it changed (see [Published figures](#published-figures))
+5. For changes to the brain, daemon, IPC, or action catalogue:
    - Run the VM tests locally (`atomic-vm.sh` flow)
    - Optionally run the CI smoke test by dispatching the `e2e` workflow
 
@@ -393,7 +424,7 @@ The maintainer runs these in order:
 2. VM tests (at least Silverblue + one other atomic variant) pass locally
 3. Manual QA on real Silverblue hardware using
    [docs/testing/user-stories.md](../testing/user-stories.md) as the
-   checklist — all 54 stories including destructive ones
+   checklist — all <!-- fig:stories_atomic floor=10 -->50+<!-- /fig --> stories including destructive ones
 4. Record the demo video on real hardware (issue #32)
 
 ## Troubleshooting

docs/contributing/ubuntu-vm-testing.md

@@ -7,9 +7,9 @@ Supported Ubuntu LTSes:
 
 | Release | Codename | SSH port | Live suite | Status |
 |---|---|---|---|---|
-| 22.04 | jammy | 2222 | 79/79 | validated |
-| 24.04 | noble | 2223 | 79/79 | validated (default) |
-| 26.04 | resolute | 2224 | 79/79 | validated |
+| 22.04 | jammy | 2222 | <!-- fig:story_run:ubuntu-22.04-gpt-oss-120b -->79/79<!-- /fig --> | validated |
+| 24.04 | noble | 2223 | <!-- fig:story_run:ubuntu-24.04-gpt-oss-120b -->79/79<!-- /fig --> | validated (default) |
+| 26.04 | resolute | 2224 | <!-- fig:story_run:ubuntu-26.04-gpt-oss-120b -->79/79<!-- /fig --> | validated |
 
 Each release has a record run and a `.replay.json` twin in
 `tests/evidence/story-runs/`, and `tests/release/cassette-replay-parity.test.sh`

docs/developer-guide.md

@@ -282,6 +282,13 @@ and the workspace test baseline. `scripts/test_baseline.sh` runs
 `cargo nextest run --workspace --locked` and checks the measured test count.
 Run `bash .githooks/pre-commit` to invoke the same gate manually.
 
+Never type a figure into the README or the docs. Add your tests, run
+`UPDATE_TEST_BASELINE=1 scripts/test_baseline.sh`, and commit the evidence file
+and whatever it re-rendered. The command records the count and then runs
+`scripts/render_figures.py --write`, which rewrites every `fig` marker from its
+source. The marker syntax is explained in
+[docs/contributing/testing.md](contributing/testing.md#published-figures).
+
 The secret screen recognizes the credential formats of every key-bearing
 provider in the setup wizard, including OpenAI service-account and `None`
 keys, DeepSeek's hexadecimal keys, and xAI keys. Mistral's keys have no
@@ -367,7 +374,7 @@ crates/
   sysknife-brain/     LLM planner, provider adapters, safety fence
   sysknife-types/     Shared domain types (CallerRole, RiskLevel, JobState, …)
   sysknife-core/      Config loading, shared constants
-  sysknife-daemon/    Privileged executor, 191 actions with an `ActionSpec`,
+  sysknife-daemon/    Privileged executor, one `ActionSpec` per action,
                       IPC, rollback, SQLite
   sysknife-proto/     Protobuf definitions (future use)
   sysknife-daemon-test/  TAP integration harness for the daemon IPC layer (VM only)