Repository navigation
docs: render every published figure from its source - #685
Merged
Merged
Conversation
Every number the README and docs state about the project is now a marker,
<!-- fig:NAME -->VALUE<!-- /fig -->, that scripts/render_figures.py renders
from the artifact or source that produces it. Figures a routine PR moves are
rounded floors ("2,000+"); recorded story pass rates stay exact.
check_evidence_claims.py now refuses a figure its checks match outside a
marker, and keeps its semantic checks (full-family runs, validated tiers,
coverage anchors) running on the rendered text. It also owns the marker
grammar and the new source readers for recorded runs. check_public_claims.sh
runs render_figures.py --check. The README shows the exact Rust and frontend
test counts as live badges read from tests/evidence/workspace-tests.json.
The README line "has since grown to 96" was stale (the Ubuntu family holds
99); it is now derived. Two figures sat inside fenced code blocks, where a
marker is refused, and were reworded without the number.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UpyrhbDTW1gBwzGAC44GYU
UPDATE_TEST_BASELINE=1 scripts/test_baseline.sh now runs render_figures.py --write after recording, so one command updates the evidence and every figure derived from it. A failed render fails the recording. The provenance test stubs the renderer and checks the call, the arguments, that a verifying run never renders, and that a render failure is not swallowed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UpyrhbDTW1gBwzGAC44GYU
Adds to public-claims.test.sh: a stale marker (names file and both values), an unknown NAME, a figure outside a marker, a marker inside a code fence, an unbalanced and a nested marker, a required doc that stopped stating the test count, the floor boundary (v = K*n, v = K*n - 1, v = K, v < K), --write then --check, a --write that touches only marker contents, a second --write that changes no byte, an unreadable and a corrupt evidence file, and a tree with no markers. Each refusal was mutation-proved: the guard removed in a scratch copy, the test seen to go red, the file restored from a pristine copy and hash-checked. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UpyrhbDTW1gBwzGAC44GYU
Count lines keep their markers; the evidence file carries main's 2,011, which still renders as 2,000+. CHANGELOG keeps both Unreleased sections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UpyrhbDTW1gBwzGAC44GYU
vladimirrott
added a commit
that referenced
this pull request
Oct 27, 2026
Minor release. #627 keys request_hash with an HMAC under the audit key (chain_version 4; previews stored by an older release fail closed after upgrading) and #639 changes the history request a newer CLI sends, so a caller relying on the old behaviour notices: the middle digit moves. Also: the planner sees only its host's distribution (#626, #656, #658, #684), checkpoint and the MCP audit tools share the audit-store resolver (#636), every workflow job has a time limit (#662), a weekly cargo audit (#640), release hardening (#635, #637, #660), and every published figure is rendered from its source (#685). Claude-Session: https://claude.ai/code/session_01UpyrhbDTW1gBwzGAC44GYU Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every figure the README and docs publish about the project is now rendered from its source. Nobody types one any more.
Before this, a PR that added two tests had to hand-edit four places: the evidence file and three docs. Two such PRs conflicted on those lines, which happened three times on 2026-10-26 alone. The checker only noticed a wrong number after someone had typed it.
How it works
<!-- fig:NAME -->VALUE<!-- /fig -->, which GitHub and mdBook both render invisibly.scripts/render_figures.pyfills each marker from the same source readerscheck_evidence_claims.pyalready trusted: the test baseline, the action catalogue, the story directories and the recorded story runs.--checkfails on a stale, unknown, malformed or fenced marker;--writefixes them.2,000+ Rust tests,190+ typed actions), so a test-adding PR touches onlytests/evidence/workspace-tests.json. Recorded story pass rates stay exact (79/79), because only a deliberate run changes them.check_evidence_claims.pynow refuses any figure outside a marker, so a hand-typed number cannot creep back.UPDATE_TEST_BASELINE=1 scripts/test_baseline.shrecords the evidence and re-renders the docs in one step.Rendering already found one stale claim: the README said the Ubuntu suite "has since grown to 96" in one place and "99" in another. Both now render from
tests/e2e/stories.Verified
At
8a231c4:scripts/check_public_claims.sh,tests/release/public-claims.test.sh,tests/release/test-baseline-provenance.test.sh,scripts/check_test_reachability.sh,tests/release/shellcheck-coverage.test.sh, the CI markdownlint step andcheck_evidence_claims.pyall pass. The CI markdownlint step reports 0 issues in 11 files.stale figure 'rust_tests': stored '3,000+', rendered '2,000+'), a hand-typed2,004 Rust tests, and a marker removed around192 typed actions.I committed the merge of main (#627) with the local hook bypassed. After its upgrade to Ubuntu 26.04 this host no longer has the GTK/WebKit dev libraries, so the hook's full workspace build cannot link the paused
sysknife-shellcrate. CI runs the full gate.🤖 Generated with Claude Code
https://claude.ai/code/session_01UpyrhbDTW1gBwzGAC44GYU