Repository navigation
Conversation
Make the CPython leg comparable with the clojure twin: - Aggregation: callgrind_instructions is now the sum of per-case MINs (was per-case medians), matching the JVM/clojure-rs least-disturbed rule; cpu_time keeps the shared sum-of-medians rule. Benchmark names are byte-identical (proven old-vs-new on identical input). - Measure names converge on the twin: instructions -> callgrind_- instructions with lower/upper bounds; new peak_memory (max per-case median RSS in bytes, mirroring the twin's PeakMemory). - Every case now runs natively under GNU time (required, fail-fast without it) for real cpu_s/maxrss_kb instead of maxrss=0/wall-cpu approximations; Callgrind runs carry Ir only (--callgrind-repeat). CI installs GNU time; summary table gains a Peak RSS column. - Bencher: track callgrind_instructions with the twin's threshold shape (percentage, 10/64 samples, 1% boundary), strip repeat-1 cpu/peak before reporting, and warn visibly while BENCHER_API_KEY is unset (history needs a maintainer to create the Bencher project and set the secret). AI-Tool: Muse Code (Meta Muse Spark)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of hhh-pa/clojure-code-snippet#40 — the python-side comparability
work: same aggregation rule, real memory signal, Bencher history prep.
Changes
tools/measure.py:instructionsΣ-median →callgrind_instructionsΣ-min with lower/upper bounds, plus
peak_memory— mirroring the twinBMF shape. Runs restructure to the lean recipe:
--repeatnative runsunder GNU time (fail-fast without it; BSD
timerejected) +--callgrind-repeatIr-only runs.maxrss_kb/cpu_sare real now(were 0/wall-fallback).
test.yml: installstime; trackscallgrind_instructionsat 0.01boundary (was
instructions/0.10), matching clojure; strip-to-tracked,skip-if-empty, and a visible warning while the Bencher key is unset.
README perf section.
Validation
measure.py --self-testpasses (new time-parse/reject, Σ-min/bounds/peak asserts, merge fan-in tests).
peak, summary table, no-valgrind degradation, mismatch→exit 1 — all
correct. No benchmark renames proven (old-vs-new BMF keys identical,
393/393 match live CI keys).
Blocked on maintainer
BENCHER_API_KEYis not set on this repo (onlyRESOURCES_PASSPHRASEexists) — history arms after the secret + project exist (10 samples).
Everything else is code-complete.
AI-Tool: Muse Code (Meta Muse Spark)