[m-hu]

Problem

Run 41713792243 (PR #45): 9 of 16 perf-callgrind shards finished green in 1-5 min having measured NOTHING — callgrind_repeat: 0, walltime-only entries, 0 benchmarks in sweep-bench.json — via the "no callgrind data; skipping Bencher report" path. The other 7 shards did real callgrind (26-91 min). Per-runner luck, not runtime-specific.

Evidence (all 16/16 shards, run 41713792243)

The workflow's apt step (awalsh128/cache-apt-pkgs-action, same packages + version: 1.2 on every leg) succeeded green in 1-13s on ALL shards — but restored different caches, because the action normalizes the package list WITH apt-resolved versions, and the fleet is mid-rollout across image builds:

shards	image	linux-tools resolves to	cache key	restored	valgrind warning
9 (rs 0,1,2,7; jvm 0,1,2... see below)	20261025.179.1 (e.g. job 126197885347)	7.0.0-38.38	79ad5552… (1341 B)	0 packages	yes → skip
7 (rs 3,4,5,6; jvm 2,5,6)	20261101.186.1 (e.g. job 126197885398)	7.0.0-42.42	085cee66… (45 MB)	9 packages incl. valgrind 1:3.26.0	no → real callgrind

Skip shards: rs 0,1,2,7 + jvm 0,1,3,4,7. Real shards: rs 3,4,5,6 + jvm 2,5,6.

Decisive log lines (skipper job 126197885347, shard 4/jvm):

• Value hashed as '79ad5552f5c1bc04dd11fc2a2cdd7690' → Cache hit, Cache Size: ~0 MB (1341 B)
• Found 3 files in the cache. (cache_key.md5, install.log, manifest_main.log — zero .tar, no manifest_all.log)
• Restoring 0 packages from cache... → step still green
• warning: valgrind unavailable; callgrind_instructions will be null → callgrind_repeat: 0 in sweep-results.json

The 79ad… entry is degenerate: it was saved by a job on which apt unpacked nothing (zero ^Unpacking lines → zero tars, not even a manifest_all.log — see get_installed_packages in the action's lib.sh), so every restore of it provides nothing while reporting success. snip measure (cmd/snip/measure.go:150-152 + internal/measure/exec.go:136-143) only warns on unusable valgrind and continues at repeat 0 — hence green vacuous shards.

Fix

One purely additive step (+25/-0) in the perf-callgrind job, right after "Install measurement tools" — no overlap with PR #45's hunks (concurrency block + clojure-rs caching), so rebase-friendly:

• Probe exactly what snip probes: valgrind on PATH + timeout 30 valgrind --version exit 0.
• On failure, reinstall once from apt (update + install -y --reinstall of the same six packages), then re-probe.
• Fail LOUDLY if valgrind is still unusable — never again a green vacuous shard. Records valgrind: <version> in the step summary for future diagnosis.

The pilot job is untouched (it never passes --callgrind). The degenerate cache entry is deliberately left in place: with this gate it self-heals per-leg via the apt reinstall, and deleting shared cache state is left to maintainers as an optional follow-up.

Verification

• Workflow YAML parses; gate script passes bash -n.
• Gate logic proven locally with stubbed valgrind/timeout/apt under bash -e: (A) usable → exit 0, no apt call, version in summary; (B) broken + apt repair works → apt update+reinstall called, exit 0; (C) broken + repair fails → exit 1 with valgrind's stderr.
• Live: this PR's own CI exercises the new step on all 16 legs within minutes of opening (before the long sweep); gate outcomes will be reported back here. A full 6h sweep solely to validate was not run.

Session: gentle-menkar

[github-actions]

Bencher Report

Project	clojure-code-snippet
Branch	fix/valgrind-verify-gate
Commit	6ddda24

Report	Testbed	Adapter	Status
1	ubuntu-latest-clojure-rs	JSON	✅ No Alerts
2	ubuntu-latest-jvm	JSON	✅ No Alerts

⚠️ WARNING: Truncated view!

The full continuous benchmarking report exceeds the maximum length allowed on this platform.

[michael-h-hu]

Review: Gate perf-callgrind on usable valgrind (PR #48)

No findings. Approve.

What was inspected

• Diff (gh pr diff 48): +25/-0 in .github/workflows/test.yml only, one new Verify valgrind usable step. Merge-base 11de561 → head 6ddda24 touches nothing else; docs/ untouched, pilot job byte-identical to base.
• Probe equivalence: gate usable() { command -v valgrind >/dev/null && timeout 30 valgrind --version >/dev/null 2>&1; } matches CallgrindUsable() in scripts/ci/go/internal/measure/exec.go:136-143 exactly (exec.LookPath("valgrind") + Run([path, "--version"], nil, 30), err==nil). measure.go:150-152 confirms this is the warn-and-continue-at-repeat-0 path being gated.
• Repair branch: single apt-get update, then one install -y --reinstall of the exact perf-callgrind package list (time linux-tools-common linux-tools-generic valgrind gcc, 5 packages — same list and order as the Install measurement tools step, version 1.2). Re-probe after repair via bare command -v valgrind + ver=$(timeout 30 valgrind --version); under the runner's default bash -e -o pipefail either failing aborts the step loudly. valgrind: <version> recorded to $GITHUB_STEP_SUMMARY.
• Shell strictness: gate script passes bash -n; contains no pipes and no tee (the pre-push pipefail-masking pattern is gone — re-verified). usable() is only invoked under if !, so -e cannot misfire on the probe; fail-loud lines are unconditional.
• Ordering: Install measurement tools (line 224) → Verify valgrind usable (245) → Measure callgrind sweep (258). Gate runs before any measurement on every perf-callgrind leg.
• Scope: pilot untouched — its Measure AOT pilot step passes --repeat/--alloc-repeat but never --callgrind, and has its own 4-package install list (no gcc, version 1.1), correctly not mirrored by the gate.
• Hygiene: title clean (no session id); body ends Session: gentle-menkar; commit message signed; session:gentle-menkar label present; YAML parses (yaml.safe_load OK).
• Mergeability: mergeable: MERGEABLE, mergeStateStatus: CLEAN; local trial merge of master (post-#45) into the branch auto-merges cleanly — the missing concurrency/cljrs-cache hunks are non-overlapping as the PR body claims, so rebase-at-land is trivial.

Live proof (production run 41737221419, this PR's own CI)

• All 26 checks green, incl. all 16 perf-callgrind legs, perf-pilot, both report-callgrind, and Bencher Report.
• Both gate paths exercised live: 13/16 legs restored the degenerate 0 packages cache and printed cached valgrind unusable; reinstalling from apt (repaired); 3/16 restored 9 packages and passed through ((0,jvm), (4,jvm), (5,clojure-rs) — exactly the non-repair set).
• All 16 legs emitted a valgrind-X.Y.Z version line post-gate; zero valgrind unavailable warnings and zero no callgrind data; skipping Bencher report paths (vs 9/16 vacuous in run 41713792243). Leg durations 7m–2h32m confirm real callgrind, not 1–5 min vacuous shards. Bencher reports created successfully.

Residual risks / gaps

• The degenerate 79ad… cache entry is deliberately left in place (self-heals per-leg via the repair branch). Optional maintainer follow-up to evict it; not a blocker.
• Branch is one commit behind master (#45); merge is clean so no rebase needed for approval, but land should rebase or merge-queue to re-run CI on the combined tree.

Round 1 — no changes requested.

Session: gentle-menkar
AI-Tool: Muse Code (Meta Muse Spark)