Fix #39: all 27 snippet complexity gaps in one batch by m-hu · Pull Request #45 · hhh-pa/clojure-code-snippet

m-hu · 2026-11-03T19:53:47Z

Closes #39 — all 27 snippets whose time/space complexity was worse than
their python twins, in one batch (merges the four area branches; files are
disjoint, merges conflict-free).

Graph (6): heaps + O(1) queues

graph/flow/min_cost_max_flow: Dijkstra heap pop re-sort → O(log h) PQ
(n=300: 10.1s → 3.9s).
graph/matching/blossom, graph/matching/gale_shapley,
graph/minimum_spanning_tree/second_best_mst, graph/shortest_path/spfa:
O(n) vec-dequeue → O(1) PersistentQueue (spfa star n=60000: 7.4s → 0.7s).
graph/shortest_path/dag_shortest_paths: topo re-sort + O(n) scans → PQ
- seen-vector (edgeless n=20000: 4.2s → 0.4s).

Strings + geometry (6): tries, bisection, band-only

string_related/trie/no_prefix_set: sort+segtree → persistent trie,
O(T log n) → O(T).
string_related/aho_corasick/determining_dna_health: prefix sums +
bisect, O(T·G) → O(T·log G) (5.8x).
string_related/banded_alignment/banded_alignment: full table →
band-only, O(nm) → O(nk) (4.4x).
computational_geometry/dcel_overlay/solution: O(n⁴) → O(n² log n) (11x).
computational_geometry/range_tree/solution: O(n² log n) → O(n log²n) (10x).
computational_geometry/segment_stabbing/solution: O(m) → O(log m)/query.

Misc time (8): heaps, hash-sets, no re-sorts

dp/huffman (17s → 0.74s), dp/offline_caching,
data_structure/offline_minimum, distributed/consensus/approx_agreement,
systems/cpu_scheduling (7.42s → 0.55s), systems/distance_vector_routing,
search/pl_resolution (19.72s → 2.65s), combinatorics/prufer.

Space (7): same time class, smaller footprint

In-place 3-way quicksorts (O(log n) aux), mult_weights flat indexing,
stirling_bell rolling rows, attribute_grammar Horner fold (3M-bit: OOM
→ 0.4s), iterative bst_lca (sorted-20k: StackOverflow → correct) and
red_black_tree inserts.

Validation

Every fix validated byte-identical before/after on hand-built small +
adversarial inputs via JVM clojure.main (some with independent python
oracles); pre-existing quirks preserved byte-identically.
Merge smoke test: 1 script per group runs on the merged tree with
master-identical behavior.
Shared GPG cases unavailable locally — CI run_tests.py +
perf-callgrind on this PR is the full gate.

Supersedes #41, #42, #43, #44.

AI-Tool: Muse Code (Meta Muse Spark)

CI: cache the built clojure-rs binary (from #46)

perf-callgrind resolves the upstream revision once but all 8
clojure-rs legs rebuilt it from scratch. The release binary is now
cached by runner OS + resolved SHA (repeat runs skip toolchain,
checkout, and build); a .rev marker fails the leg loudly on any
revision mismatch.

Idiomatic rewrite (review feedback)

The four fixes that used Java interop now use idiomatic Clojure with the
same complexity: PriorityQueue → sorted-set in atoms (mcmf, dag),
long-array/object-array + ArrayDeque → transient vectors +
persistent-list stacks (both 3-way quicksorts), .charAt → nth.
Old-vs-new outputs byte-identical on all completing inputs (incl. 2k/3k
sorts, 250-node complete-DAG mcmf, 200-node DAG); two oversized random
mcmf inputs OOM identically before and after.

CI run superseding

concurrency cancels stale branch runs on new pushes; master keeps
every run for Bencher continuity.

huffman: binary heap keyed [weight,min-leaf,id], no per-merge re-sort. offline_caching: eviction scans cache once, no per-eviction sort. offline_minimum: binary search for first extract after insert. approx_agreement: sort once per round, hoist trimmed mean. cpu_scheduling: RR ready queue via head index + amortized compaction. distance_vector: on-demand poison-reverse lookup, no N^3 table. pl_resolution: hash-set clause membership instead of linear scan. prufer: sorted-set min-leaf heap for encode and decode. AI-Tool: Muse Code (Meta Muse Spark)

- sort/three_way_quicksort: in-place 3-way quicksort on long-array with explicit stack (smaller side first, O(log n) depth); no 3 filterv copies per level. Avg O(n log n) time kept. - sort/three_way_string_quicksort: same in-place 3-way scheme on an object array with depth-tagged stack entries. - math/approximation/mult_weights: index the flat gain vector directly instead of building a T-by-n gain matrix. O(T n) time kept. - combinatorics/stirling_bell: rolling two-row DP (stirling-row) instead of the full (n+1)x(k+1) table with assoc-in. O(n k) time kept. - compiler/attribute_grammar: Horner fold over the bitstring instead of building an n-node map tree plus O(n) synth recursion. O(n) kept. - data_structure/bst_lca: iterative insert via explicit descent path with bottom-up rebuild; duplicates still go right. O(h) per insert kept. - data_structure/red_black_tree: iterative LLRB insert with explicit path and bottom-up balance fixup (duplicates balance on the way up, as before); root recolored black. O(log n) insert kept. AI-Tool: Muse Code (Meta Muse Spark)

no_prefix_set: O(T log n) sort+segtree -> O(T) linear trie, input-order insert. determining_dna_health: O(T.G) out-scan -> O(T.log G) via per-state prefix health sums plus bisection. banded_alignment: O(nm) full-table scan -> O(nk) band-only enumeration with transient rows. dcel_overlay: O(n^4) per-vertex/per-edge full scans -> single-pass grouping of crossings onto edges and half-edges onto tails. range_tree: O(n^2 log n) take-while scans -> O(n log^2 n) bisection in cascade build and query bounds. segment_stabbing: O(m) per-query rank scan -> O(log m) bisection. AI-Tool: Muse Code (Meta Muse Spark)

- min_cost_max_flow: sorted-vector heap re-sorted per pop -> java.util.PriorityQueue (pop O(h log h) -> O(log h)) - blossom, gale_shapley, second_best_mst, spfa: vector queue copied per dequeue -> PersistentQueue (dequeue O(n) -> O(1) amortized) - dag_shortest_paths: topo vector re-sorted per vertex + O(n) acc-membership scan per edge -> PriorityQueue + seen vector (poll O(h log h)+scan -> O(log h)+O(1)) Validation: 38 hand-built before/after runs (small + adversarial), all byte-identical; e.g. gale n=1500 11.2s->2.3s, spfa star60k 7.4s->0.7s, dag indep20k 4.2s->0.4s, mcmf n=300 10.1s->3.9s, sbmst star250 5.4s->2.0s. AI-Tool: Muse Code (Meta Muse Spark)

Each perf-callgrind run resolves the upstream revision once but all 8 clojure-rs legs still check out + build it from scratch. Cache the release binary in actions/cache keyed by runner OS + resolved SHA so repeat runs skip toolchain, checkout, and build; a .rev marker file fails the leg loudly if a cached binary ever mismatches the revision. AI-Tool: Muse Code (Meta Muse Spark)

PriorityQueue heaps become persistent sorted-sets in atoms (first/disj/conj, still O(log h) per op); array quicksorts become transient vectors with persistent-list stacks (nth/assoc! O(1), smaller-first keeps O(log n) stack); String .charAt becomes nth. Old-vs-new outputs byte-identical on all completing inputs. AI-Tool: Muse Code (Meta Muse Spark)

concurrency cancel-in-progress on non-master refs so a new push replaces the previous commit's sweep; master keeps every run so no landed commit loses its Bencher record. AI-Tool: Muse Code (Meta Muse Spark)

m-hu · 2026-11-03T20:57:28Z

Coordination note from issue #40 (harness side; no snippet changes here, nothing duplicated):

The Make perf-callgrind comparable with python-code-snippet #40 harness adds 13 dedicated slow callgrind legs (3600s/case, benchmark-pinned) plus a merge gate that fails on any remaining callgrind skip. It recovers whatever timeout budget alone can recover.
Overlap with this PR: string_related/trie/no_prefix_set (12 timeout cases x2 runtimes), dp/edit_distance/solution, and graph/shortest_path/* timeouts. If any of those still exceed 3600s/case pre-Fix #39: all 27 snippet complexity gaps in one batch #45, the Make perf-callgrind comparable with python-code-snippet #40 gate stays red until this PR lands; if they fit in 3600s, Make perf-callgrind comparable with python-code-snippet #40 goes green first and Fix #39: all 27 snippet complexity gaps in one batch #45 then shrinks the slow legs' runtime.
Suggested sequence: land Fix #39: all 27 snippet complexity gaps in one batch #45 first if its CI is green, then rebase/re-run Make perf-callgrind comparable with python-code-snippet #40 so the gate measures the fixed snippets. Either order is safe (legs are additive; main shards just exclude the slow set).

AI-Tool: Muse Code (Meta Muse Spark)

github-actions · 2026-11-03T21:20:55Z

Bencher Report

Project	clojure-code-snippet
Branch	fix/39-batch-all
Commit	`96ac83e`

Report	Testbed	Adapter	Status
1	ubuntu-latest-clojure-rs	JSON	✅ No Alerts
2	ubuntu-latest-jvm	JSON	✅ No Alerts

⚠️ WARNING: Truncated view!
The full continuous benchmarking report exceeds the maximum length allowed on this platform.

michael-h-hu

Review: PR #45 (fix/39-batch-all @ `96ac83e`) — round 1

Focused re-review of the full master...HEAD diff, all 28 files. No blocking findings.

What I checked (correctness of each transformation, not just shape)

sort (2 files): in-place 3-way partition on transient vectors is a faithful
Dijkstra partition (pivot value captured before swaps; empty ranges skipped on
pop). Smaller-first stack push in three-way-qsort bounds depth at O(log n).
MSD variant pushes the equal partition with (inc dd) only when pivot-c != -1.
heaps/queues: huffman binary heap preserves the [weight, min-leaf, id] total
order (init ms0 reproduces the old sort keys); mcmf/dag sorted-set heaps keep
stale-entry skipping / zero-indegree-min-first semantics; the four
vector-queue → PersistentQueue swaps preserve FIFO order.
prufer: first on the neighbor set is safe because a min-leaf always has
degree exactly 1 (single-element set), so min == only element. Leaf-set
maintenance matches the old boolean-vector logic.
stirling_bell: rolling-row recurrences match the old table, including the
signed-first-kind (- i) == -(i1-1) factor.
attribute_grammar: synth-bits computes the same MSB-first value as the old
nested-map synth, and removes deep recursion. (Empty bit-string now yields 0
instead of nil — pre-existing edge, arguably a fix.)
dna_health: bisection invariant verified in build-automaton — genes insert
in gid order via map-indexed, so every :out is ascending and :pref
differences are exact.
dcel/range_tree/segment_stabbing/offline_minimum: grouping passes and binary
searches reproduce the old scans (cross-multiplied comparator equals ratio order
for positive denominators; bisect indices equal the old take-while counts).
rb_tree/bst_lca: iterative rebuilds apply balance at every level including
the duplicate path, matching the recursive versions.
pl_resolution/offline_caching/approx_agreement/mult_weights/cpu_scheduling/
dv_routing/banded/no_prefix_set: set-membership / hoisted-sort / direct-index /
head-index-queue / on-demand-advd / band-only / trie rewrites all preserve
output semantics (checked tie-breaks, index mappings, and bad-word conditions).
workflow: concurrency group cancels stale branch runs but never master;
cljrs-bin cache key is exact-SHA with a .rev marker check that fails loudly
on mismatch. Missing trailing newline in banded_alignment is pre-existing.

Verdict

APPROVE. CI-gated behavior (sweep diff-gates on both runtimes) corroborates the
read-through; the one pending leg (perf-callgrind (5, jvm)) just needs to go
green before landing.

m-hu added 8 commits November 3, 2026 20:25

Merge fix/39-graph-complexity into fix/39-batch-all

8041ce1

Merge fix/39-str-geom-complexity into fix/39-batch-all

93021af

Merge fix/39-misc-complexity into fix/39-batch-all

56e6cb6

Merge fix/39-space-complexity into fix/39-batch-all

cfd7960

m-hu added 2 commits November 3, 2026 21:24

Merge perf-cache-cljrs-bin into fix/39-batch-all

6870f2d

m-hu mentioned this pull request Nov 3, 2026

Cache built clojure-rs binary by resolved SHA in perf-callgrind #46

Closed

m-hu added 2 commits November 3, 2026 21:34

Supersede stale runs: one active run per branch

96ac83e

concurrency cancel-in-progress on non-master refs so a new push replaces the previous commit's sweep; master keeps every run so no landed commit loses its Bencher record. AI-Tool: Muse Code (Meta Muse Spark)

m-hu mentioned this pull request Nov 3, 2026

Recover slow perf-callgrind legs; document unified rule (#40) #47

Open

michael-h-hu approved these changes Nov 3, 2026

View reviewed changes

m-hu merged commit 7e0d386 into master Nov 3, 2026
26 checks passed

m-hu mentioned this pull request Nov 3, 2026

Gate perf-callgrind on usable valgrind (verify, repair, fail loud) #48

Merged

Navigation Menu

Repository navigation

Fix #39: all 27 snippet complexity gaps in one batch - #45

Fix #39: all 27 snippet complexity gaps in one batch#45
m-hu merged 12 commits into
masterfrom
fix/39-batch-all

m-hu commented Nov 3, 2026 •

edited

Loading

Uh oh!

m-hu commented Nov 3, 2026

Uh oh!

github-actions Bot commented Nov 3, 2026 •

edited

Loading

⚠️ WARNING: Truncated view!

Uh oh!

michael-h-hu left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

m-hu commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Graph (6): heaps + O(1) queues

Strings + geometry (6): tries, bisection, band-only

Misc time (8): heaps, hash-sets, no re-sorts

Space (7): same time class, smaller footprint

Validation

CI: cache the built clojure-rs binary (from #46)

Idiomatic rewrite (review feedback)

CI run superseding

Uh oh!

m-hu commented Nov 3, 2026

Uh oh!

github-actions Bot commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bencher Report

⚠️ WARNING: Truncated view!

Uh oh!

michael-h-hu left a comment

Choose a reason for hiding this comment

Review: PR #45 (fix/39-batch-all @ 96ac83e) — round 1

What I checked (correctness of each transformation, not just shape)

Verdict

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

m-hu commented Nov 3, 2026 •

edited

Loading

github-actions Bot commented Nov 3, 2026 •

edited

Loading

Review: PR #45 (fix/39-batch-all @ `96ac83e`) — round 1