Skip to content

[PERF] Update reverse starts-match min boundary - #1685

Closed
samukweku wants to merge 1 commit into
devfrom
issue-85-min-rev-starts-matches
Closed

[PERF] Update reverse starts-match min boundary#1685
samukweku wants to merge 1 commit into
devfrom
issue-85-min-rev-starts-matches

Conversation

@samukweku

Copy link
Copy Markdown
Collaborator

Summary

  • remove the redundant length argument from the starts+matches reverse min helper
  • pass length only to the remaining starts-only reverse aggregations that still require it
  • follow the compact-state implementation in janitor-rs #88
  • enforce benchmark evidence in performance PRs and tracking issues via AGENTS.md

Performance and memory

The paired Rust implementation was compared with the old two-HashMap kernel:

Scale Unique labels Duplicate labels
Tiny 0.10 -> 0.05 ms 0.09 -> 0.05 ms
Large 9.70 -> 5.30 ms 9.73 -> 5.21 ms
Very large 101.14 -> 52.34 ms 95.67 -> 51.07 ms
Super large 537.81 -> 282.81 ms 483.93 -> 254.12 ms

Allocation bytes for one call:

Scale Unique labels Duplicate labels
Tiny 4,368 -> 8,232 4,368 -> 2,856
Large 69,648 -> 83,880 69,648 -> 35,496
Very large 139,280 -> 167,848 139,280 -> 70,312
Super large 278,544 -> 532,392 278,544 -> 139,944

The benchmark uses the same aggregation traversal and a counting allocator; it is not a full PyO3 call benchmark. Unique-label memory increases because compact vectors retain output state; duplicate-label memory decreases substantially.

Verification

  • Python compileall
  • pre-commit suite: Ruff, Ruff format, pydoclint, and repository checks

Closes #85
Part of #23

@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://pyjanitor-devs.github.io/pyjanitor/pr-preview/pr-1685/

Built to branch gh-pages at 2026-08-25 23:37 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@samukweku

Copy link
Copy Markdown
Collaborator Author

Superseded by #1684, which now contains both the sum and min boundary updates.

@samukweku samukweku closed this Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant