Skip to content

perf: use range query kernel for wide dual ranges - #1693

Open
samukweku wants to merge 2 commits into
devfrom
issue-1652-pyjanitor
Open

perf: use range query kernel for wide dual ranges#1693
samukweku wants to merge 2 commits into
devfrom
issue-1652-pyjanitor

Conversation

@samukweku

@samukweku samukweku commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • use the janitor-rs direct range min/max kernel for wide unsorted dual-range selections
  • retain existing sorted-right, width-one, unsupported-dtype, and narrow-workload paths
  • fall back cleanly when the installed janitor-rs extension does not expose the new functions
  • add focused dispatch tests for first/last selection and fallback behavior

The supporting janitor-rs PR is pyjanitor-devs/janitor-rs#111.

Performance

End-to-end prototype timings on wide unsorted dual-range joins:

  • 10,000 rows: 9.4 ms -> 3.3 ms
  • 100,000 rows: 111 ms -> 36 ms

Validation

  • conditional_join suite: 241 passed, 1 skipped
  • dispatch + conditional_join suite: 245 passed, 1 skipped
  • randomized parity: 100 seeds x both keep modes
  • pre-commit hooks for changed files passed; repository-wide lint still reports pre-existing unrelated violations

Fixes #1652

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://pyjanitor-devs.github.io/pyjanitor/pr-preview/pr-1693/

Built to branch gh-pages at 2026-08-26 05:21 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@samukweku samukweku self-assigned this Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[PERF] Optimize unsorted dual-range keep first/last selection

1 participant