Skip to content

perf: update reverse ends aggregation boundary - #1686

Open
samukweku wants to merge 3 commits into
devfrom
issue-94-pyjanitor-ends-matches
Open

perf: update reverse ends aggregation boundary#1686
samukweku wants to merge 3 commits into
devfrom
issue-94-pyjanitor-ends-matches

Conversation

@samukweku

Copy link
Copy Markdown
Collaborator

Closes the pyjanitor side of janitor-rs issue #94.

Updates reverse ends+matches aggregation dispatch for the length-free Rust API and includes a narrow fallback for older installed janitor_rs releases that still require length. The ordinary reverse ends path keeps its existing length argument.

Tests: pixi run pytest tests/functions/test_conditional_join.py -k agg_rev -q (20 passed).

Benchmark: the corresponding Rust PRs compare old maps vs compact slots across tiny, large, very large, and super large duplicate/unique workloads; results are documented on janitor-rs issue #94.

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://pyjanitor-devs.github.io/pyjanitor/pr-preview/pr-1686/

Built to branch gh-pages at 2026-08-27 04:37 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@samukweku

Copy link
Copy Markdown
Collaborator Author

Contract documentation is now included in this PR and its branch AGENTS.md.

  • Direct Rust reverse-match kernels reject an empty flattened matches tape; malformed tape shape is not silently accepted.
  • A batch with no candidates at all is valid at the Python level, so pyjanitor pre-filters that case and returns the normal typed empty result before dispatching to Rust.
  • Individual zero-width ranges remain valid when the overall tape is non-empty.
  • The comparison/producer layer owns the matches values-in-{0,1} invariant; the Rust boundary validates tape shape.
  • The compatibility length argument exists only for older janitor-rs wheels.

An ELI5 explanation is included in the Python module docstring and AGENTS.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant