[PERF] Selective predicate selection doesn't reach equi+non-equi combined joins

[samukweku]

Background

Found while adversarially reviewing #1663 (the fix for #1659, range-join le_lt/ge_gt selection).

_helpers._separate_conditions_based_on_op produces mapping["le_lt"]/["ge_gt"]/["le_or_ge"], consumed by two different dispatch trees:

• _get_indices_non_equi.py - pure non-equi joins. This is what [PERF] Selectivity-aware anchor predicate selection for conditional_join #1658 (same-direction le_or_ge, via _select_anchor) and [PERF] Selective bound selection for range joins #1663 (range-join le_lt/ge_gt, via _maybe_select_better_range_bounds) both fixed.
• _get_indices_equi.py - joins combining an == predicate with non-equi predicates. Neither fix reaches this path: _equi_uniq_join.py and _equi_range_join.py consume mapping["le_lt"]/["ge_gt"] directly (still first-encountered, never selectivity-aware), and _equi_not_range_join.py consumes mapping["le_or_ge"] the same pre-[PERF] Selectivity-aware anchor predicate selection for conditional_join #1658 way ((l1_col, r1_col, op), *rest = mapping["le_or_ge"], no _select_anchor call at all).

Confirmed empirically (during #1663's review): a query combining grp ==, a > lo, and two <-type candidates with real, measured selectivity skew dispatches through _equi_range_join._get_indices with le_lt still bound to the unswapped, less-selective candidate.

Why this matters

This isn't a new inconsistency - _equi_not_range_join.py's gap predates #1658 and #1663 just didn't close the equi-adjacent copies either, for the same reason (each fix was scoped to the non-equi dispatch tree it touched). But it means none of #1657/#1658/#1659/#1663's fixes actually apply once an == predicate is combined with the inequality/range predicates - and an id/category equality plus a date/numeric range is a very ordinary join shape, arguably more common than a pure non-equi join in practice. The underlying pathology is the same one #1641/#1658/#1659 already measured (up to ~77x for same-direction, ~7-11x for range joins, worse again under join_algorithm="regions") - it just isn't fixed for this dispatch tree yet.

Correctness

Same invariance argument as #1641/#1658/#1659 applies unchanged: _equi_range_join.py/_equi_uniq_join.py still process ge_gt before le_lt structurally regardless of which candidate occupies either slot, so picking independently per bound (or per same-direction anchor) can't produce wrong output here either - this is a missed-optimization gap, not a latent correctness risk.

Ask

Extend selectivity-aware selection to the _get_indices_equi.py dispatch tree:

• _equi_uniq_join.py / _equi_range_join.py: reuse [PERF] Selective bound selection for range joins #1663's _maybe_select_better_range_bounds (or hoist it somewhere both dispatch trees can call) for the le_lt/ge_gt case.
• _equi_not_range_join.py: reuse [PERF] Selectivity-aware anchor predicate selection for conditional_join #1658's _select_anchor for the le_or_ge case.

Both fixes already exist and are algorithm-agnostic at the point they're called (after null-stripping, before dispatch) - this should mostly be a matter of calling them from _get_indices_equi.py's own null-stripping point too, not new selection logic.

Related

• Sibling follow-up to [PERF] Optimize conditional_join anchor predicate selection #1641, alongside [PERF] Extend selectivity-aware anchor predicate selection to keep='all' #1657, [PERF] Selectivity-aware anchor selection for range joins (le_lt/ge_gt) #1659, [PERF] Predicate-order sensitivity in join_algorithm='regions' (_not_range_join_regions.py) #1660.
• Found during review of [PERF] Selective bound selection for range joins #1663.

[samukweku]

Scope correction

Read all three consumers directly before implementing. _equi_uniq_join.py doesn't actually need this fix: the == predicate already reduces to at most one candidate per left row via a hash indexer (`r_cols.get_indexer(l_cols)`), and every non-equi predicate - regardless of whether it's in `le_or_ge`, `le_lt`, or `ge_gt` - just gets flattened into one unordered `rest` list and applied as a plain boolean check against that single candidate (`dict.fromkeys(rest)`, no anchor/window narrowing at all). There's no "which one narrows first" question there to begin with, so selectivity-aware selection is a no-op for that path - not a correctness issue, just nothing to fix.

`_equi_not_range_join.py` and `_equi_range_join.py` both confirmed to need it - they directly destructure `mapping["le_or_ge"]`/`["le_lt"]`/`["ge_gt"]` the same first-picked way as the pre-#1658/#1663 non-equi paths, then narrow within each equi-group using that single picked candidate before applying the rest as post-filters.

Proceeding with the fix scoped to those two.