fix: reject non-sentinel negative starts in compare_start_end's width precheck - #52
Merged
Merged
Conversation
5 tasks
… precheck compare_start_end_*'s expected_matches_width precheck filtered rows with `**s != -1 && **e != -1 && **s < **e` before casting both bounds to usize and subtracting. That only rejects the exact -1 sentinel -- any other malformed negative start (e.g. -2) that still satisfies `start < end` in i64 space (starts=[-2], ends=[1]) slipped through, then underflowed `(1usize) - ((-2i64) as usize)` before ensure_tape_width ever ran: a panic in debug builds, a silently bogus width in release. Requiring both bounds to already be non-negative (`**s >= 0 && **e >= 0`) closes the gap without changing which rows compare_start_end_core itself treats as contributing candidates: the core's own cast-then- range-index already naturally produces zero ticks for a row like this (the resulting usize range is inverted, which Rust's Range iterator treats as empty rather than panicking), so tightening the precheck to agree with that doesn't reject anything a real caller would send. Found during review of #51 (issue #24's comparison-operator PR); not introduced there -- this precheck predates it. Same bug class as #38 (caller-supplied bounds needing validation before an unchecked cast), folded in as a standalone fix once #45 (which addressed #38's other instances) had already merged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V3KAbkp6JV96KYXNc6EJmN
The previous commit fixed the *wrapper*'s width precheck (a separate computation used only for ensure_tape_width), but compare_start_end_core has its own independent row-acceptance check that still let a malformed row through: `*start == -1 || *end == -1 || *start >= *end` only rejects the exact -1 sentinel, not any negative value. starts=[-3], ends=[-2]: -3 isn't -1, -2 isn't -1, and -3 >= -2 is false (-3 < -2), so the row is accepted. Both bounds then cast to i64::MAX-adjacent usize values that are *still correctly ordered* relative to each other (start_ < end_ survives the cast, since twos-complement wraparound preserves relative order for two negative numbers of similar magnitude), so the loop walks right/matches at an enormous offset instead of recognizing the range as invalid -- confirmed with a reproduction that panics with "index 18446744073709551613 is out of bounds for array of shape [1]" before this fix. Requiring `start >= 0` (replacing the `start == -1`-only check) closes this: once a row is accepted, start >= 0 and start < end together prove end > 0 too, so no separate end < 0 check is needed to reach the same guarantee the previous commit's wrapper-level fix already established independently. Found by adversarial review of this PR (P1). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V3KAbkp6JV96KYXNc6EJmN
samukweku
force-pushed
the
fix-compare-width-precheck-underflow
branch
from
August 24, 2026 07:49
2175ce9 to
6ea0aaa
Compare
This was referenced Aug 24, 2026
samukweku
added a commit
that referenced
this pull request
Aug 24, 2026
compare_start_end_core validated negative and inverted start/end ranges (PR #52) but never checked a positive `end` against `right.len()`, so a call with `ends[i] > right.len()` walked `right[nn]` out of bounds and panicked instead of raising a normal ValueError -- reported in issue #53. Adds the missing `*end as usize > right_len` check, in both the core loop and the pyfunction wrapper's matches-tape width precheck (which must reject the same rows, or it demands a needlessly large tape for rows the core now silently skips). Deliberately kept as a plain comparison rather than routed through the crate's existing `checked_range` helper: benchmarking `compare_start_end` (a hot, per-row comparison kernel) showed checked_range's Option-returning usize::try_from calls cost ~25-30% more wall time here than the equivalent plain comparisons, consistent with compare::op's own doc comment on indirection being measurably costly in this same hot path. No regression in the final version (benches/kernels.rs's compare_start_end group, n=100 and n=100,000, before vs after). Adds regression tests for end == right.len() (valid boundary), end == right.len() + 1 (issue #53's exact repro, at both the core and Python wrapper level), and confirms via a built wheel that the previously-panicking call now returns cleanly.
4 tasks
samukweku
added a commit
that referenced
this pull request
Aug 24, 2026
Part of #56: compare_start_end_core (comp.rs) is not the only file in src/compare/ with unguarded start/end range indexing -- 13 sibling files share the pattern, none with any bound validation at all (not even the negative-bound check #52 added to comp.rs). This fixes the first 5: - comp_ends.rs / comp_starts.rs ("matches already exist"): these take a matches/counts tape already sized by the caller, so an invalid row can't be silently skipped the way comp.rs skips its own self-owned tape -- doing so would desynchronize the shared tape index for every row after it. Reject the whole call with a PyValueError up front instead. - comp_first_ends.rs / comp_first_starts.rs / comp_first.rs ("matches does not exist yet"): these own their output array end to end, so they follow comp.rs's precedent exactly -- an invalid row silently contributes zero ticks, computed consistently between the output-size precomputation and the main loop. comp_first_starts.rs needed the least: its start_..end loop was already safe for a bad start (an empty Range, not a panic) since end is always right.len(); only the length-sum precomputation (end - start_) could underflow. All bound checks are plain comparisons, not the crate's checked_range helper -- PR #55 measured checked_range costing ~25-30% more wall time than plain comparisons in this same per-row-comparison hot path, so that lesson is applied here preemptively rather than rediscovered per file. No dedicated benchmark exists for these 5 (unlike comp.rs, they have no extracted _core function benches/kernels.rs can call without a Python interpreter), so this is a qualitative call, not a measured one. 12 new tests (Python-wrapper level, matching each file's exported function directly), covering: the out-of-bounds repro, a negative-bound variant, and the valid boundary (start/end == right.len()) staying accepted. Verified via a built wheel that every one of the 5 previously- panicking calls now returns cleanly. Remaining files under #56: comp_ne.rs and its 5 siblings (comp_ne_1st, comp_ne_ends, comp_ne_ends_1st, comp_ne_starts, comp_ne_starts_1st), plus comp_posns.rs/comp_posns_ne.rs (double indirection through `positions`, needs its own look at what bound `nn`/`indexer` should each check against).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
compare_start_end_*'sexpected_matches_widthprecheck (insrc/compare/comp.rs) filtered candidate rows with**s != -1 && **e != -1 && **s < **ebefore casting both bounds tousizeand subtracting. That only rejects the exact-1sentinel — any other malformed negativestart(e.g.-2) that still satisfiesstart < endini64space (starts=[-2], ends=[1]) slipped through, then underflowed(1usize) - ((-2i64) as usize)beforeensure_tape_widthever ran: a panic in debug builds, a silently bogus width in release.Fix (commit 1): require both bounds to already be non-negative (
**s >= 0 && **e >= 0) in the wrapper's precheck instead of only excluding-1.Adversarial review of the first commit found a second, deeper instance of the same root cause (P1):
compare_start_end_coreitself — not just the wrapper's precheck — has its own independent row-acceptance check,*start == -1 || *end == -1 || *start >= *end, with the identical gap.starts=[-3], ends=[-2]isn't caught by the wrapper's now-fixed precheck's rejection (it correctly contributes 0 to the width sum), but the wrapper still calls into the core regardless, and the core's own check accepts the row (-3 >= -2is false), casts both bounds to huge-but-still-orderedusizevalues, and panics indexingright/matchesfar out of bounds. Reproduced directly against the core (index 18446744073709551613 is out of bounds for array of shape [1]) before the fix.Fix (commit 2): the same "require non-negative before cast" fix, applied to the core's own check (
*start < 0 || *end == -1 || *start >= *end— onlystartneeds the explicit non-negativity check, sincestart >= 0 && start < endtogether proveend > 0too).Neither fix changes which rows a well-formed caller's
start/endpairs produce — both only tighten rejection of malformed input that was already reachable, just not exercised by any prior test.Test plan
cargo test --no-default-features-- 169 passed, 0 failed:non_sentinel_negative_start_does_not_underflow_the_width_precheck-- exercises the wrapper-level fix through the actualcompare_start_end_int64pyfunction via an embedded Python interpreterboth_bounds_negative_but_start_less_than_end_contributes_nothing_not_a_panic-- exercises the core-level fix directly againstcompare_start_end_corewith the review's exact counterexample (starts=[-3], ends=[-2]), confirmed to panic before the fix and pass cleanly aftercargo clippy --all-targets --all-features -- -D warnings-- cleancargo fmt --check-- cleanmaturin develop --releaseand ran the review's exact counterexample end-to-end throughcompare_start_end_int64in Python: returns([0], [0], 0)cleanly instead of panicking