You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some current Rust kernels use compensated summation (e.g. Kahan/Neumaier) for
floats. A naive prefix-sum-and-subtract rewrite can change rounding or
cancellation behavior versus the current output, so this isn't a drop-in
replacement the way it is for integers.
Proposed scope
Define and test an explicit numerical-accuracy contract for float prefix
sums (e.g. acceptable ULP/relative-error bounds vs. the current
compensated-summation kernels), or decide compensated summation must be
preserved and design a prefix-sum variant that keeps it (e.g. a
compensated prefix sum with a compensated subtraction step).
Brief description
Follow-up to #1648, which restricts the prefix-sum rewrite of the forward
range-sum kernels used by
join_aggto integer dtypes.This issue tracks extending the same optimization to
float32/float64,which is deliberately out of scope for #1648.
Relevant Rust implementations:
src/aggs/sum/sum_starts.rssrc/aggs/sum/sum_ends.rssrc/aggs/sum/sum_starts_ends.rsRelevant Python dispatch:
janitor/functions/_conditional_join/_agg_functions.pyjanitor/functions/_conditional_join/_get_join_aggs.pyWhy this is separate from #1648
Some current Rust kernels use compensated summation (e.g. Kahan/Neumaier) for
floats. A naive prefix-sum-and-subtract rewrite can change rounding or
cancellation behavior versus the current output, so this isn't a drop-in
replacement the way it is for integers.
Proposed scope
sums (e.g. acceptable ULP/relative-error bounds vs. the current
compensated-summation kernels), or decide compensated summation must be
preserved and design a prefix-sum variant that keeps it (e.g. a
compensated prefix sum with a compensated subtraction step).
float32/float64covering suffix, prefix, and arbitrary
[start:end)ranges, mirroring theinteger implementation landed in [PERF] Use prefix sums for conditional_join range aggregations (integer dtypes) #1648.
empty ranges, and extension-array reconstruction.
Acceptance criteria
numerical-accuracy contract, across null placements, empty inputs, and
range shapes.
improvement.
number, and
@samukweku.Related aggregation work:
sum, this issue's prerequisite/sibling)min/maxalgorithms.prodalgorithms and numerical contracts.aggregation kernels.