feat: reserve GROUPS in the native window-frame protocol#4878
Closed
manuzhang wants to merge 2 commits into
Closed
feat: reserve GROUPS in the native window-frame protocol#4878manuzhang wants to merge 2 commits into
manuzhang wants to merge 2 commits into
Conversation
Add GROUPS to the window frame proto and serialize physical offsets like ROWS. Map the native frame to DataFusion WindowFrameUnits::Groups and cover the conversion in unit tests. Co-authored-by: Codex <codex@openai.com>
Spark's sealed FrameType and SQL grammar cannot produce GROUPS frames. Remove the unreachable Spark serialization path and mark the proto/native conversion as forward-compatible groundwork. Co-authored-by: Codex <codex@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Related to #4836. This PR does not close it because supported Spark versions cannot yet construct GROUPS window frames.
Rationale for this change
DataFusion supports GROUPS frames, but Apache Spark versions supported by Comet expose only ROWS and RANGE in both the SQL parser and the sealed Catalyst FrameType. This PR adds forward-compatible protocol and native-planner groundwork without claiming that Spark SQL can originate GROUPS frames today.
What changes are included in this PR?
How are these changes tested?