Skip to content

feat: reserve GROUPS in the native window-frame protocol#4878

Closed
manuzhang wants to merge 2 commits into
apache:mainfrom
manuzhang:codex/issue-4836-groups-window-frame
Closed

feat: reserve GROUPS in the native window-frame protocol#4878
manuzhang wants to merge 2 commits into
apache:mainfrom
manuzhang:codex/issue-4836-groups-window-frame

Conversation

@manuzhang

@manuzhang manuzhang commented Jul 10, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

Related to #4836. This PR does not close it because supported Spark versions cannot yet construct GROUPS window frames.

Rationale for this change

DataFusion supports GROUPS frames, but Apache Spark versions supported by Comet expose only ROWS and RANGE in both the SQL parser and the sealed Catalyst FrameType. This PR adds forward-compatible protocol and native-planner groundwork without claiming that Spark SQL can originate GROUPS frames today.

What changes are included in this PR?

  • Add a reserved Groups value to the window-frame proto enum.
  • Convert incoming GROUPS protocol frames to DataFusion WindowFrameUnits::Groups.
  • Use unsigned physical offsets for GROUPS, matching the protocol convention used by ROWS.
  • Keep Spark window-frame serialization unchanged until Spark exposes a reachable GROUPS representation.
  • Add focused native tests for protocol conversion of GROUPS offsets and default unbounded bounds.

How are these changes tested?

  • cargo test -p datafusion-comet test_groups_window_frame --lib
  • ./mvnw -Pjdk17 -DskipTests compile
  • The follow-up scope correction was independently reviewed; no additional tests were run.

manuzhang and others added 2 commits July 10, 2026 08:38
Add GROUPS to the window frame proto and serialize physical offsets like ROWS. Map the native frame to DataFusion WindowFrameUnits::Groups and cover the conversion in unit tests.

Co-authored-by: Codex <codex@openai.com>
Spark's sealed FrameType and SQL grammar cannot produce GROUPS frames. Remove the unreachable Spark serialization path and mark the proto/native conversion as forward-compatible groundwork.

Co-authored-by: Codex <codex@openai.com>
@manuzhang manuzhang changed the title feat: support GROUPS window frame units feat: reserve GROUPS in the native window-frame protocol Jul 10, 2026
@manuzhang manuzhang closed this Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant