Repository navigation
Support serializing and executing explodes as table functions over an ExplodeValue - #4819
Merged
g31pranjal merged 16 commits intoNov 9, 2026
Merged
Conversation
g31pranjal
force-pushed
the
apple/g31pranjal/unnested/explode-value-serialization
branch
from
November 2, 2026 17:43
c8f9e16 to
c83eeb6
Compare
g31pranjal
changed the base branch from
main
to
apple/g31pranjal/push-in-comparand-join-through-fetch
November 3, 2026 15:35
g31pranjal
force-pushed
the
apple/g31pranjal/unnested/explode-value-serialization
branch
from
November 3, 2026 15:35
c83eeb6 to
c631a7f
Compare
Base automatically changed from
apple/g31pranjal/push-in-comparand-join-through-fetch
to
main
November 3, 2026 18:58
g31pranjal
marked this pull request as ready for review
November 3, 2026 22:22
g31pranjal
changed the base branch from
main
to
apple/g31pranjal/port-affected-core-tests-to-yamsql
November 5, 2026 00:25
g31pranjal
force-pushed
the
apple/g31pranjal/unnested/explode-value-serialization
branch
3 times, most recently
from
November 5, 2026 11:34
1492507 to
fee82ec
Compare
normen662
requested changes
Nov 5, 2026
Base automatically changed from
apple/g31pranjal/port-affected-core-tests-to-yamsql
to
main
November 5, 2026 14:39
g31pranjal
force-pushed
the
apple/g31pranjal/unnested/explode-value-serialization
branch
2 times, most recently
from
November 6, 2026 00:07
80ac3f9 to
30f79e7
Compare
g31pranjal
commented
Nov 9, 2026
normen662
approved these changes
Nov 9, 2026
… ExplodeValue Adds what a server needs to deserialize and execute plans in which explodes are table functions over an `ExplodeValue`, ahead of the planner producing them: - `ExplodeValue`, a streaming value producing a struct `(element)`, or `(element, ordinal)` if it is with ordinality, per element of an array, and its continuation. - `ArraySortValue`, a sorted array, which a sorted IN source explodes. - `InExplodeSource`, an IN source binding the structs of an explode. - Serialization of list literals in `LiteralValue`. - `RecordQueryInUnionPlan` reports the dynamic types of its sources, and `InComparandSource` tolerates comparisons without a comparand value. Nothing plans these yet.
An array sort compares its elements as Comparables, so reject element types whose representation is not, such as records, bytes and vectors, when it is created.
The derivations of a table function over an ExplodeValue are a struct of the first element of the collection (and an ordinal, if the explode has one), as for the explode plan before, rather than the first struct the explode flows. That keeps the correlations of the collection and the shape of the struct the explode flows.
The planner no longer builds explodes over literal lists or parameters, so the list serialization of literals, the literal and parameter accessors of the in-explode source, and the coercion of literal elements are not needed.
Every explode flows the ordinal now, so the flag no longer says whether there is one. It says whether the planner must keep the collection in its order, which is what keeps the ordinals meaningful.
They existed for the explodes of literal lists and parameters, which are gone. InExplodeSource overrides both methods itself, so every remaining InComparandSource has a comparand value.
…ection ExplodeValue gains sorted and reverse flags, and sorts the elements itself. An IN source over a sorted explode no longer has to recognise an ArraySortValue as the collection, which would need digging up once anything else wraps the sort, so ArraySortValue is removed.
…sh and explain of an in-explode source
g31pranjal
force-pushed
the
apple/g31pranjal/unnested/explode-value-serialization
branch
from
November 9, 2026 19:03
30f79e7 to
a7d806c
Compare
g31pranjal
deleted the
apple/g31pranjal/unnested/explode-value-serialization
branch
November 9, 2026 19:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds what a server needs to deserialize and execute plans in which explodes are table functions over an
ExplodeValue, ahead of the planner producing such plans in #4816. Shipping this first lets a server running this version execute plans and continuations produced by a server running #4816.Nothing plans these constructs yet, so plans are unchanged.
Changes
ExplodeValue: aStreamingValuethat produces an(element, ordinal)struct per element of an array, akin toUNNEST ... WITH ORDINALITY. It streams with anExplodeCursorContinuation, or evaluates eagerly (evalAsList).preserveOrder: the ordinals are observed, so the collection must not be reordered. Otherwise the planner may replace it by any permutation.sorted/reverse: the explode produces its elements in ascending or descending order and numbers the ordinals in that order. Only an explode that does not preserve its order can be sorted, and only elements with aComparablerepresentation.InExplodeSource: anInComparandSourcethat binds the structs of an explode, so the inner plan reads the element and ordinal as it would from the explode. It is sorted and reverse when its explode is.RecordQueryInUnionPlanreports the dynamic types of its IN sources.DerivationsProperty: a table function over an explode derives the struct it flows, built from a representative element of the collection.PExplodeValue,PInExplodeSourceandExplodeCursorContinuation.ExplodeValueTest(streaming, continuations, sorting, cardinalities, equality, explain, derivations, proto round trips) andInExplodeSourceTest(evaluation, flowed type, hashing and proto round trips).Compatibility
A server without this change cannot deserialize
PExplodeValueorPInExplodeSource, so it cannot execute plans or resume continuations that contain them. That is why this change has to be deployed before #4816 starts producing such plans.