Skip to content

Port the record query tests affected by ExplodeValue to yamsql - #4843

Merged
g31pranjal merged 8 commits into
mainfrom
apple/g31pranjal/port-affected-core-tests-to-yamsql
Nov 5, 2026
Merged

g31pranjal merged 8 commits into
mainfrom
apple/g31pranjal/port-affected-core-tests-to-yamsql

Conversation

@g31pranjal

@g31pranjal g31pranjal commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Ports 37 fdb-record-layer-core record query tests to the yamsql suite under yaml-tests/src/test/resources/cascades-query-tests/. A trimmed-down version of #4588 holding only these tests.

These are the tests whose IN source (a LiteralValue list or a named parameter) is no longer compatible with the Cascades run of the JUnit test after #4816, so their Cascades coverage moves to SQL. The JUnit tests stay in place, are marked with // SQL Port status: (#4843) …, and skip the Cascades run via Assumptions.assumeFalse(useCascadesPlanner) (FDBInQueryTest) or are restricted to the old planner.

What is in the PR

  • 17 yamsql files (84 queries) plus the shared includes/simple-record-100.yamsql, registered in CascadesQueryTests. Metrics are generated for every file.
  • MIGRATION.md (per-method state) and README.md, cut down to these three classes.
  • SQL Port status / SQL Port comments on all 37 JUnit tests.

How the IN source is ported

Core test uses Port
literal list (ListComparison), or ConstantObjectValue IN (…) literal list. AstNormalizer lifts every literal into a ConstantObjectValue, so both are the same SQL
named parameter (ParameterComparison, equalsParameter) parameter injection, in !! !in {[…]} !! / = !! 0 !!, in a separate test block with statement_type: prepared (named <file>-prepared)

Every query has its own test: nothing is shared between two tests, and each reverse / p / str_list / ordering variant is its own query. The remaining parameterizations (InAsOrUnionMode, omitPrimaryKeyInOrdering, replans, normalizeNestedFields, attemptFailedInJoinAsUnionMaxSize) are planner or old-planner settings with no SQL surface and collapse. LIMIT is not supported in SQL, so testInWithLimit pages with maxRows: 3. Where a variant needs its own schema (dropNumValue3Index), it has its own file.

The 37 tests

qN is the position among the executed queries in the file; the -prepared block follows the literal one.

FDBInQueryTest (31)

Test IN source in core yamsql (queries) State
testInQueryNoIndex literal (0, 2), residual filter in-query q1 ported
testInQueryNoIndexWithParameter parameter valuesThree, residual filter in-query q10 (prepared) ported
testInQueryIndex literal (1, 2, 4), forward and reverse in-query q2-q3 ported
testInQueryIndexSorted literal (1, 4, 2) (out of order), forward and reverse in-query q4-q5 ported
testInQueryCoveringIndex literal (1, 2, 4), forward and reverse, covering in-query q6-q7 ported
testInQueryIndexSortedDifferently literal (1, 4, 2), sorted on another column, forward and reverse in-query q8-q9 ported
testInQueryParameter parameter valueThrees, no sort in-query q11 (prepared) ported
testInQuerySortedByParameter parameter valueThrees, forward and reverse in-query q12-q13 (prepared) ported
testInQueryParameter2 parameter valueThrees + equalsParameter("p"), p = 0..4, forward and reverse in-query-multi-index q1-q10 (prepared) ported
testInQueryParameter2UsingCoveringIndexScans same, covering in-query-multi-index q11-q20 (prepared) ported
testInQueryParameterBad parameter bound to a non-list none not portable: a SQL IN list is always a list
testListInQuerySortedWithExtraUnorderedColumns literal (1, 2, 4), forward and reverse in-query-sorted-extra-unordered-columns q1-q2 ported
testParameterizedInQuerySortedWithExtraUnorderedColumns parameter list + equalsParameter, forward and reverse in-query-sorted-extra-unordered-columns q3-q4 (prepared) ported
inQueryWithSortBySecondFieldOfCompoundIndex literal (1, 4, 2) in-query-compound-index q1 ported
inQueryWithSortAndRangePredicateOnSecondFieldOfCompoundIndex literal (1, 4, 2) + range in-query-compound-index q2 ported
inQueryWithSortAndParameter parameter inList + range in-query-compound-index q3 (prepared) ported
testInWithNesting literal, 4 strings, nested field in-query-header q1 ported
testMultipleInQueryIndex two literals (rec_no, path) in-query-header q2 ported
testMultipleInQueryIndexSorted two literals, index order requested in-query-header q3 ported
testInWithLimit literal ls, returned-row limit in-query-header q4 (maxRows: 3) ported
testInWithContinuation literal ls, continuations in-query-header q5 (maxRows: 1) ported
testInQueryOr literal (903, 905, 901) OR a range in-query-or q1 ported
testInQueryOrDifferentCondition literal (2, 0) OR a range, residual filter in-query-or q2 ported
testInQueryOrMultipleIndexes literal (1, 3) OR another index in-query-or q3 ported
testInUnionWithNonSargable literal (1, 3, 5, 7, 9), residual filter in-query-or q4 ported
testInQueryEmptyList empty literal list in-query-or q5 (prepared; in () does not parse) ported
testInQueryOrOverlap two overlapping literals in-query-or-overlap q1 ported
testInQueryOrCompound literal (1, 3) in OR in AND in-query-or-compound q1 ported
testInUnionWithSomeSargable parameters nv2_list, str_list in-query-union-sargable q1-q3 (prepared) ported
testInUnionWithIntersectionOnTwoPredicates parameters nv2_list, str_list; dropNumValue3Index changes the schema in-query-union-intersection q1-q3, in-query-union-intersection-no-nv3 q1-q3 (prepared) ported
enumIn literal list of enum values none deferred: needs CREATE TYPE AS ENUM

FDBPermutedMinMaxQueryTest (4)

Each is parameterized over InComparisonCase: byLiteral and byConstantObjectValue are the literal block, byParameter is the -prepared block.

Test yamsql (queries) State
selectMaxWithInOrderByMax permuted-min-max q1-q3 literal, q4-q6 prepared ported
testMaxWithInAndDupes permuted-min-max-num-value-2 q1 literal, q2 prepared ported
testSortedMaxWithInOnRepeater permuted-min-max-repeater q1-q2 literal, q3-q4 prepared ported
testMaxUniqueByStr2And3WithDifferentOrderingKeys permuted-min-max-ordering-keys q1-q4 literal, q5-q8 prepared (one per ordering key) ported

FDBSimpleQueryGraphTest (2)

Test IN source in core yamsql (queries) State
testSimpleExistentialPredicateOnSimpleIndex parameter t; inComparison flips EQUALS / IN simple-query-graph-tag-index q1 (=), q2 (IN), prepared ported
testEqualityAndSimpleExistentialPredicate parameters on name and t simple-query-graph-name-tag-index q1 (prepared) ported

Totals: 35 ported, 1 not portable (testInQueryParameterBad), 1 deferred (enumIn).

Notes

  • Expectations are generated against current main, so plans carry the covering-scan-plus-fetch shape from Push in-joins over comparands through fetches #4830.
  • Plan-shape divergence from the JUnit plans: for the extra-columns tests the in-union comparison key has three values instead of four, and SQL always emits a MAP above a covering scan. This is documented in the header of in-query-sorted-extra-unordered-columns.yamsql.

@g31pranjal g31pranjal added the testing improvement Change that improves our testing label Nov 4, 2026
@g31pranjal
g31pranjal force-pushed the apple/g31pranjal/port-affected-core-tests-to-yamsql branch from e4b7f5b to 099f1da Compare November 4, 2026 22:16
@g31pranjal
g31pranjal force-pushed the apple/g31pranjal/port-affected-core-tests-to-yamsql branch from e203d65 to f038db0 Compare November 4, 2026 23:56
@g31pranjal
g31pranjal marked this pull request as ready for review November 5, 2026 00:13
@g31pranjal
g31pranjal requested a review from normen662 November 5, 2026 00:13
…des assertions

The tests ported to yamsql are restricted with @DualPlannerTest(planner = OLD) instead of
assuming the old planner in their bodies. DualPlannerExtension now allows such a parameterized
test to provide no contexts of its own, since the parameterized extension already runs it with
the old planner.
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

📊 Metrics Diff Analysis Report

Summary

  • New queries: 58
  • Dropped queries: 0
  • Plan changed + metrics changed: 0
  • Plan unchanged + metrics changed: 0
ℹ️ About this analysis

This automated analysis compares query planner metrics between the base branch and this PR. It categorizes changes into:

  • New queries: Queries added in this PR
  • Dropped queries: Queries removed in this PR. These should be reviewed to ensure we are not losing coverage.
  • Plan changed + metrics changed: The query plan has changed along with planner metrics.
  • Metrics only changed: Same plan but different metrics

The last category in particular may indicate planner regressions that should be investigated.

New Queries

Count of new queries by file:

  • yaml-tests/src/test/resources/cascades-query-tests/in-query-compound-index.metrics.yaml: 3
  • yaml-tests/src/test/resources/cascades-query-tests/in-query-header.metrics.yaml: 4
  • yaml-tests/src/test/resources/cascades-query-tests/in-query-multi-index.metrics.yaml: 4
  • yaml-tests/src/test/resources/cascades-query-tests/in-query-or-compound.metrics.yaml: 1
  • yaml-tests/src/test/resources/cascades-query-tests/in-query-or-overlap.metrics.yaml: 1
  • yaml-tests/src/test/resources/cascades-query-tests/in-query-or.metrics.yaml: 5
  • yaml-tests/src/test/resources/cascades-query-tests/in-query-sorted-extra-unordered-columns.metrics.yaml: 4
  • yaml-tests/src/test/resources/cascades-query-tests/in-query-union-intersection-no-nv3.metrics.yaml: 1
  • yaml-tests/src/test/resources/cascades-query-tests/in-query-union-intersection.metrics.yaml: 1
  • yaml-tests/src/test/resources/cascades-query-tests/in-query-union-sargable.metrics.yaml: 1
  • yaml-tests/src/test/resources/cascades-query-tests/in-query.metrics.yaml: 13
  • yaml-tests/src/test/resources/cascades-query-tests/permuted-min-max-num-value-2.metrics.yaml: 2
  • yaml-tests/src/test/resources/cascades-query-tests/permuted-min-max-ordering-keys.metrics.yaml: 8
  • yaml-tests/src/test/resources/cascades-query-tests/permuted-min-max-repeater.metrics.yaml: 3
  • yaml-tests/src/test/resources/cascades-query-tests/permuted-min-max.metrics.yaml: 4
  • yaml-tests/src/test/resources/cascades-query-tests/simple-query-graph-name-tag-index.metrics.yaml: 1
  • yaml-tests/src/test/resources/cascades-query-tests/simple-query-graph-tag-index.metrics.yaml: 2

@g31pranjal
g31pranjal merged commit 43a9a0e into main Nov 5, 2026
15 checks passed
@g31pranjal
g31pranjal deleted the apple/g31pranjal/port-affected-core-tests-to-yamsql branch November 5, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing improvement Change that improves our testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants