Problem
Several source profilers expose a variant prompt that segregates the SQL pipeline by development type or optional feature availability instead of discovering it automatically, causing a friction-heavy user experience and duplicated pipeline logic:
- Redshift (
provisioned, provisioned_multi_az, serverless)
- Teradata (
core, pdcr)
We should discover this rather than exposing it.
Proposed Solution
For each affected source, collapse its variants into a single, self-describing pipeline. Move deployment/feature identification downstream to the analysis/model layer based on the extracted data artifacts, relying on the best-effort/partial-run contract from #2541 to tolerate queries that are only valid for a subset of environments.
High-Level Acceptance Criteria
Problem
Several source profilers expose a variant prompt that segregates the SQL pipeline by development type or optional feature availability instead of discovering it automatically, causing a friction-heavy user experience and duplicated pipeline logic:
provisioned,provisioned_multi_az,serverless)core,pdcr)We should discover this rather than exposing it.
Proposed Solution
For each affected source, collapse its variants into a single, self-describing pipeline. Move deployment/feature identification downstream to the analysis/model layer based on the extracted data artifacts, relying on the best-effort/partial-run contract from #2541 to tolerate queries that are only valid for a subset of environments.
High-Level Acceptance Criteria
variantselection #2542)core/pdcrchoice or prompting the user.optional, mutually exclusive or additive SQL queries (Decide how the profilerpipelineshould handle partial / best-effort runs. #2541).SOURCE_SYSTEM_VARIANTSno longer carries entries for fully-collapsed sources.