Skip to content

Reference table with no satellite assignments generates an empty ref_satellites instead of falling back to the hub's reference satellites #187

Description

@obause

Summary

ExportBuilder._build_reference_tables derives a reference table's satellites purely from its ReferenceTableSatelliteAssignment rows. When a ReferenceTable has no assignment rows, the generated dbt model renders ref_satellites: with nothing under it, so the resulting view exposes the reference hub alone — while its companion .yml still claims it combines the hub "with satellites".

This contradicts the "empty means all" semantics the model itself documents, and it silently produces a degenerate model rather than an error.

Found while building the reference-table surfaces in Turbovault Studio. Reproduced against engine 0.16.3.

Why "empty" should mean "all"

Reference satellites belong to the reference hub (Satellite.parent_hub, satellite_type='reference'). The assignment rows exist to narrow which satellites/columns a reference table exposes — that is exactly how the column-level fields are already documented in engine/models/reference_table.py:

include_columns = models.ManyToManyField(
    SatelliteColumn,
    ...
    help_text="Specific columns to include (if empty, includes all except excluded)",
)

An empty assignment set should follow the same rule: use all reference satellites of reference_hub. A reference table pointing at a hub that has reference satellites is a complete definition on its own.

Reproduction

Minimal metadata:

  • reference hub country_ref_h (hub_type='reference', hashkey hk_country)
  • two reference satellites on that hub (satellite_type='reference')
  • ReferenceTable(reference_table_physical_name='country_ref', reference_hub=country_ref_h) with no ReferenceTableSatelliteAssignment rows

Generate dbt for that single entity (GenerationOptions.entity_selection.only_entities=[EntityRef(type='reference_table', name='country_ref')], dry_run=True, return_content=True).

Actual — country_ref.sql

{{ config(
    materialized='view',
    tags=['reference_table', 'raw_vault']
) }}

{%- set yaml_metadata -%}
ref_hub: country_ref_h
ref_satellites:
{%- endset -%}

{{ datavault4dbt.ref_table(yaml_metadata=yaml_metadata) }}

ref_satellites is an empty key — the macro receives no satellites at all.

Actual — country_ref.yml

models:
  - name: country_ref
    description: "Reference table combining country_ref_h with satellites"

The description promises satellites the model does not contain.

Expected

ref_satellites lists every reference satellite of reference_hub (each still subject to per-assignment include/exclude when assignment rows do exist).

Where

  • engine/services/export/builder.pyExportBuilder._build_reference_tables (~L1132): the for assignment in ref_table.satellite_assignments.all() loop is the only source of satellite_assignments, so an empty relation yields satellites=[].
  • engine/services/generation/templates/sql/reference_table.sql.j2[% for sat in satellites %] simply renders nothing.
  • engine/services/generation/templates/yaml/reference_table.yml.j2 → hardcodes "combining … with satellites" regardless.

Suggested fix

In _build_reference_tables, when ref_table.satellite_assignments is empty, fall back to the reference hub's reference satellites:

assignments = list(ref_table.satellite_assignments.all())
if not assignments:
    satellite_assignments = [
        ReferenceTableSatelliteAssignment(
            satellite_name=sat.satellite_physical_name,
            include_columns=[],
            exclude_columns=[],
        )
        for sat in Satellite.objects.filter(
            parent_hub=ref_table.reference_hub,
            satellite_type=Satellite.SatelliteType.REFERENCE,
        ).order_by("satellite_physical_name")
    ]

(Add the hub's reference satellites to the existing prefetch_related so this stays a fixed number of queries.)

Worth deciding alongside it:

  • Should a reference hub with no reference satellites at all be a validation issue (Issue/Code) rather than silently emitting an empty ref_satellites? Today it is neither.
  • The .yml description could drop or condition the "with satellites" wording.

Related: importer only ever assigns one satellite

engine/services/imports/executor.py_upsert_reference_table creates an assignment only when d.referenced_satellite_name is set, and the comment notes the IR supports a single satellite per reference table:

# Satellite assignment (single satellite per ref table in our IR).
if d.referenced_satellite_name:
    ...

So imported reference tables commonly arrive with zero assignments (or at most one, even when the hub has several) — which makes this the common case, not an edge case. If the fallback above lands, imported tables behave correctly by default; separately, the import IR may want to carry multiple referenced satellites.

Impact

  • Any reference table without explicit assignments generates a dbt model that resolves to the hub only — likely a broken or empty view at dbt run, depending on how datavault4dbt.ref_table handles an absent ref_satellites.
  • Most visible for imported projects, per the note above.
  • Turbovault Studio treats an empty assignment set as "all satellites" (matching the documented semantics) and therefore reports these tables as fine, so nothing surfaces the problem to the user until generation output is inspected.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions