Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
45f67cf
feat: add YML and MD documentation for all macros (#374)
pkumar-data Jul 11, 2026
1289192
fix(docs): correct ref_hub usage example to use ref_keys instead of b…
pkumar-data Jul 13, 2026
38199a4
fix(docs): correct bigquery adapter YML macro names to default__ prefix
pkumar-data Jul 13, 2026
2cd2fb2
refactor: consolidate adapter macro docs into parent YML files
pkumar-data Jul 14, 2026
f43dabe
docs(macros): add yaml_metadata context and fix stage.yml argument types
pkumar-data Jul 14, 2026
21d1299
Potential fix for pull request finding
pkumar-data Jul 14, 2026
07a43f6
fix(macros): change yaml_metadata type from dict to string in all YML…
pkumar-data Jul 14, 2026
236a0b6
fix(docs): correct ledts_alias description and hub.md metadata example
pkumar-data Jul 14, 2026
954178b
docs(macros): add YML documentation for backend macros
pkumar-data Jul 15, 2026
2753d15
fix(docs): use double quotes around SQL expression examples in stage.yml
pkumar-data Jul 15, 2026
e3ee9e4
fix(docs): correct empty_is_false description in helpers.yml
pkumar-data Jul 15, 2026
5944c2a
fix(docs): remove non-existent default__control_snap_v1 and clarify c…
pkumar-data Jul 15, 2026
291d43f
fix(docs): clarify limit_rows empty-string condition in supporting.yml
pkumar-data Jul 15, 2026
ab4f0a6
Merge branch 'main' into feat/374-macro-yml-documentation
tkiehn Jul 17, 2026
61182f0
fix(docs): use valid, bracketed types for validate_macro_args and doc…
pkumar-data Aug 3, 2026
f59e3b1
fix(docs): address Copilot review on prejoined_columns docs
pkumar-data Aug 3, 2026
5ca110f
fix(docs): correct alias_all columns type per Copilot review
pkumar-data Aug 3, 2026
ce3c47f
Merge branch 'main' into feat/374-macro-yml-documentation
tkirschke Aug 4, 2026
c86b248
fix(docs): wrap live Jinja examples in {% raw %} to fix manifest parsing
pkumar-data Aug 4, 2026
a59f2a8
Merge branch 'feat/374-macro-yml-documentation' of github.com:Scalefr…
pkumar-data Aug 4, 2026
810c83d
Merge branch 'main' into feat/374-macro-yml-documentation
tkirschke Aug 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions macros/hooks/clean_up_pit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{% docs clean_up_pit %}

## Clean Up PIT (Post-Hook)

This macro should be used as a post-hook for each PIT table whenever a logarithmic snapshot logic is used.
The macro deletes all records in a PIT table that are no longer active according to the snapshot view.
Deletion is safe here because no actual data is deleted — only pointers to satellite entries.

### Usage as a post-hook

```jinja
{% raw %}
{{ config(
post_hook="{{ datavault4dbt.clean_up_pit('control_snap_v1') }}"
) }}
{% endraw %}
```

### With custom column names

```jinja
{% raw %}
{{ config(
post_hook="{{ datavault4dbt.clean_up_pit(
snapshot_relation='control_snap_v1',
snapshot_trigger_column='is_active',
sdts='sdts'
) }}"
) }}
{% endraw %}
```

{% enddocs %}
21 changes: 21 additions & 0 deletions macros/hooks/clean_up_pit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
version: 2

macros:
- name: clean_up_pit
description: '{{ doc("clean_up_pit") }}'
arguments:
- name: snapshot_relation
type: string
description: >
The name of the dbt model that creates the snapshot table / view that has the logarithmic snapshot
logic applied.
- name: snapshot_trigger_column
type: string
description: >
The name of the boolean column inside the snapshot tables that activates/deactivates single snapshots.
If not set, the name defined inside the global variable 'datavault4dbt.snapshot_trigger_column' is used.
- name: sdts
type: string
description: >
The name of the snapshot date timestamp column inside the snapshot table. If not set, the name defined
inside the global variable 'datavault4dbt.sdts_alias' is used.
82 changes: 82 additions & 0 deletions macros/internal/helpers/helpers.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
version: 2

macros:
- name: is_list
description: >
Internal helper. Returns true if the given object is a list (iterable, not a string,
not a mapping). Used throughout the package to distinguish list inputs from scalars.
arguments:
- name: obj
type: any
description: The object to check.
- name: empty_is_false
type: bool
description: If true, returns false when obj is None/undefined (empty lists still return true). Default is false.

- name: is_nothing
description: >
Internal helper. Returns true if the given object is None, undefined, or falsy.
arguments:
- name: obj
type: any
description: The object to check.

- name: is_something
description: >
Internal helper. Returns true if the given object is not None, is defined, and is truthy.
Inverse of is_nothing.
arguments:
- name: obj
type: any
description: The object to check.

- name: is_expression
description: >
Internal helper. Returns true if the given string looks like a SQL expression — i.e. it is
single-quoted, contains parentheses, uses :: (cast syntax) or || (concatenation), or is a
recognized SQL keyword on Trino.
arguments:
- name: obj
type: any
description: The object to check.

- name: is_attribute
description: >
Internal helper. Returns true if the given string is a plain column name reference —
i.e. not a static string literal (no leading '!') and not a SQL expression.
Raises a compiler error if obj is null or not a string.
arguments:
- name: obj
type: any
description: The object to check. Must be a non-null string.

- name: is_something_or_false
description: >
Internal helper. Returns true if the given object is not None and is defined,
even if it evaluates to false (e.g. 0 or empty string).
arguments:
- name: obj
type: any
description: The object to check.

- name: prepend_generated_by
description: >
Internal helper. Outputs an ASCII art banner identifying SQL as generated by datavault4dbt
by Scalefree International GmbH. Prepended as a comment block in generated SQL files.

- name: replace_standard
description: >
Internal helper. Returns the input_variable if it is not None and not empty. Otherwise falls
back to the value of the given dbt project variable (global_variable), using default_value
if the variable is not set.
arguments:
- name: input_variable
type: any
description: The value passed in by the caller. If None or empty string, the fallback is used.
- name: global_variable
type: string
description: >
The name of the dbt project variable to use as fallback (resolved via var()).
- name: default_value
type: any
description: The value returned if both input_variable and global_variable are unset.
181 changes: 179 additions & 2 deletions macros/internal/metadata_processing/metadata_processing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,185 @@ macros:
type: string
description: The forwarded parameter of the top-level macro. This is used, if the yaml-metadata is none.
- name: required
type: boolean
type: bool
description: Whether this parameter is required for the top-level macro. Default is False.
- name: documentation
type: string
description: A string that holds documentation of this parameter.
description: A string that holds documentation of this parameter.

- name: alias
description: >
Internal helper. Renders a single column expression as either 'source_column AS alias' or
a plain column reference with an optional table prefix. Dispatches to default__alias.
arguments:
- name: alias_config
type: any
description: >
Either a dict with 'source_column' and 'alias' keys (produces 'source_column AS alias'),
or a plain column name string (produces the column with optional prefix).
- name: prefix
type: string
description: Optional table prefix prepended to the column reference (e.g. 'src').

- name: default__alias
description: Default implementation of alias. Accepts the same arguments as the parent alias macro.

- name: alias_all
description: >
Internal helper. Iterates over a list of column definitions and calls alias() on each one,
returning a comma-separated list of column expressions. Also accepts a single column definition
directly, in which case it is forwarded to alias() as-is. Dispatches to default__alias_all.
arguments:
- name: columns
type: any
description: >
A list of column definitions, or a single column definition. Each definition is either a plain
column name string or a dict with 'source_column' and 'alias' keys, passed individually to alias().
- name: prefix
type: string
description: Optional table prefix forwarded to each alias() call.

- name: default__alias_all
description: Default implementation of alias_all. Accepts the same arguments as the parent alias_all macro.

- name: as_constant
description: >
Internal helper. Renders a column reference as a SQL literal or expression.
A leading '!' prefix strips the '!' and returns the value as a SQL string literal.
A SQL expression (quoted, contains parentheses, cast or concatenation operators) is passed
through unchanged. A plain identifier is wrapped in the adapter-appropriate escape characters.
Dispatches to default__as_constant.
arguments:
- name: column_str
type: string
description: The column string to evaluate. Leading '!' indicates a static literal.

- name: default__as_constant
description: Default implementation of as_constant. Accepts the same arguments as the parent as_constant macro.

- name: check_required_parameters
description: >
Internal helper. Inspects the kwargs dict of a calling macro and raises a dbt compiler
error listing every parameter whose value is None. Used in front-end macros to enforce
that all required arguments have been supplied before code generation begins.

- name: concat_ws
description: >
Internal helper. Generates a SQL concatenation expression for a list of column strings
using an adapter-appropriate syntax. The default implementation uses the SQL CONCAT
function with the separator interleaved between arguments. Adapter overrides produce
platform-specific syntax (|| operator, CAST to VARCHAR, etc.).
arguments:
- name: string_list
type: list[string]
description: List of column names or expressions to concatenate.
- name: separator
type: string
description: The string placed between each element. Default is '||' (Redshift/Oracle default to '|').

- name: default__concat_ws
description: Default implementation of concat_ws using CONCAT with interleaved separator. Accepts the same arguments as the parent concat_ws macro.

- name: exasol__concat_ws
description: Exasol-specific implementation of concat_ws. Accepts the same arguments as the parent concat_ws macro.

- name: redshift__concat_ws
description: Amazon Redshift-specific implementation of concat_ws using || operator. Accepts the same arguments as the parent concat_ws macro.

- name: oracle__concat_ws
description: Oracle-specific implementation of concat_ws using || operator. Accepts the same arguments as the parent concat_ws macro.

- name: trino__concat_ws
description: Trino-specific implementation of concat_ws with CAST to VARCHAR. Accepts the same arguments as the parent concat_ws macro.

- name: escape_column_names
description: >
Internal helper. Applies adapter-specific column name escaping to a single column string,
a list of column strings, or a dict with 'source_column' and 'alias' keys. Returns the
same structure as the input with each column name escaped.
arguments:
- name: columns
type: any
description: >
A column name string, a list of column name strings, or a dict with
'source_column' and 'alias' keys.

- name: escape_column_name
description: >
Internal helper. Escapes a single column name using the adapter-appropriate escape characters.
The default implementation uppercases the name and wraps it in double quotes.
Dispatches per adapter.
arguments:
- name: column
type: string
description: The raw column name to escape.

- name: default__escape_column_name
description: Default implementation of escape_column_name. Uppercases the name and wraps it in double quotes. Accepts the same arguments as the parent escape_column_name macro.

- name: synapse__escape_column_name
description: Azure Synapse Analytics-specific implementation of escape_column_name. Accepts the same arguments as the parent escape_column_name macro.

- name: bigquery__escape_column_name
description: BigQuery-specific implementation of escape_column_name using backtick escape characters. Accepts the same arguments as the parent escape_column_name macro.

- name: postgres__escape_column_name
description: PostgreSQL-specific implementation of escape_column_name. Lowercases the name. Accepts the same arguments as the parent escape_column_name macro.

- name: redshift__escape_column_name
description: Amazon Redshift-specific implementation of escape_column_name. Lowercases the name and wraps it in double quotes. Accepts the same arguments as the parent escape_column_name macro.

- name: exasol__escape_column_name
description: Exasol-specific implementation of escape_column_name. Accepts the same arguments as the parent escape_column_name macro.

- name: fabric__escape_column_name
description: Microsoft Fabric-specific implementation of escape_column_name. Respects the datavault4dbt.set_casing variable. Accepts the same arguments as the parent escape_column_name macro.

- name: databricks__escape_column_name
description: Databricks-specific implementation of escape_column_name. Respects the datavault4dbt.set_casing variable. Accepts the same arguments as the parent escape_column_name macro.

- name: sqlserver__escape_column_name
description: SQL Server-specific implementation of escape_column_name. Accepts the same arguments as the parent escape_column_name macro.

- name: trino__escape_column_name
description: Trino-specific implementation of escape_column_name. Accepts the same arguments as the parent escape_column_name macro.

- name: expand_column_list
description: >
Internal helper. Flattens a list that may contain nested sub-lists and filters out None
values, returning a single flat list of column names. Used to normalize column list inputs
before further processing.
arguments:
- name: columns
type: list[any]
description: A list of column names, possibly containing nested lists or None values.

- name: multikey
description: >
Internal helper. Generates a multi-column SQL condition string for JOIN ON clauses or
WHERE filters. Pairs each column from 'columns' with the corresponding column from
'right_columns' (if provided) or with itself, applying an optional table prefix and
comparison operator. Dispatches to default__multikey.
arguments:
- name: columns
type: list[string]
description: List of column names on the left side of the condition.
- name: prefix
type: string
description: Optional table alias prefix prepended to each left-side column.
- name: condition
type: string
description: >
The SQL logical operator used to join multiple conditions. Typically 'AND' or 'OR'.
- name: operator
type: string
description: >
The SQL comparison operator between left and right columns. Typically '=' or 'IS NOT DISTINCT FROM'.
- name: right_columns
type: list[string]
description: >
Optional list of column names for the right side. If not provided, each left column
is compared to itself (useful for self-joins or deduplication conditions).

- name: default__multikey
description: Default implementation of multikey. Accepts the same arguments as the parent multikey macro.
Loading