Skip to content

Commit 7333fe4

Browse files
committed
Update docs
1 parent 63ddfdb commit 7333fe4

2 files changed

Lines changed: 40 additions & 0 deletions

File tree

docs/concepts/macros/macro_variables.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ Prefixes:
6868
Postfixes:
6969

7070
* dt - A python datetime object that converts into a native SQL `TIMESTAMP` (or SQL engine equivalent)
71+
* dtntz - A python datetime object that converts into a native SQL `TIMESTAMP WITHOUT TIME ZONE` (or SQL engine equivalent)
7172
* date - A python date object that converts into a native SQL `DATE`
7273
* ds - A date string with the format: '%Y-%m-%d'
7374
* ts - An ISO 8601 datetime formatted string: '%Y-%m-%d %H:%M:%S'
@@ -83,6 +84,11 @@ All predefined temporal macro variables:
8384
* @end_dt
8485
* @execution_dt
8586

87+
* dtntz
88+
* @start_dtntz
89+
* @end_dtntz
90+
* @execution_dtntz
91+
8692
* date
8793
* @start_date
8894
* @end_date

docs/integrations/engines/redshift.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,37 @@ pip install "sqlmesh[redshift]"
3333
| `serverless_acct_id` | The account ID of the serverless cluster | string | N |
3434
| `serverless_work_group` | The name of work group for serverless end point | string | N |
3535
| `enable_merge` | Whether the incremental_by_unique_key model kind will use the native Redshift MERGE operation or SQLMesh's logical merge. (Default: `False`) | bool | N |
36+
37+
## Performance Considerations
38+
39+
### Timestamp Macro Variables and Sort Keys
40+
41+
When working with Redshift tables that have a `TIMESTAMP` sort key, using the standard `@start_dt` and `@end_dt` macro variables may lead to performance issues. These macros render as `TIMESTAMP WITH TIME ZONE` values in SQL queries, which prevents Redshift from performing efficient pruning when filtering against `TIMESTAMP` (without timezone) sort keys.
42+
43+
This can result in full table scans instead, causing significant performance degradation.
44+
45+
**Solution**: Use the `_dtntz` (datetime no timezone) variants of macro variables:
46+
47+
- `@start_dtntz` instead of `@start_dt`
48+
- `@end_dtntz` instead of `@end_dt`
49+
50+
These variants render as `TIMESTAMP WITHOUT TIME ZONE`, allowing Redshift to properly utilize sort key optimizations.
51+
52+
**Example**:
53+
54+
```sql linenums="1"
55+
-- Inefficient: May cause full table scan
56+
SELECT * FROM my_table
57+
WHERE timestamp_column >= @start_dt
58+
AND timestamp_column < @end_dt
59+
60+
-- Efficient: Uses sort key optimization
61+
SELECT * FROM my_table
62+
WHERE timestamp_column >= @start_dtntz
63+
AND timestamp_column < @end_dtntz
64+
65+
-- Alternative: Cast to timestamp
66+
SELECT * FROM my_table
67+
WHERE timestamp_column >= @start_ts::timestamp
68+
AND timestamp_column < @end_ts::timestamp
69+
```

0 commit comments

Comments
 (0)