Skip to content

Commit 9ae16d6

Browse files
committed
clear up concepts
1 parent 54642d5 commit 9ae16d6

1 file changed

Lines changed: 8 additions & 8 deletions

File tree

docs/examples/sqlmesh_cli_crash_course.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -992,11 +992,11 @@ This command is intended be run on a schedule. It will skip the physical and vir
992992
);
993993
```
994994

995-
### **Forward-Only Development Workflow**
995+
## **Forward-Only Development Workflow**
996996

997-
This is an advanced workflow and specifically designed for extremely large incremental models that take a long time to run even during development. It solves for:
997+
This is an advanced workflow and specifically designed for large incremental models (ex: > 200 million rows) that take a long time to run even during development. It solves for:
998998

999-
- Transformaing data with constant schema evolution in json and nested array data types.
999+
- Transforming data with constant schema evolution in json and nested array data types.
10001000
- Retaining history of a calculated column and applying a new calculation to new rows going forward.
10011001

10021002
When you apply the plan to `prod` after the dev worfklow, it will NOT backfill historical data. It will only execute model batches **forward only** for new intervals (new rows).
@@ -1025,8 +1025,8 @@ When you apply the plan to `prod` after the dev worfklow, it will NOT backfill h
10251025

10261026
- I applied a change to a new column
10271027
- It impacts 2 downstream models
1028-
- I enforced a forward-only plan to avoid backfilling historical data
1029-
- I previewed the changes in a clone of the models impacted (clones will NOT be reused in production)
1028+
- I enforced a forward-only plan to avoid backfilling historical data for the incremental model (ex: `preview` language in the CLI output)
1029+
- I previewed the changes in a clone of the incremental impacted (clones will NOT be reused in production) along with the full and view models (these are NOT clones).
10301030

10311031
```bash
10321032
Differences from the `dev` environment:
@@ -1058,7 +1058,7 @@ When you apply the plan to `prod` after the dev worfklow, it will NOT backfill h
10581058
├── sqlmesh_example__dev.full_model (Forward-only)
10591059
└── sqlmesh_example__dev.view_model (Forward-only)
10601060
Models needing backfill:
1061-
├── sqlmesh_example__dev.full_model: [full refresh] (preview) -- preview is a signal that a clone will be created if supported by the query engine
1061+
├── sqlmesh_example__dev.full_model: [full refresh] (preview)
10621062
├── sqlmesh_example__dev.incremental_model: [2025-04-17 - 2025-04-17] (preview)
10631063
└── sqlmesh_example__dev.view_model: [recreate view] (preview)
10641064
Apply - Preview Tables [y/n]: y
@@ -1080,7 +1080,7 @@ When you apply the plan to `prod` after the dev worfklow, it will NOT backfill h
10801080
```
10811081

10821082

1083-
When this is applied to `prod`, it will only execute model batches for new intervals (new rows). That's why you notice it only made a virtual layer update for this example.
1083+
When this is applied to `prod`, it will only execute model batches for new intervals (new rows). This will NOT re-use `preview` models in development.
10841084

10851085
```bash
10861086
Differences from the `prod` environment:
@@ -1131,7 +1131,7 @@ When you apply the plan to `prod` after the dev worfklow, it will NOT backfill h
11311131
```
11321132

11331133

1134-
### **Miscellaneous**
1134+
## **Miscellaneous**
11351135

11361136
If you notice you have a lot of old development schemas/data, you can clean them up with the following command. This process runs automatically during the `sqlmesh run` command. This defaults to deleting old data past 7 days.
11371137

0 commit comments

Comments
 (0)