Skip to content

Commit 766695d

Browse files
sungchun12sungchun12
committed
more examples
1 parent c0d8ec6 commit 766695d

1 file changed

Lines changed: 137 additions & 2 deletions

File tree

‎docs/examples/sqlmesh_cli_crash_course.md‎

Lines changed: 137 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ All these steps are bundled into a single command below:
4848
```
4949

5050
??? "Example Output"
51-
I made a big change to the `incremental_model` and `full_model`.
51+
I made a breaking change to `incremental_model` and `full_model`.
5252

5353
- Showed me the models impacted by the changes.
5454
- Showed me the changes that will be made to the models.
@@ -149,7 +149,6 @@ Run data diff against prod. This is a good way to verify the changes are behavin
149149
sqlmesh table_diff <environment>:<environment> <model_name> --show-sample
150150
```
151151

152-
153152
=== "Tobiko Cloud"
154153

155154
```bash
@@ -160,6 +159,74 @@ Run data diff against prod. This is a good way to verify the changes are behavin
160159
tcloud sqlmesh table_diff <environment>:<environment> <model_name> --show-sample
161160
```
162161

162+
??? "Example Output"
163+
I compare the `prod` and `dev` environments for `sqlmesh_example.full_model`.
164+
165+
- Verified environments and models to diff along with the join on grain configured.
166+
- Showed me schema diffs between the environments.
167+
- Showed me row count diffs between the environments.
168+
- Showed me common rows stats between the environments.
169+
- Showed me sample data differences between the environments.
170+
- This is where your human judgement comes in to verify the changes are behaving as expected.
171+
172+
```sql
173+
-- models/full_model.sql
174+
MODEL (
175+
name sqlmesh_example.full_model,
176+
kind FULL,
177+
cron '@daily',
178+
grain item_id, -- grain is optional BUT necessary for table diffs to work correctly. It's your primary key that is unique and not null.
179+
audits (assert_positive_order_ids),
180+
);
181+
182+
SELECT
183+
item_id,
184+
COUNT(DISTINCT id) AS num_orders,
185+
new_column
186+
FROM
187+
sqlmesh_example.incremental_model
188+
GROUP BY item_id, new_column
189+
```
190+
191+
```bash
192+
Table Diff
193+
├── Model:
194+
│ └── sqlmesh_example.full_model
195+
├── Environment:
196+
│ ├── Source: prod
197+
│ └── Target: dev
198+
├── Tables:
199+
│ ├── Source: db.sqlmesh_example.full_model
200+
│ └── Target: db.sqlmesh_example__dev.full_model
201+
└── Join On:
202+
└── item_id
203+
204+
Schema Diff Between 'PROD' and 'DEV' environments for model 'sqlmesh_example.full_model':
205+
└── Schemas match
206+
207+
208+
Row Counts:
209+
└── PARTIAL MATCH: 5 rows (100.0%)
210+
211+
COMMON ROWS column comparison stats:
212+
pct_match
213+
num_orders 100.0
214+
new_column 0.0
215+
216+
217+
COMMON ROWS sample data differences:
218+
Column: new_column
219+
┏━━━━━━━━━┳━━━━━━┳━━━━━┓
220+
┃ item_id ┃ PROD ┃ DEV ┃
221+
┡━━━━━━━━━╇━━━━━━╇━━━━━┩
222+
│ -11 │ 5 │ 7 │
223+
│ -3 │ 5 │ 7 │
224+
│ 1 │ 5 │ 7 │
225+
│ 3 │ 5 │ 7 │
226+
│ 9 │ 5 │ 7 │
227+
└─────────┴──────┴─────┘
228+
```
229+
163230
Apply the changes to prod. This step is recommended only in CICD as best practice.
164231

165232
For learning purposes and hot fixes, you can apply the changes to prod by entering `y` at the prompt.
@@ -176,6 +243,74 @@ For learning purposes and hot fixes, you can apply the changes to prod by enteri
176243
tcloud sqlmesh plan
177244
```
178245

246+
??? "Example Output"
247+
After I feel confident about the changes, I apply them to `prod`.
248+
249+
- Showed me the models impacted by the changes.
250+
- Showed me the changes that will be made to the models.
251+
- Showed me the models that need to be backfilled. None in this case as it was already backfilled earlier in `dev`.
252+
- Prompted me to apply the changes to `prod`.
253+
- Showed me physical layer and execution steps are skipped as the changes were already applied to `dev`.
254+
- Updated the virtual layer with view pointers to reflect the changes.
255+
256+
```bash
257+
Differences from the `prod` environment:
258+
259+
Models:
260+
├── Directly Modified:
261+
│ ├── sqlmesh_example.full_model
262+
│ └── sqlmesh_example.incremental_model
263+
└── Indirectly Modified:
264+
└── sqlmesh_example.view_model
265+
266+
---
267+
268+
+++
269+
270+
@@ -9,7 +9,8 @@
271+
272+
SELECT
273+
item_id,
274+
COUNT(DISTINCT id) AS num_orders,
275+
- 5 AS new_column
276+
+ new_column
277+
FROM sqlmesh_example.incremental_model
278+
GROUP BY
279+
- item_id
280+
+ item_id,
281+
+ new_column
282+
283+
Directly Modified: sqlmesh_example.full_model (Breaking)
284+
285+
---
286+
287+
+++
288+
289+
@@ -15,7 +15,7 @@
290+
291+
id,
292+
item_id,
293+
event_date,
294+
- 5 AS new_column
295+
+ 7 AS new_column
296+
FROM sqlmesh_example.seed_model
297+
WHERE
298+
event_date BETWEEN @start_date AND @end_date
299+
300+
Directly Modified: sqlmesh_example.incremental_model (Breaking)
301+
└── Indirectly Modified Children:
302+
└── sqlmesh_example.view_model (Indirect Breaking)
303+
Apply - Virtual Update [y/n]: y
304+
305+
SKIP: No physical layer updates to perform
306+
307+
SKIP: No model batches to execute
308+
309+
Updating virtual layer ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.0% • 3/3 • 0:00:00
310+
311+
✔ Virtual layer updated
312+
```
313+
179314
---
180315

181316
=== "SQLMesh"

0 commit comments

Comments
 (0)