Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ The host runner is `tests/smoke/run_example.py`. Shipped examples are listed in
`$BDT_SMOKE_SIDECAR` (set by the runner). The harness asserts after Blender
exits. Not a gallery still.
- **Falsifiers in CI:** every catalog row carries `falsifiers`, at least one
`{"args": [...], "expect_exit": N}` (N not 0, 1 or 77) or, for a contract
`{"args": [...], "expect_exit": N}` (N not 0, 1, 2 or 77; 2 is argparse usage) or, for a contract
the sidecar check owns, `{"args": [...], "expect_sidecar_fail": true}`.
A falsifier that targets a version-specific API change takes
`"min_version": "5.2"` and is recorded as SKIP below it (its README must
Expand Down
14 changes: 9 additions & 5 deletions examples/gn-bundle-roundtrip/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ not read at thumbnail.
**What failure each check would catch:**

- exit 77 — Blender < 5.0 and not `--force-run`
- exit 2 — `GeometryNodeCombineBundle` on 5.x (`--legacy-rna`)
- exit 7 — `GeometryNodeCombineBundle` on 5.x (`--legacy-rna`); exit 8 if
that legacy identifier is ever accepted again
- exit 3 — item names do not pair, so Separate yields no geometry
(`--mismatch` renames the geometry item → 0/0)
- exit 4 — count matches but the unpacked Scale/Offset did not arrive
Expand All @@ -78,10 +79,13 @@ against it. `77` is the smoke skip protocol, not a product check.
| --- | --- |
| 0 | Success |
| 1 | Uncaught exception (FATAL wrapper); `--force-run` on 4.5 lands here |
| 2 | argparse / usage; carrier mesh rewritten; `--legacy-rna` on 5.x |
| 2 | argparse / usage |
| 3 | Evaluated vert/face count off closed form (`--mismatch`) |
| 4 | Unpacked Scale/Offset x-extent off (`--bypass`, `--pack-scale 1`) |
| 5 | `bundle_mark` missing or off closed form (`--mismatch-mark`) |
| 6 | Carrier mesh rewritten |
| 7 | `--legacy-rna`: 5.x RNA rejects `GeometryNodeCombineBundle` (the catalog falsifier) |
| 8 | `--legacy-rna`: `GeometryNodeCombineBundle` was accepted (the rejection contract broke) |
| 77 | `SMOKE_SKIP:` Bundles require Blender 5.0+ |

Measured on 4.5.11 / 5.0.1 / 5.1.2 / 5.2.1:
Expand All @@ -91,12 +95,12 @@ Measured on 4.5.11 / 5.0.1 / 5.1.2 / 5.2.1:
| (none) | 77 | 0 | 0 | 0 |
| `--force-run` | 1 | 0 | 0 | 0 |
| `--bypass` | 77 | 4 | 4 | 4 |
| `--legacy-rna` | 77 | 2 | 2 | 2 |
| `--legacy-rna` | 77 | 7 | 7 | 7 |
| `--mismatch` | 77 | 3 | 3 | 3 |
| `--mismatch-mark` | 77 | 5 | 5 | 5 |
| `--pack-scale 1` | 77 | 4 | 4 | 4 |

The `blender-smoke` workflow runs the check on Blender 5.2 LTS (5.1 on the
weekly cron, the `needs-5.1` PR label, or manual dispatch) and skips on 4.5
LTS. Smoke does not pass `--bypass`, `--legacy-rna`, `--mismatch`,
`--mismatch-mark`, or `--force-run`.
LTS. Its catalog falsifier is `--legacy-rna` (expects exit 7). Smoke does
not pass `--bypass`, `--mismatch`, `--mismatch-mark`, or `--force-run`.
8 changes: 5 additions & 3 deletions examples/gn-bundle-roundtrip/gn_bundle_roundtrip.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ def build(pair, match_names, pack_scale, match_mark=True):
def check(ob):
if len(ob.data.vertices) != 1:
print("ERROR: carrier mesh was rewritten", file=sys.stderr)
return 2
return 6

coords, faces, marks = eval_mesh(ob)
xs = [c[0] for c in coords]
Expand Down Expand Up @@ -225,12 +225,14 @@ def main():
tree.nodes.new("GeometryNodeCombineBundle")
except RuntimeError as exc:
print(f"ERROR: {type(exc).__name__}: {exc}", file=sys.stderr)
return 2
return 7
# A distinct code so the catalog falsifier (expect 7) fails if the
# legacy identifier ever comes back, instead of passing on any 2.
print(
"ERROR: GeometryNodeCombineBundle existed; 5.x RNA should reject it",
file=sys.stderr,
)
return 2
return 8

ob = build(
pair=not args.bypass,
Expand Down
8 changes: 5 additions & 3 deletions tests/check_smoke_catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
Schema: only the known row keys are accepted (a typo such as "falsifer" would
otherwise be ignored and the falsifier silently never run), and every row
carries at least one falsifier: {"args": [...], "expect_exit": N} with N not
0, 1 or 77, or {"args": [...], "expect_sidecar_fail": true} on a row that
0, 1, 2 (argparse usage) or 77, or {"args": [...], "expect_sidecar_fail": true} on a row that
declares expect_sidecar.

python tests/check_smoke_catalog.py [--root PATH]
Expand All @@ -31,7 +31,9 @@
ROW_KEYS = {"name", "script", "args", "min_version", "expect_file",
"expect_sidecar", "sidecar_contains", "falsifiers"}
FALSIFIER_KEYS = {"args", "expect_exit", "expect_sidecar_fail", "min_version"}
FORBIDDEN_EXITS = {0, 1, 77}
# 2 is argparse's usage-error exit: an unknown or deleted flag also exits 2,
# so a falsifier expecting 2 passes even when its flag no longer exists.
FORBIDDEN_EXITS = {0, 1, 2, 77}


def check_schema(row: dict) -> list[str]:
Expand Down Expand Up @@ -60,7 +62,7 @@ def check_schema(row: dict) -> list[str]:
elif has_exit:
code = f["expect_exit"]
if not isinstance(code, int) or isinstance(code, bool) or code in FORBIDDEN_EXITS:
errors.append(f"{where}: expect_exit {code!r} must be an int other than 0, 1, 77")
errors.append(f"{where}: expect_exit {code!r} must be an int other than 0, 1, 2, 77")
elif not row.get("expect_sidecar"):
errors.append(f"{where}: expect_sidecar_fail on a row with no expect_sidecar")
mv = f.get("min_version")
Expand Down
2 changes: 1 addition & 1 deletion tests/smoke/catalog.json
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
"name": "gn-bundle-roundtrip",
"script": "examples/gn-bundle-roundtrip/gn_bundle_roundtrip.py",
"min_version": "5.0",
"falsifiers": [{"args": ["--legacy-rna"], "expect_exit": 2}]
"falsifiers": [{"args": ["--legacy-rna"], "expect_exit": 7}]
},
{
"name": "exit-pre-sidecar",
Expand Down
Loading