Skip to content

feat(shot): act on previews before the capture #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
koji-1009 merged 1 commit into from
Sep 23, 2026
Merged

feat(shot): act on previews before the capture #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 18 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,16 +136,25 @@ shutter shot --widget 'PrimaryButton(label: "OK")' --import lib/ui/button.dart -
`--import` names a file under `lib/`, or a `package:` URI of a dependency, that the widget expression needs imported (repeatable; `package:flutter/widgets.dart` is always imported).
The same `--widget` and `--import` give the same shot id, so two such runs line up in `diff`.

A state that comes from a gesture is shot by acting on the preview first:

```bash
shutter shot lib/preview/button_preview.dart --press text:Save
```

`--tap` (repeatable, in the order given), then `--press`, `--hover`, or `--focus`, name their widget by `key:`, `text:`, or `type:`.
Each is followed by `--settle` milliseconds; shoot again with another `--settle` for another point of an animation.

## Subcommands

| Command | Purpose |
| -------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `agent` | The step-by-step playbook for AI agents. |
| `manual` | The reference: preview files, drawing model, engine, runs, ids, diff, output, exit codes. |
| `doctor` | Check the Flutter SDK version, its font cache, and the project's shell (`--shell`). |
| `init` | Write `<preview dir>/shell.dart`, or the `--shell` file. |
| `shot` | Render the named preview files, or one `--widget`, into a new run (`--widget`/`--import`/`--size`, `--settle`, `--shell`). |
| `diff <a> <b>` | Compare two runs (`--images`). |
| Command | Purpose |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `agent` | The step-by-step playbook for AI agents. |
| `manual` | The reference: preview files, drawing model, engine, runs, ids, diff, output, exit codes. |
| `doctor` | Check the Flutter SDK version, its font cache, and the project's shell (`--shell`). |
| `init` | Write `<preview dir>/shell.dart`, or the `--shell` file. |
| `shot` | Render the named preview files, or one `--widget`, into a new run (`--widget`/`--import`/`--size`, `--settle`, `--shell`, `--tap`/`--press`/`--hover`/`--focus`). |
| `diff <a> <b>` | Compare two runs (`--images`). |

## Exit codes

Expand All @@ -159,7 +168,7 @@ The same `--widget` and `--import` give the same shot id, so two such runs line

## Limits

* One frame, no interaction: taps, hovers, scrolling, and mid-animation states are not shot. State comes from the widget's construction expression.
* One capture per preview: state comes from the widget's construction expression and from taps, a press, a hover, or focus before the capture; scrolling, dragging, and typing are not shot.
* HTTP is blocked while rendering, so network images fail to load.
* Text renders with the project's fonts plus Roboto; CJK and emoji fall back to the host's system fonts, and Cupertino text uses SF Pro on macOS (Roboto elsewhere), as a device would. Compare runs made on the same machine.

Expand Down
15 changes: 15 additions & 0 deletions doc/agent.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,21 @@ shutter shot --widget 'PrimaryButton(label: "OK")' --import lib/ui/button.dart -
* Without `--size` the widget is shot at its own size; `--size` fixes both dimensions and stretches the widget to them.
* The same `--widget` and `--import` give the same shot id in every run, so before and after line up in `diff`.

## Pressed, hovered, focused, opened

A preview shows the state its construction gives. For a state a gesture gives, act on the preview in the shot:

```bash
shutter shot lib/preview/button_preview.dart --press text:Save
```

* `--tap` (repeatable, run in the order given), then at most one of `--press`, `--hover`, `--focus`. A target is `key:<ValueKey<String>>`, `text:<Text data>`, or `type:<Widget>`.
* The actions apply to every preview of the named files; a preview where the target matches no widget, or several, is an `error` shot. Name the files whose previews have the target, and give the same actions to the before and after shots.
* `--settle` (default 300 ms) is how long after the last action the image is taken. A tapped button's ink is still fading at 300 ms and gone by 700 ms: pass `--settle 700` for the state after a tap without it.
* A page a tap navigates to is shot through its own preview, not through the tap.
* An animation is shot point by point: shoot again with another `--settle`. The clock is simulated, so the same `--settle` gives the same image, and `diff` pairs a shot across the runs.
* Ink lands on the nearest `Material`, like the background above: an `InkWell` or `ListTile` wrapped in `Material` in the preview shows its press, hover, or focus; buttons carry their own.

## Exit codes

* `shot`: 0 all ok, 2 any error.
Expand Down
56 changes: 41 additions & 15 deletions doc/manual.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,35 @@ The shot id hashes the expression and the resolved imports, so the same `--widge
A widget expression that does not compile is an `error` shot carrying the compiler message; it has no `file` or `at`.
`--widget` and preview files cannot be combined in one shot.

## Actions

A state that comes from a gesture is shot by acting on the preview before the capture:

* `--tap <target>` taps the widget: a pointer down and up at its centre. Repeatable; the taps run in the order given.
* `--press <target>` holds a pointer down on it through the capture: its pressed state, with the ink the press has drawn by then.
* `--hover <target>` keeps a mouse pointer over it through the capture.
* `--focus <target>` gives it keyboard focus, highlighted as with a keyboard: `flutter_test` runs as a touch device, where focus draws no highlight. The node is the first focus node inside the widget (a button's or text field's own), else the one around it.
* At most one of `--press`, `--hover`, and `--focus`; it follows the taps.

A target is `key:<value>` (a `ValueKey<String>`), `text:<string>` (a `Text` showing exactly that string, or an `EditableText` holding it), or `type:<Widget>` (a widget of that type, with or without type arguments: `type:Checkbox`, `type:DropdownButton<String>`); offstage widgets do not count.
Each action is followed by `--settle` milliseconds drawn in 16 ms frames, as on a device, so `--settle` is also how long after an action the capture comes: an animation the action starts (ink, a check mark, a menu opening) is shot at that point of its course.
`flutter test` runs as Android, so Material 3 presses use `InkSparkle`: its sparkle shows from about 100 ms to about 600 ms of a press, then the flat pressed overlay stays, as on a device.

The actions apply to every preview of the run.
A preview is an `error` shot, without a PNG and with `at` pointing at the preview, when a target matches no widget or more than one, when a pointer at the target's centre does not reach it (covered, outside the viewport, or ignoring pointers), or when a `--focus` target cannot take focus.

The run records its actions in `manifest.json`; `shot` prints them, and `diff` prints both runs' when they differ, since they change every image without any widget changing.
They are not part of shot ids, so a run with actions pairs shot for shot with one without.

### Taps that navigate

A page is shot through a preview of its own: returned from a preview function, it is shot from its first frame, with an id of its own.
A tap that pushes a page covers the preview, which is then no longer painted: the shot is an `error` saying so.

## Drawing model

The engine renders each preview in a `flutter_test` binding, one frame, no interaction.
State comes from the widget's construction expression; a widget that fetches or builds state internally can only be shot in the state it reaches on its own.
The engine renders each preview in a `flutter_test` binding: one frame, then the run's actions, then the capture.
State comes from the widget's construction expression or from the actions; a widget that fetches or builds state internally can only be shot in the state it reaches on its own.

Per preview, from the outside in:

Expand All @@ -56,6 +81,7 @@ Per preview, from the outside in:
The PNG holds what is painted inside the captured region, and nothing outside it: the shell's surface lies outside, so where the preview paints no background the PNG is transparent, and the viewer's own background shows through.
A screen with a `Scaffold` paints its own; a widget is shot on no background, since shutter cannot know where the app places it.
To shoot a widget on the surface it sits on, paint it inside the preview: a `wrapper`, or a `ColoredBox` or `Material` around the widget in the preview function.
The same holds for ink: a press, hover, or focus is drawn on the nearest `Material` above the widget, which for a widget without its own (an `InkWell`, a `ListTile`) is the shell's surface; a `Material` in the preview brings it into the PNG.

The project's shell replaces the default one entirely, so it decides what surrounds every shot: a Material surface (what `shutter init` writes, and what widgets such as `ListTile` or `TextField` need), a `CupertinoApp`, or a `WidgetsApp` with the app's own design system.
Shutter adds no design library to a shell without one.
Expand All @@ -79,7 +105,7 @@ The test engine has no system font fallback, so a glyph missing from the style's
Cupertino text names the system font through the families `CupertinoSystemText` and `CupertinoSystemDisplay`, which the test engine does not resolve. On a macOS host they get SF Pro from `/System/Library/Fonts/`, as a macOS app does and as iOS draws; on other hosts, Roboto, the Android system font.
Host fonts come from the machine that shoots, so compare runs made on the same machine.

Settling: `Image` widgets are precached, then one `pump(settle)` (`--settle`, default 300 ms). `pumpAndSettle` is never used, because a loading indicator never settles.
Settling: `Image` widgets are precached, then one `pump(settle)` (`--settle`, default 300 ms); after each action, `--settle` ms more in 16 ms frames. `pumpAndSettle` is never used, because a loading indicator never settles.

HTTP is blocked by the test binding. A `NetworkImage` fails and the shot is `error`.

Expand Down Expand Up @@ -131,7 +157,7 @@ Fonts bundled in the project's assets are used as they are.

These come from rendering through `flutter test`.

* One frame, no interaction: taps, hovers, scrolling, and mid-animation states are not shot.
* One capture per preview, after the actions: scrolling, dragging, and typing are not shot. A sequence of states, such as the course of an animation, is shot as one run per point in time, each with its own `--settle`.
* HTTP is blocked, so network images fail to load.
* `flutter test` runs the engine with test fonts, which has no system font fallback. Shutter's host fonts are added to the text themes and the default text style, so a text style that sets its own `fontFamilyFallback` does not get them. google_fonts styles do this: glyphs outside the Google font (for example Japanese in a Latin-only font) render as boxes, where a device would fall back to a system font.

Expand All @@ -145,7 +171,7 @@ When Flutter ships a capture command in the previewer itself, it replaces v1 wit

## Runs

`.dart_tool/shutter/runs/<run-id>/` holds `<id>.png` per shot and `manifest.json` (`run`, `shell` when a shell file was used, `shots`).
`.dart_tool/shutter/runs/<run-id>/` holds `<id>.png` per shot and `manifest.json` (`run`, `shell` when a shell file was used, `actions` when given, `shots`).
`<run-id>` is the UTC time of the shot (`20260918T101530Z`), suffixed `-2`, `-3`, ... when taken; a hidden `.<run-id>` file claims the name, so runs started in the same second get distinct ids.
`shot` prints the run directory as `run:`; `diff` accepts a run's directory, its id, `latest` for the newest run, or `latest~N` for the run N before it.
`latest` counts runs in id order (time, then suffix) and skips a run still being shot, whose `manifest.json` is not written yet.
Expand Down Expand Up @@ -181,8 +207,8 @@ When the two runs were shot with different shells (path or sha256), `shell` show
## Output

`shot` and `diff` print YAML starting with the comment `# shutter ai-report v1`, with absolute paths to open.
`shot` gives `run`, `shell` (the shell file with its sha256, or `default`), `summary`, and `shots`, errors first.
`diff` gives `diff` (with `--images`), `before`, `after`, `shell` (when the shells differ), `summary`, and `entries` in the order changed → added → removed → unchanged.
`shot` gives `run`, `shell` (the shell file with its sha256, or `default`), `actions` (when given), `summary`, and `shots`, errors first.
`diff` gives `diff` (with `--images`), `before`, `after`, `shell` and `actions` (each when the two runs differ in it), `summary`, and `entries` in the order changed → added → removed → unchanged.

## Exit codes

Expand All @@ -196,11 +222,11 @@ Other failures follow sysexits: 64 usage, 66 missing run or file, 69 no Flutter

## Commands

| command | purpose |
| -------- | ------------------------------------------------------------------------------------------------------------------------- |
| `agent` | the step-by-step playbook |
| `manual` | this document |
| `doctor` | SDK version, font cache, shell (`--shell`) |
| `init` | write `shell.dart` (`--shell`) |
| `shot` | render the named preview files, or one `--widget`, into a new run (`--widget`/`--import`/`--size`, `--settle`, `--shell`) |
| `diff` | compare two runs (`--images`) |
| command | purpose |
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `agent` | the step-by-step playbook |
| `manual` | this document |
| `doctor` | SDK version, font cache, shell (`--shell`) |
| `init` | write `shell.dart` (`--shell`) |
| `shot` | render the named preview files, or one `--widget`, into a new run (`--widget`/`--import`/`--size`, `--settle`, `--shell`, `--tap`/`--press`/`--hover`/`--focus`) |
| `diff` | compare two runs (`--images`) |
15 changes: 15 additions & 0 deletions lib/src/cli/agent_text.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,21 @@ shutter shot --widget 'PrimaryButton(label: "OK")' --import lib/ui/button.dart -
* Without `--size` the widget is shot at its own size; `--size` fixes both dimensions and stretches the widget to them.
* The same `--widget` and `--import` give the same shot id in every run, so before and after line up in `diff`.

## Pressed, hovered, focused, opened

A preview shows the state its construction gives. For a state a gesture gives, act on the preview in the shot:

```bash
shutter shot lib/preview/button_preview.dart --press text:Save
```

* `--tap` (repeatable, run in the order given), then at most one of `--press`, `--hover`, `--focus`. A target is `key:<ValueKey<String>>`, `text:<Text data>`, or `type:<Widget>`.
* The actions apply to every preview of the named files; a preview where the target matches no widget, or several, is an `error` shot. Name the files whose previews have the target, and give the same actions to the before and after shots.
* `--settle` (default 300 ms) is how long after the last action the image is taken. A tapped button's ink is still fading at 300 ms and gone by 700 ms: pass `--settle 700` for the state after a tap without it.
* A page a tap navigates to is shot through its own preview, not through the tap.
* An animation is shot point by point: shoot again with another `--settle`. The clock is simulated, so the same `--settle` gives the same image, and `diff` pairs a shot across the runs.
* Ink lands on the nearest `Material`, like the background above: an `InkWell` or `ListTile` wrapped in `Material` in the preview shows its press, hover, or focus; buttons carry their own.

## Exit codes

* `shot`: 0 all ok, 2 any error.
Expand Down
Loading
Loading