Commit be96508
committed
refactor(12/12): add snapshot test fixtures and benchmarks (#330)
This is **PR 12 of 12**, the final PR in the stacked series that decouples the rendering pipeline from MCP transport. Depends on PR 11.
Adds the snapshot test suite and performance benchmarks that validate the entire rendering pipeline end-to-end. These are large in line count but are almost entirely test fixtures (expected output files) and benchmark scripts.
The snapshot test infrastructure captures the rendered output of tool invocations and compares against expected fixtures. This provides regression protection for the rendering pipeline -- any change to event formatting, diagnostic grouping, or output ordering will be caught by a fixture mismatch.
**Test harness** (`src/snapshot-tests/`):
- `harness.ts`: Core test runner that invokes tools with mock executors and captures rendered output
- `fixture-io.ts`: Reads/writes fixture files, handles normalization (timestamps, paths, UUIDs)
- `flowdeck-fixture-io.ts`: Flowdeck-specific fixture handling
- `normalize.ts`: Output normalization for stable comparisons across environments
- `resource-harness.ts`: Resource-specific snapshot testing
**Fixtures**: Expected output files for each tool covering success, error, and edge case scenarios. These serve as living documentation of what each tool's output looks like.
Performance benchmarks for the rendering pipeline and xcodebuild parsing:
- Parser throughput: lines/second for xcodebuild output parsing
- Render session performance: events/second for text and JSON strategies
- End-to-end tool invocation timing
These benchmarks establish baselines and can be run in CI to catch performance regressions.
This PR is large by line count but low in conceptual complexity. The fixture files are auto-generated expected outputs. The benchmark scripts are straightforward timing loops. The meaningful code is the ~500 lines of test harness infrastructure.
- PR 1-11/12: All code and configuration changes
- **PR 12/12** (this PR): Snapshot tests and benchmarks
- [ ] `npx vitest run` passes -- snapshot tests match expected fixtures
- [ ] `npx vitest run --config vitest.snapshot.config.ts` runs snapshot suite specifically
- [ ] Benchmarks execute without errors (performance numbers are informational)1 parent 211b14c commit be96508
File tree
221 files changed
+6343
-476
lines changed- docs/dev
- src
- mcp/tools
- debugging/__tests__
- device/__tests__
- doctor
- macos/__tests__
- project-discovery/__tests__
- project-scaffolding/__tests__
- session-management/__tests__
- simulator-management/__tests__
- simulator
- __tests__
- swift-package/__tests__
- ui-automation
- __tests__
- utilities/__tests__
- workflow-discovery/__tests__
- xcode-ide/__tests__
- snapshot-tests
- __fixtures__
- coverage
- debugging
- device
- macos
- mcp-integration
- project-discovery
- project-scaffolding
- resources
- session-management
- simulator-management
- simulator
- swift-package
- ui-automation
- utilities
- __tests__
- types
- utils
- __tests__
- responses
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
221 files changed
+6343
-476
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
201 | 201 | | |
202 | 202 | | |
203 | 203 | | |
204 | | - | |
| 204 | + | |
205 | 205 | | |
206 | 206 | | |
207 | 207 | | |
208 | | - | |
| 208 | + | |
| 209 | + | |
209 | 210 | | |
210 | 211 | | |
211 | 212 | | |
| |||
216 | 217 | | |
217 | 218 | | |
218 | 219 | | |
219 | | - | |
220 | | - | |
| 220 | + | |
| 221 | + | |
221 | 222 | | |
222 | 223 | | |
223 | 224 | | |
224 | | - | |
225 | | - | |
226 | | - | |
227 | | - | |
228 | | - | |
229 | | - | |
230 | | - | |
231 | | - | |
232 | | - | |
233 | | - | |
234 | | - | |
235 | | - | |
236 | | - | |
237 | | - | |
238 | | - | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
239 | 250 | | |
240 | 251 | | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
245 | | - | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
253 | | - | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
254 | 260 | | |
255 | 261 | | |
256 | 262 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | | - | |
51 | 50 | | |
52 | 51 | | |
53 | 52 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
13 | 12 | | |
14 | 13 | | |
15 | 14 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
9 | 8 | | |
10 | 9 | | |
11 | 10 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
12 | 11 | | |
13 | 12 | | |
14 | 13 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
219 | | - | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
220 | 223 | | |
221 | 224 | | |
222 | 225 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
9 | 8 | | |
10 | 9 | | |
11 | 10 | | |
| |||
0 commit comments