Skip to content

feat: subtract a warm-up boot baseline from Tia dependency edges - #1807

Open
iAmKevinMcKee wants to merge 1 commit into
pestphp:5.xfrom
iAmKevinMcKee:feat/tia-warmup-boot-baseline
Open

feat: subtract a warm-up boot baseline from Tia dependency edges#1807
iAmKevinMcKee wants to merge 1 commit into
pestphp:5.xfrom
iAmKevinMcKee:feat/tia-warmup-boot-baseline

Conversation

@iAmKevinMcKee

@iAmKevinMcKee iAmKevinMcKee commented Jul 31, 2026

Copy link
Copy Markdown

The problem

Tia derives dependency edges from per-test coverage windows, and those windows include everything that runs in setUp(). Frameworks that boot inside every test — Laravel being the canonical case — re-execute the same bootstrap lines in every test's window: service providers, bootstrap/app.php, route registration. Filament makes this dramatic: panel route registration executes getPages()/getRelations() in every registered resource class on every boot (verified by dumping executed lines; Laravel route caching does not avoid it).

The result on a production Laravel 12 + Filament 5 app (~100 resources):

  • a pure enum unit test recorded 146 dependency edges, including all ~100 Filament resource classes, every service provider, and the core models;
  • a dashboard feature test recorded 1018 edges;
  • editing any Filament resource — where most day-to-day work happens — invalidated the entire suite. Tia effectively degraded to "always run everything" for the changes developers actually make.

Any Filament project will hit this; any Laravel project hits a milder version (providers/routes as universal edges).

The fix

pest()->tia()->warmupUsing(Closure $callback) — a callback that runs once per process under the coverage driver, before the first test. Every line it executes forms a warm-up baseline that is subtracted from each test's recorded coverage before edges are derived. A file only becomes a dependency when a test executes lines beyond the baseline — so a feature test that genuinely renders a resource's form keeps that edge, while a unit test that merely booted the framework loses it.

// tests/Pest.php
pest()->tia()->warmupUsing(function (): void {
    $app = require __DIR__.'/../bootstrap/app.php';
    $app->make(Illuminate\Contracts\Console\Kernel::class)->bootstrap();

    // Undo global state the boot mutated before returning — see note below.
});

Measured results (same app as above)

Metric Before After
Enum unit test edges 146 6 (its enum, TestCase, the tenant models its setUp genuinely touches)
Dashboard feature test edges 1018 27
Edit an unrelated Filament resource full suite reruns 0 tests rerun (682-test set replays in ~260ms)
Edit a service with 1 dependent test full suite reruns exactly 1 test file reruns
Edit a broadly-used model (Project) full suite reruns its 57 genuine dependents rerun, rest replay

Warm-up responsibility (important caveat)

The callback runs in the test process, so it must clean up global state it mutates. For Laravel that includes a subtle one we hit in practice: booting an app loads .env through Dotenv, and the static Illuminate\Support\Env repository then treats those variables as Dotenv-owned — when the real test application boots later, Dotenv overwrites the restored APP_ENV=testing with the .env value, which (among other things) silently disables Filament's fillFormDataForTesting() and produces confusing downstream failures. A working Laravel warm-up restores $_ENV/$_SERVER/putenv snapshots, resets Env::$repository via reflection, resets the container/facades, and unwinds error/exception handlers back to their pre-boot state. The warmupUsing() PHPDoc calls this out; if maintainers prefer, a follow-up could ship a ready-made Laravel warm-up (Pest already has Laravel awareness in WatchDefaults\Laravel) so users get this behavior with zero config — happy to do that in this PR or a follow-up, whichever you prefer.

Implementation notes

  • The baseline is collected lazily at the first beginTest() with coverage capture active: start driver → run callback → collect scoped lines → reset for the per-test window. No behavior change when no warm-up is registered.
  • Subtraction happens in Recorder::filesWithExecutedLines() per file/line; the existing single-line/max-line heuristic is preserved.
  • Piggyback-coverage mode (--coverage + Tia) is unaffected — the warm-up only runs when the Recorder drives pcov/xdebug itself.
  • Driver-dependent tests probe that the driver can actually instrument the fixtures (pcov only instruments under pcov.directory) and skip otherwise, so they are safe in any CI configuration.

Tests

  • warmupUsing() excludes files fully covered by the baseline.
  • Files where a test goes beyond the baseline keep their edge.
  • No-warm-up behavior unchanged.
  • The callback is never invoked without a coverage driver (link-tracking mode).

composer lint, composer test:type:check pass; composer test:unit passes apart from Backtrace::it gets file name from called file, which fails identically on a clean checkout in this environment (pcov extension loaded) and is unrelated.

🤖 Generated with Claude Code

…dges

Frameworks that boot inside every test's setUp() re-execute the same
bootstrap lines in every per-test coverage window. In a Laravel app every
service provider, route file, and (with Filament) every panel resource
executes getPages()/getRelations() on every boot, so Tia links every test
to every bootstrap-executed file — a change to any of ~100 resource
classes reruns the entire suite, defeating test impact analysis.

pest()->tia()->warmupUsing(fn () => ...) registers a callback that runs
once per process under the coverage driver before the first test. Every
line it executes becomes a baseline that is subtracted from each test's
recorded coverage before dependency edges are derived; a file only
becomes an edge when a test executes lines beyond the baseline.

Measured on a production Laravel 12 + Filament 5 app (~100 resources):
a pure enum unit test dropped from 146 recorded edges to 6, a dashboard
feature test from 1018 to 27, and editing an unrelated Filament resource
went from invalidating the whole suite to invalidating nothing, while
changes to genuinely shared models still selected their real dependents.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant