project dev makes validation and code health visible and first-class #1523

Problem statement

Developers have to understand several commands, modes and scopes to know what was actually checked. My recent hands-on tour found that:

• 0 problems can mean a requested check did not actually run or findings were suppressed.
  • Extension validation now honors explicit --only selections and reports invoked/skipped checkers. fix: extension validate + fix + format should report what actually ran #1611
  • Validation identifiers are more granular, so individual findings can be ignored without hiding unrelated findings. Make metadata.description and metadata.label identifiers unique #1533
  • Suppressed findings are now represented in the validation output instead of being silently removed. 0 problems` no longer hides ignored findings. Ignored findings can make the validation report a clean result #1529
  • Extension validation now runs full validation by default; --full is deprecated. feat!: deprecate full and make it default for extension validate #1618
  • The built-in checker is now named builtin; sw-cli remains a deprecated alias for selections. feat!: rename sw-cli tool to builtin #1627
  • Project validation still does not report checker invocation status; project validate --only sw-cli --format json can still return only results: []. Validation tells developers what actually ran #1502, Aggregate project and extension validation results #1310

• Full validation can appear stuck while Composer/tool setup runs without visible progress. Would differentiate between commands and project dev TUI.
  • Composer still runs with --no-progress and captures output until completion. Couldn't be sure that cancellation works; "Working" or a spinner.
  • The project dev Validation tab should show progress, elapsed time, coverage and grouped results, and distinguish passed, failed and not-run checks. Run and inspect project validation in project dev #1524
  • Validation should expose the concrete Shopware baseline and avoid stale vendor/ or composer.lock state overriding the requested comparison. Make validation use an explicit Shopware baseline #1308

• Copy mode protects the source tree but adds repeated setup cost; --no-copy is faster but can leave vendor/ and composer.lock behind.
  • fix: QA follow-ups for faster directory copies #1637 fixes project temporary-directory cleanup and cancellation.
  • fix: project validate reading extension git directories #1629 skips nested VCS and development-environment directories.
  • Repeated copying, Composer bootstrap and tool setup remain expensive across validate → fix → revalidate workflows. Reduce wait time in validation workflows #1385
  • --no-copy can still modify the source tree through generated dependencies or lock files.
  • Target-version validation must not be silently overridden by existing vendor/ or composer.lock state. Make validation use an explicit Shopware baseline #1308
  • performance of extension validate and project validate without --no-copy flag is bad on MacOS with MS Defender #1634; slow perf would block potential internal adoption
  • Safer default not to modify the files. It's now copying files; if you don't want that, use the --no-copy flag.

• project validate and extension validate cover different things, so a green project does not mean its extensions are valid.
  • The Validation tab should discover local extensions and reuse normal extension validate results as a distinct scope. Show local extension validation health in project dev #1526
  • Project and extension results should be aggregated without losing their source or scope, while remaining distinguishable. Aggregate project and extension validation results #1310
  • A valid local extension source must not be presented as proof that its distributable package is valid. Show local extension validation health in project dev #1526

• extension validate --format json returned results: [] for a plugin, while project validate --only sw-cli --format json also returned results: [] even though sw-cli performs no project-level check. Empty results therefore do not reliably communicate whether validation passed or whether nothing applicable was checked.
  • Extension JSON now includes a tools array with invoked/skipped status. fix: extension validate + fix + format should report what actually ran #1611
  • Project JSON still does not include tool statuses.
  • Machine-readable output should distinguish passed, not checked, suppressed and execution-failed states. Validation tells developers what actually ran #1502, Aggregate project and extension validation results #1310, Run and inspect project validation in project dev #1524
  • CI output should preserve consistent severity, source locations and gating semantics across JSON, GitHub, GitLab, JUnit and Markdown. Run project validate in CI #1528

• extension fix --only sw-cli and project fix --only sw-cli exited successfully without changing files or reporting that no fix was available. A zero-exit invocation of fix therefore does not prove that any fix was performed.
  • sw-cli is now rejected as an unsupported fixer in both commands because it implements checking, not fixing. fix: extension validate + fix + format should report what actually ran #1611
  • Valid fixers still do not report whether files changed or whether findings remain.
  • Findings need explicit ownership and fixability metadata; a finding should only be marked automatically fixable when its corresponding fix path resolves it. Show which validation findings extension fix can resolve #1309
  • Nested extensions still have inconsistent Git-safety behavior between extension fix and project fix. extension fix rejects extensions inside a Git project while project fix accepts them #1530

• On a disposable copy of FroshTools, extension fix --only eslint --allow-non-git exited silently and changed no files. The CLI does not explain whether ESLint found nothing fixable, whether no applicable files were selected, or whether no fix was performed.
  • Extension fix now prints a Fixers table showing that ESLint was invoked. fix: extension validate + fix + format should report what actually ran #1611
  • Tool selection is checked before setup. fix: check --only and --exclude before tool setup in extension fix and format #1624
  • The CLI still does not distinguish no applicable files, no fixable findings and no file changes.
  • Fixability should be reported per finding rather than inferred from the selected tool. Show which validation findings extension fix can resolve #1309
  • Formatter and fixer output should be normalized instead of leaking raw tool banners, progress bars and interleaved output. See Run project validate in CI #1528, Quiet formatter output when it isn't going to a terminal #1544

• In a disposable copy of FroshTools, ESLint reported one no-unused-expressions error. Running extension fix --only eslint --allow-non-git printed the same finding but produced no file changes, showing that validation can report an error without the selected fixer resolving it.
  • ESLint still runs --fix without post-fix validation and does not surface a normalized “remaining findings” result.
  • Findings without a working fix path must not be represented as automatically fixable. Show which validation findings extension fix can resolve #1309

Goal

Following the mockup in #1524, provide a Validation tab that displays findings and provides an ignore flow.

The Validation experience must:

• distinguish passed, not checked, suppressed and execution-failed checks;
• show progress, elapsed time, coverage and grouped results;
• keep project validation and local-extension validation as distinct scopes;
• identify the source scope and baseline for every result;
• preserve consistent findings and statuses across CLI, CI and machine-readable output;
• expose an ignore flow without making suppressed findings disappear from the report;
• distinguish findings that are automatically fixable from those requiring manual action.

Ongoing shopware-lsp investigations will be enabling work for us to understand where and how errors and suppression state are stored, which will impact related CLI operations.

Out of scope

New validation rules, upgrade-specific analysis, Store/account extension management.

Business / user impact

Lower TCO.

Acceptance criteria

Completion of the linked stories.

Readiness checklist

• Acceptance criteria are clearly defined.
• Backward compatibility impact addressed.
• Developer docs written.
• Tests added or adjusted accordingly.