Skip to content

refactor: route the remaining user-facing errors through the message bundle - #164

Merged
scolladon merged 1 commit into
mainfrom
fix/route-remaining-user-errors-through-bundle
Aug 25, 2026
Merged

scolladon merged 1 commit into
mainfrom
fix/route-remaining-user-errors-through-bundle

Conversation

@scolladon

Copy link
Copy Markdown
Owner

Explain your changes


Completes the work started in dd69ee2 (#162), which moved ten ConfigReader errors into
messages/apex.mutation.test.run.md but left three behind because they lived in classes with
no Messages dependency. All three are user-actionable — each is caused by a flag the user
typed and fixed by changing it — so they belong on the bundle side of the line #162 drew:

Site New key
HTMLReporter.ts — --report-dir resolves outside the cwd error.reportDirOutsideCwd
HTMLReporter.ts — --report-dir dereferences outside the cwd error.reportDirSymlinkOutsideCwd
mutantGenerator.ts — every mutator excluded by configuration error.allMutatorsExcluded

The wording is byte-identical to what shipped before; only the source of the string moves. The
point is visibility: test/unit/messages/apexMutationTestRunBundle.test.ts loads the real
bundle unmocked, so from now on a renamed or missing key fails that suite instead of throwing
MissingMessageError in front of a user, on a failure path no happy-path run reaches.

Design decision — constructor injection, matching ConfigReader's existing shape.
Neither class had a constructor. ApexMutationHTMLReporter was cheap (one construction site in
run.ts, one in its test). MutantGenerator was the real question: 21 construction sites across
unit, integration, perf and NUT suites. Passing Messages per call was still worse — compute()
already takes seven parameters, an eighth positional would touch the same 21 sites anyway, and
making it optional would reintroduce the hardcoded fallback this change exists to remove. The 20
sites that never trip the filter take a shared keyEchoingMessages() bundle from
test/utils/testUtil.ts rather than each carrying its own template map.

Internal invariant throws are deliberately left hardcoded — apexClassRepository's
container/request ID guards and poll options, orgMutationTestBed's prepare-before-evaluate
precondition, mutationLocation, exactColoring, and mutantGenerator's overlapping-token-range
check. They are programmer errors, and putting them in a user-facing catalogue would imply the
user can act on them.

Also documents one testing hazard in DESIGN.md that cost real debugging time and is invisible on
inspection: await expect(p).rejects.toThrow('message') passes when the promise rejects with
undefined. It is only reachable where the thrown value is computed rather than constructed inline
(throw someFactory(...)), because a mutant can gut the factory into returning undefined. Such a
site should also assert rejects.toBeInstanceOf(Error), sharing one promise variable so the subject
is not executed twice. Exactly one such site exists today (mutationTestingService.ts, already
fixed), so this is a documented convention rather than a test helper.

Does this close any currently open issues?


No open issue — follow-up to #162.

  • Unit tests added to cover the fix.
  • NUT tests added to cover the fix.
  • E2E tests added to cover the fix.

E2E is unaffected: the rendered sentences are unchanged, and neither error is on the path the
E2E snapshot exercises.

Any particular element that can be tested locally


No new flags and no behaviour change. The three messages render exactly as before:

# resolves outside the cwd
node ./bin/run.js apex mutation test run -o <org> -c Mutation -t MutationTest -r /tmp

# every mutator excluded
node ./bin/run.js apex mutation test run -o <org> -c Mutation -t MutationTest \
  --include-mutators NonExistentMutator

Any other comments


Gates run locally

Gate Result
npx tsc -p . --noEmit clean
npx @biomejs/biome check --error-on-warnings src test clean
npx vitest run --coverage 2124 passed, 100% on all four metrics
npx vitest run --config vitest.config.nut.ts 56 passed
npm run lint:dependencies (knip) clean
npx commitlint --from main --to HEAD clean
npm outdated clean — nothing to upgrade
npm ls zod 4.4.3, deduped under @salesforce/core

Mutation testing on the diff — run per-hunk against the changed ranges under a temporary
typescript@6.0.3 (Stryker 10.0.0 crashes on ts.parseConfigFileTextToJson under TS 7):

File                        | % score | killed | survived | errors
All files                   | 100.00  |     33 |        0 |      0
 HTMLReporter.ts            | 100.00  |     25 |        0 |      0
 mutantGenerator.ts         | 100.00  |      6 |        0 |      0
 mutationTestingService.ts  | 100.00  |      2 |        0 |      0

Message plumbing is exactly where StringLiteral mutants survive — four field-name literals
survived in dd69ee2 until tests pinned the rendered sentences. All three new key literals and both
argument arrays were mutated and killed, because each test asserts the whole rendered sentence with
its interpolated values rather than a fragment. No // Stryker disable was added; the single
Ignored mutant in the range is the pre-existing disable on INPUT_STREAM_NAME.

@github-actions

Copy link
Copy Markdown

Preview build for this pull request:

sf plugins install https://pkg.pr.new/apex-mutation-testing@829e6c9

@github-actions

Copy link
Copy Markdown

Performance Comparison (same runner)

Stable

Benchmark Base PR Ratio Change
antlr-lex-small 7825 6840 1.14 -14.4%
antlr-parse-small 134 124 1.08 -8.1%
antlr-lex-medium 1687 1477 1.14 -14.2%
antlr-parse-medium 26 24 1.08 -8.3%
antlr-lex-large 522 456 1.14 -14.5%
antlr-parse-large 10 9 1.11 -11.1%
pipeline-small-compute-mutations 127 120 1.06 -5.8%
pipeline-small-type-discovery 168 150 1.12 -12.0%
pipeline-medium-compute-mutations 31 30 1.03 -3.3%
pipeline-medium-type-discovery 36 33 1.09 -9.1%
pipeline-large-compute-mutations 11 10 1.10 -10.0%
pipeline-large-type-discovery 13 12 1.08 -8.3%
pipeline-apply-all-mutations 3 3 1.00 +0%
antlr-lex-small (mean) 0.1278ms 0.1462ms 1.14 +14.4%
antlr-parse-small (mean) 7.4404ms 8.0698ms 1.08 +8.5%
antlr-lex-medium (mean) 0.5929ms 0.6771ms 1.14 +14.2%
antlr-parse-medium (mean) 39.0923ms 41.1836ms 1.05 +5.3%
antlr-lex-large (mean) 1.9172ms 2.1941ms 1.14 +14.4%
antlr-parse-large (mean) 98.7277ms 116.6644ms 1.18 +18.2%
pipeline-small-compute-mutations (mean) 7.9046ms 8.3369ms 1.05 +5.5%
pipeline-small-type-discovery (mean) 5.957ms 6.6793ms 1.12 +12.1%
pipeline-medium-compute-mutations (mean) 32.26ms 33.4236ms 1.04 +3.6%
pipeline-medium-type-discovery (mean) 27.5501ms 29.9605ms 1.09 +8.7%
pipeline-large-compute-mutations (mean) 94.2524ms 101.8263ms 1.08 +8.0%
pipeline-large-type-discovery (mean) 76.3063ms 80.8368ms 1.06 +5.9%
pipeline-apply-all-mutations (mean) 308.0874ms 316.2131ms 1.03 +2.6%

@scolladon
scolladon merged commit 18f0b72 into main Aug 25, 2026
22 checks passed
@scolladon
scolladon deleted the fix/route-remaining-user-errors-through-bundle branch August 25, 2026 12:54
@github-actions

Copy link
Copy Markdown

Shipped in release v1.9.0.
Version v1.9.0 will be assigned to the latest npm channel soon
Install it using either v1.9.0 or the latest-rc npm channel

$ sf plugins install apex-mutation-testing@latest-rc
# Or
$ sf plugins install apex-mutation-testing@v1.9.0

💡 Enjoying apex-mutation-testing?
Your contribution helps us provide fast support 🚀 and high quality features 🔥
Become a sponsor 💙
Happy zombies detection!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant