ci: gate pull requests on coverage and mutation score of the lines they change

[rgamba]

What

Two new merge-gating jobs in build.yml, both behind the existing jvm-build fan-in, so no ruleset change is needed:

• coverage runs the suite under Kover. diff-cover then fails the PR when fewer than 90% of the executable main-source lines it adds or modifies are covered, and lists the uncovered lines in the job summary. koverVerify also holds the merged total to a floor of 83% (CI measured 83.2%), so it can't slide.
• mutation answers whether the tests actually check the code, not only run it. PIT makes small breaking edits to the lines the PR changes (flips a condition, deletes a call, returns a constant) and reruns the covering tests against each one. PIT's own testStrengthThreshold fails the job when the tests detect fewer than 80% of the mutants on covered lines. Surviving mutants are listed in the job summary: each one is a change to the code that no test noticed.

Key decisions and trade-offs

• 90% on changed lines, not on the total. The merged total is 83%, so a total gate at 90% would fail every PR until the backlog is paid down. The changed-lines gate makes every new PR meet 90%, which raises the total over time, and the floor blocks regressions in the meantime.
• Off-the-shelf where possible. diff-cover (pinned, run through pipx) handles changed-line coverage, and PIT's built-in threshold handles the mutation score. The source roots diff-cover needs are discovered from the tree, so a new project is picked up without editing the workflow.
• The one piece of custom code is tools/pitest-filters, a small PIT plugin (not published):
  • CHANGED_LINES reads git diff --unified=0 and drops every mutant off the added or modified lines before PIT plans any. Open-source PIT can only narrow to whole classes. The engine's tests are end-to-end and poll for workflow state, so a broken engine makes them wait until they time out: mutating the full classes PR feat: cooperative in-flight cancellation checkpoint (opt-in) #7 touched ran for over 35 minutes locally without finishing. With the filter it takes 5 min 36 s. A project the diff doesn't touch has nothing left to mutate, and PIT skips it in about a second, so one ./gradlew pitest covers every project.
  • KOTLIN_NULL_CHECKS drops mutants that only remove a null check the Kotlin compiler inserted (Intrinsics::checkNotNull…). No test can catch those, and open-source PIT doesn't filter them.
  • The commercial Arcmutate plugins would replace this (their git plugin mutates changed lines only, and their Kotlin plugin also filters coroutine and inlining noise). Their free open-source licence has to be requested and renewed, so I didn't use it.
• 80% mutation threshold, not 100%. Some mutants are equivalent: they change nothing observable, so no test can kill them. PIT can't mark individual mutants as acceptable, so the threshold leaves room for them.
• Separate jobs, not part of jvm-build-jackson. Each failure is its own signal, they start immediately instead of waiting on the matrix, and coverage instrumentation stays out of the builds the release relies on.
• testutils/trace is excluded from coverage. Only the formal workflow's trace-validation job runs it (under -PskipperTraceDir), so here it would always read as untested.
• Known limitations.
  • PIT runs per project, so testutils changes are mutation-tested only by testutils' own tests, while most of its coverage comes from the engine's tests. Those mutants show up as uncovered, and PIT leaves uncovered mutants out of the score.
  • Both diff-cover and CHANGED_LINES match files by package and file name, so they rely on every source file sitting in the directory its package names. All 264 main sources do today. A file that didn't would be skipped, not wrongly scored.

Versions: Kover 0.9.1, diff-cover 10.6.0, gradle-pitest-plugin 1.19.0, PIT 1.30.0, pitest-junit5-plugin 1.2.2.

Testing

Calibrated against PR #7 (cooperative cancellation, ~150 changed lines in the engine core, unchanged since it merged):

• Coverage: diff-cover and the custom script it replaced give identical results, 95% (78/82). The 4 missing lines are real untested fallback paths in WorkflowExecutionTaskHandler.kt (341–342, 424–425).
• Mutation: 88% test strength (23/26 detected), passes at 80%. The 3 survivors are real gaps:
  • ActionExecutor.kt:125 and WorkflowExecutor.kt:420: removing the cancellation SkipperCounter::inc breaks no test.
  • FeatureGate.kt:54: Keys.enabledByDefault can always return false without any test noticing.
• The threshold fails the build when it should. With the filter tests' assertions temporarily weakened, pitest fails with Test strength score of 75 is below threshold of 80.
• The gates caught two problems in this PR's own earlier runs:
  • A filter test compared two mutants that share an identifier, which is all MutationDetails.equals looks at, so it couldn't tell them apart. Fixed.
  • Kover silently skipped the plain-Java filter project, so the coverage gate saw 0/0 changed lines. Fixed by applying the Kotlin plugin there.
• On this PR: coverage 100% (66/66 changed lines), mutation 100% (35/35). The plugin has unit tests (FiltersTest). spotlessCheck and actionlint pass.