ci: measure pull request coverage on the PR head

[oakimov]

Problem

Codecov's patch check can fail on lines a PR never touched.

On a pull_request event, actions/checkout checks out the merge commit with main (its default ref), so the tests and the lcov report come from the merged files. codecov-action v5 then files that report under the PR head SHA (its "Override commits and pr for pull requests" step falls back to github.event.pull_request.head.sha), and Codecov draws it on the head commit's lines.

When main has changed a file the PR also touches, those line numbers differ. On #107, #105 had shifted src/language-model.ts by about 22 lines. Codecov reported 4 missing lines, including a comment line; in the merged file those numbers are a pre-existing uncovered failRunProtocol(… request could not be handled …) branch. The PR's own lines, at their merged positions, were hit 104, 11 and 75 times. A Linux bun test --coverage run, merged or not, covered all 42 changed lines.

Fix

• The required Typecheck, test, build job is unchanged on pull requests: it still tests the merge with main. main has no branch protection, so this is the only check that a PR still works on current main. It now uploads coverage only on pushes to main, where the tested commit is the reported one.
• A new Coverage job runs on pull requests only. It checks out github.event.pull_request.head.sha and uploads from there, so the report matches the commit Codecov draws it on.
• Both jobs run the same two commands as before (bun test --coverage --coverage-reporter=lcov, then bun scripts/strip-copied-coverage.ts), spelled out in the workflow. The Coverage job checks out PR heads that may predate any new script, while on pull_request the workflow itself comes from the merge commit. bun run test:coverage wraps the same commands for local runs.

Trade-off

Tests run twice on a pull request, in parallel (about 3 more runner minutes). The alternatives were worse:

• Checking out the head in the single job would stop testing the merge with main before merging.
• Uploading against the merge SHA (override_commit) files the report on a commit that isn't on the PR branch. codecov-action's docs don't say how Codecov compares a PR from such a commit, so I didn't rely on it.

Verification

• yamllint passes, and both jobs parse with the expected names and conditions.
• bun run test:coverage locally: 2377 pass, 0 fail, lcov written, 101 copied-module records stripped as before.
• test/architecture.test.ts passes (it scans package.json).
• This PR runs both jobs. The new Coverage job's upload is the check that matters.

No user-facing change, so no changelog entry. Adds a lesson and the test:coverage command to AGENTS.md.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Confidence Score: 5/5

[High impact] The PR appears safe to merge; the earlier coverage failure is fixed.

Summary

The PR separates coverage from merge testing so Codecov receives coverage measured on the PR head.

• Pull requests get coverage measured on their head commit.
• Developers can run CI's coverage steps with one command.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Pull request] --> B[Required job: test merge with main]
  A --> C[Coverage job: test PR head]
  C --> D[Upload PR-head coverage]
  E[Push to main] --> F[Required job: test main]
  F --> G[Upload main coverage]

Loading

_{Reviews (2) · Last reviewed commit: "ci: run the coverage commands without th..." · Reviewed by Greptile}