feat(ci): make the PR report easy to read and scan (STE, folds, colour, icons) by pauldambra · Pull Request #155 · PostHog/coherence

pauldambra · 2026-10-25T20:41:24Z

Why

People could not read the Coherence comment on posthog-js PRs. It used Coherence's own words (spec gap, entrance, trust, chokepoint, guard, invariant, crossing, baseline) and did not define them. The goal is a report that a new reader can act on, written in Simplified Technical English (ASD-STE100).

What changed

One sentence at the top says what Coherence is.
Each kind of finding has a name that says what it means for the reader:

Before After

spec problem spec file error

failing chokepoint broken rule

spec gap unchecked input

guard failure bug that came back
The sentences are short and active. The fix is a list of steps.
Spec syntax shows only as code that the reader types. It uses control: none - <reason> (the grammar accepts the hyphen), not an em-dash.
The layout lets a familiar reader scan the comment:
- Each section lists its findings first.
- "What this means and how to fix it" and "What is Coherence?" are folded in <details> blocks.
- The verdict is a GitHub alert, so it has a colour: green TIP (nothing to fix), blue NOTE (findings, none in this PR's files), purple IMPORTANT (some findings in this PR's files), and yellow WARNING (Coherence could not check everything).
- Icons mark each kind of finding (📄 ⛔ 🚪 🐛 ⚙️ ⏸️) and the changed-file mark (📍).
Sections with nothing in them are left out: "Spec size", "Chokepoints: none declared", and rules that pass.
The plain closing steps (CLOSE_STEPS) are next to CLOSE_WAYS in gaps.ts, so the two lists stay in step.
Version 1.10.1, so merging publishes it.

The posthog-js comment, after this change

This is the real output for posthog-js, as GitHub draws it:

Coherence

📄 0 spec file errors · ⛔ 0 broken rules · 🚪 2 unchecked inputs · 🐛 0 bugs that came back

What is Coherence?

Coherence compares the code with its spec files (*.spec.md). A spec file lists the rules for one part of the code.

Note

These findings do not make this check fail.

None of these findings are in files that this pull request changes. You do not need to fix them in this pull request.

🚪 Unchecked inputs (2)

public api in packages/web/Web.spec.md. The input comes from: host-app.
webpack plugin in packages/webpack-plugin/Webpack-plugin.spec.md. The input comes from: build.

What this means and how to fix it

Input from outside this code comes in at these places. The spec does not say what checks that input. 10 more places were already like this when Coherence was added. This list does not show them.

To close one of these, find the place by its name in the ## entrances section of its spec file. Add one line under it:

If the input needs no check, add control: none - <reason>.
If all of the input goes through one function that a rule in the spec protects, add guard: <function>.
If a rule in the spec already checks this input, add the place's name to the entrances: line of that rule.

To get the line to add, run npx coherence scaffold control "<name>".

Testing

The ci.test.ts cases are rewritten for the new text. They pass, and so do typecheck, lexicon:check and spec:check (0 problems).
I recorded two refutations for the spec bullet. Without the sentence that says what Coherence is, 2 test cases failed. Without the folds, 6 failed.
GitHub's Markdown API draws the alert, both folds, and all 8 code parts as planned.
The version tests (mod-install.test.ts, and the bundled plugin case in bundled.e2e.ts) pass at 1.10.1.

Created with PostHog Desktop

🤖 Generated with Claude Code

…e 1.10.1 The report used Coherence's own words (spec gap, entrance, trust, chokepoint, guard, invariant, crossing, baseline) with no definition. It now says in one sentence what Coherence is, names each kind of finding by what it means to the reader (spec file error, broken rule, unchecked input, bug that came back), uses short active sentences, gives the fix as a list, shows spec syntax only as code to type, and leaves out sections with nothing in them. The plain closing steps sit beside CLOSE_WAYS in gaps.ts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 4189fcb8-6fd5-428d-8c5f-4314d28e4dac

posthog

PostHog Review · Standard · reviewed 76d8c2d: 1 Should fix.

This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

… icons A familiar reader can now scan the report. Each section lists its findings first, and what the finding means and how to fix it sits in a details block below the list, as does "What is Coherence?". The verdict is a GitHub alert, so it has a colour: TIP when there is nothing to fix, NOTE for findings outside the pull request's files, IMPORTANT when some are in them, and WARNING when Coherence could not check everything. Icons mark each kind of finding and the changed-file mark. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 4189fcb8-6fd5-428d-8c5f-4314d28e4dac

pauldambra

Note

🤖 Automated comment by QA Swarm — not written by a human

QA Swarm review complete. See inline comments.

pauldambra · 2026-10-25T21:10:08Z

Note

🤖 Automated comment by QA Swarm — not written by a human

Multi-perspective review: router (cheap-first pass) + delegated lenses (qa-team, paul-reviewer, xp-reviewer, security-audit, engineering-systems-thinking as warranted)

Verdict: ⚠️ REQUEST CHANGES _{(round 1 @ 4534f26)}

There are 2 HIGH findings, both in CLOSE_STEPS. Its steps are in a different order from CLOSE_WAYS, and the entrances: step drops the "same source" condition. The rest is about information the report no longer shows, and about wording.

Key findings

🟠 HIGH: the CLOSE_STEPS entrances: step does not say that the rule must check input from the same source (gaps.ts:80).
🟠 HIGH: CLOSE_STEPS and CLOSE_WAYS are kept in two places and are already in a different order (gaps.ts:77).
🟡 MEDIUM: the route of an unchecked input, the spec size, and the passing-rule count are gone. "registry top" is not defined. The fold is written out 6 times.
🟢 LOW / ⚪ NIT: no test for "no rules declared", title() has two output shapes, "session" is not defined, the bugs fold title promises a fix, and "fix/change/close" are mixed.

Convergence

The two lists in two places (gaps.ts:77): ste + xp.
The unchecked input route is gone (ci-report.ts:177): router + xp.

Reviewer summaries

Reviewer	Assessment
🧭 router	Danger MEDIUM: 584 lines, and a release path. Flagged 4 removed pieces of information. Sent 2 reviews to Sonnet.
🧪 xp	The rewrite is good. One HIGH (with ste) on the two lists in two places, plus the repeated fold, the lost route, and the `title()` shapes.
✍️ general/ste	The STE wording is mostly strong. Two HIGH on `CLOSE_STEPS` (order, and the dropped condition), plus undefined "registry top" and "session".

Automated by QA Swarm — not a human review

Addresses the posthog bot thread and 12 QA Swarm findings on PR #155, per the shepherd's agreed direction: - every report command now reads `npx --no -- coherence ...`, so a reader without a local install does not fetch the wrong package - CLOSE_WAYS and CLOSE_STEPS build from one CLOSE_PARTS list (guard, entrances, control none, in that order), so the two forms cannot drift apart; the entrances step now says "from the same source" - an unchecked input's route is shown again as "It reaches: `<stops>`." when it goes further than the entrance's own component - the report now says how many parts of the code and rules it read, and how many rules pass, inside the "What is Coherence?" fold - extracted `howToFix(...)` and `whatThisMeans(...)` helpers for the repeated fold pattern - "at the registry top" reads "for the list of projects in `coherence.config.json`" - added a test for a recorded run that declares no rules (nothing about rules shown) - split `title()` into `heading()` (counted sections) and `label()` (colon-terminated sections) - the no-run message now reads "Coherence has no record of running its check (session `<id>`)<where>, so it did not check the rules." - the bugs fold is titled "What this means" since Coherence cannot promise a fix there - "close" stays for unchecked inputs; the broken-rules fold now says "Fix the code..." / "fix the rule in the spec" Updated the matching assertions in ci.test.ts and the spec bullet in Lifecycle.spec.md that described the old behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 4189fcb8-6fd5-428d-8c5f-4314d28e4dac

posthog

PostHog Review · Standard · reviewed 806d8e8: 1 Should fix.

… only when the check ran them A new code() helper writes a Markdown code span with a fence longer than any backtick run in the text, so a folder or file name cannot end the span early and add links to the comment. It replaces every dynamic code span, which also stops escape backslashes from showing inside a span (`ci\_7`). The "What is Coherence?" fold no longer says "0 rules pass" for a project whose rules the check did not run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 4189fcb8-6fd5-428d-8c5f-4314d28e4dac

pauldambra

Note

🤖 Automated comment by QA Swarm — not written by a human

QA Swarm round 2 review complete. See inline comments.

… projects A route starts where its entrance is declared, so "It reaches:" now lists route.stops after the first. Before, it removed the gap's own component, which differs from the first stop for a deeply nested folder. The routeReach comment now sits above routeReach. A new test checks the fold's counts across two projects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 4189fcb8-6fd5-428d-8c5f-4314d28e4dac

pauldambra self-assigned this Oct 25, 2026

posthog Bot reviewed Oct 25, 2026

View reviewed changes

Comment thread src/lifecycle/ci-report.ts Outdated

pauldambra changed the title ~~feat(ci): write the PR report in Simplified Technical English~~ feat(ci): make the PR report easy to read and scan (STE, folds, colour, icons) Oct 25, 2026

pauldambra marked this pull request as ready for review October 25, 2026 21:03

pauldambra commented Oct 25, 2026

View reviewed changes

posthog Bot reviewed Oct 25, 2026

View reviewed changes

Comment thread src/lifecycle/ci-report.ts Outdated

pauldambra commented Oct 25, 2026

View reviewed changes

Comment thread src/lifecycle/ci-report.ts

Comment thread src/lifecycle/ci-report.ts Outdated

Comment thread src/lifecycle/ci-report.ts

pauldambra merged commit 1aef204 into main Oct 25, 2026
19 checks passed

pauldambra deleted the posthog/ci-report-ste branch October 25, 2026 21:46

pauldambra mentioned this pull request Oct 25, 2026

chore: upgrade @posthog/coherence to 1.10.1 PostHog/posthog-js#5432

Open

4 of 5 tasks

Before	After
spec problem	spec file error
failing chokepoint	broken rule
spec gap	unchecked input
guard failure	bug that came back

Conversation

pauldambra commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

The posthog-js comment, after this change

Coherence

Testing

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pauldambra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pauldambra commented Oct 25, 2026

Verdict: ⚠️ REQUEST CHANGES (round 1 @ 4534f26)

Key findings

Convergence

Reviewer summaries

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pauldambra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pauldambra commented Oct 25, 2026 •

edited

Loading

Verdict: ⚠️ REQUEST CHANGES _{(round 1 @ 4534f26)}