From 56fcfb60859244cd373420ea5873186a49386be5 Mon Sep 17 00:00:00 2001 From: anandgupta42 Date: Wed, 22 Jul 2026 23:49:55 -0700 Subject: [PATCH 1/4] =?UTF-8?q?feat:=20[release]=20harden=20/release=20fro?= =?UTF-8?q?m=203-release=20retro=20=E2=80=94=20preflight=20script,=20fail-?= =?UTF-8?q?closed=20tags,=20publish=20gates?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Retro of the v0.8.10 / v0.9.1 / v0.9.2 release traces (.github/meta/release-retro-2026-07-22.md) found recurring failure classes; this change closes them: - `script/release-preflight.ts` (+34 unit tests): deterministic fail-closed gate — clean tree, HEAD == fresh `origin/main` (stage `pre`) or ancestor (stage `tag`), local+remote tag-collision check (never auto-delete), version sanity vs npm, prerelease-line ancestry, `release-blocker` PRs, `PREV_TAG` dry-run, marker guard with full output in one pass - `release.yml`: `publish-npm` now gated on the `test` job (npm could previously publish with tests red); tag format/SHA/CHANGELOG validated BEFORE publish; pre-publish smoke test asserts the exact tag version (a stale 0.7.3 binary once passed while releasing 0.9.2) - `/release` skill: preflight as Step 2 gate; three-step verified commit/tag + `git push --atomic origin HEAD:main v{V}`; stop-never-improvise on structural failures; deferred findings require durable issue links; P0-candidates verified or explicitly downgraded; exit-status integrity rule (no trailing status echoes); smoke-test the dist binary by path; every step visible in the summary table; personas deliver via SendMessage before idling - RELEASING.md: targeted staging instead of `git add -A`, atomic push, honest AUR-disabled / Docker-best-effort wording Co-Authored-By: Claude Fable 5 --- .claude/commands/release.md | 148 +++++-- .github/meta/release-retro-2026-07-22.md | 95 ++++ .github/workflows/release.yml | 46 +- docs/RELEASING.md | 39 +- script/release-preflight.test.ts | 179 ++++++++ script/release-preflight.ts | 539 +++++++++++++++++++++++ 6 files changed, 984 insertions(+), 62 deletions(-) create mode 100644 .github/meta/release-retro-2026-07-22.md create mode 100644 script/release-preflight.test.ts create mode 100644 script/release-preflight.ts diff --git a/.claude/commands/release.md b/.claude/commands/release.md index daae4e7533..a18a5ae71a 100644 --- a/.claude/commands/release.md +++ b/.claude/commands/release.md @@ -36,18 +36,25 @@ npm info @altimateai/altimate-code version Confirm with user: "Releasing **v{NEXT_VERSION}** (current: v{CURRENT_VERSION}). Proceed?" -## Step 2: Ensure on Main and Clean +## Step 2: Deterministic Preflight + +Run the preflight script — it is the gate, not a suggestion: ```bash -git branch --show-current -git status --short -git fetch origin main -git log HEAD..origin/main --oneline +bun script/release-preflight.ts --version {NEXT_VERSION} --stage pre ``` -- Must be on `main`. If not, stop. -- Working tree must be clean. If dirty, stop. -- Must be up to date with remote. If behind, stop. +It verifies, fail-closed: +- clean worktree (untracked files are only a warning — they must NOT be staged later) +- **HEAD equals freshly fetched `origin/main`** — this is the invariant, not "on branch main". Releasing from a worktree or a branch whose HEAD equals `origin/main` is fine; push via `HEAD:main`. If HEAD diverges, STOP. +- no local or remote tag `v{NEXT_VERSION}` already exists (stale fork-inherited tags nearly shipped a wrong artifact in v0.9.1) +- version is valid SemVer, greater than npm `latest`, not already published +- every prerelease tag on the target line (`v{X.Y}.*-*`) is an ancestor of `origin/main` +- no open PRs labeled `release-blocker` +- which previous tag the auto-generated release notes will compare against (sanity-check it: for v0.9.1 the old logic would have diffed against a stray divergent tag and omitted a 165-commit merge) +- marker guard clean + +**If preflight fails: STOP and report.** Do not improvise fixes for structural problems (diverged beta line, unmerged prerequisite PR, tag collision) inside this skill — resolve them as their own task, then re-invoke `/release`. Never auto-delete a colliding tag. If the user resolves a prerequisite by admin-merging a PR (bypassing branch protection), record that fact for the Step 12 summary. ## Step 3: Identify What Changed @@ -76,7 +83,7 @@ Gather: ### 4b: Launch the five-member evaluation team -Spawn **five agents in parallel**. Each agent MUST read the actual source code diffs — not just the summary. Each reviews from their unique perspective: +Spawn **five agents in parallel**, each with `model: "sonnet"` (a full-weight model per persona once exhausted the account's weekly usage budget mid-release). Each agent MUST read the actual source code diffs — not just the summary. Each reviews from their unique perspective: 1. **CTO** — Technical risk, security exposure, breaking changes, operational readiness. "Would I deploy this on a Friday?" @@ -88,6 +95,8 @@ Spawn **five agents in parallel**. Each agent MUST read the actual source code d 5. **Chaos Gremlin** — Random adversarial perspective (security auditor, support engineer, new hire, compliance officer, etc.). Different each release. Asks the uncomfortable question nobody else thought of. +Each persona prompt MUST end with: "Send your complete review via SendMessage to main as your FINAL action before going idle." (In the v0.9.2 release, 3 of 5 reviewers went idle without delivering and needed manual nudges.) + Each agent produces: ```markdown @@ -135,6 +144,7 @@ After all five agents complete: ### 4d: Gate - **Any P0** → Release blocked. Fix first, re-run from Step 3. +- **P0-candidate that can't be immediately confirmed** → it must be either empirically verified (build the repro, however awkward) or explicitly downgraded with written rationale that the user reviews. A suspected P0 never ships on indirect reasoning alone (v0.9.1 shipped a Chaos-Gremlin config-loss P0-candidate after two failed verification attempts — that's the one place a real P0 could have hidden). - **3+ HOLD verdicts** → Release blocked. - **Actionable P1s exist** → Proceed to Step 5 (fix them). - **No actionable items** → Skip Step 5, proceed to Step 6. @@ -157,9 +167,9 @@ echo "fix: {description}" > .github/meta/commit.txt git commit -F .github/meta/commit.txt ``` -### 5b: File issues for deferred items +### 5b: File issues for deferred items — mechanically enforced -For each deferred item, create a GitHub issue: +Every deferred item needs a **durable disposition** before the ship gate: a link to a GitHub issue (newly created, or an existing one that already covers it), or an explicit user opt-out recorded in the summary. Chat prose does not count — the v0.8.10 release deferred 3 findings that now exist only in a transcript nobody will reread. ```bash gh issue create --repo AltimateAI/altimate-code \ @@ -169,11 +179,11 @@ gh issue create --repo AltimateAI/altimate-code \ ### 5c: Re-verify after fixes -Run typecheck and marker guard to confirm fixes are clean: +Run typecheck and marker guard to confirm fixes are clean. Capture the marker guard's FULL output and fix ALL flagged sites in one pass, then re-run once to confirm zero — do not fix one hunk per run (v0.9.2 burned 7 serial guard rounds this way): ```bash bun turbo typecheck -bun run script/upstream/analyze.ts --markers --base main --strict +bun run script/upstream/analyze.ts --markers --base origin/main --strict ``` **If fixes introduced new issues,** fix those too. Loop until clean. @@ -207,6 +217,8 @@ cd packages/opencode && bun test --timeout 30000 test/skill/release-v{NEXT_VERSI cd packages/opencode && bun test --timeout 30000 ``` +When running the full suite in the background, tee complete output to a file (`bun test ... 2>&1 | tee /tmp/release-tests.log`) — a truncated background tail once forced re-running an 11,551-test suite just to learn which test failed. + ### 6c: Gate - **All pass** → Continue @@ -215,11 +227,15 @@ cd packages/opencode && bun test --timeout 30000 ## Step 7: UX Verification -### 7a: Smoke test +### 7a: Smoke test the ACTUAL release candidate + +Never smoke-test the `$PATH`-resolved `altimate` — in v0.9.2 it silently validated a stale globally-installed 0.7.3. Build and run the real artifact: ```bash -altimate --version -altimate --help +cd packages/opencode && bun run pre-release # builds dist + checks NAPI externals + binary starts +BINARY=$(find packages/opencode/dist -path '*bin/altimate*' -type f | head -1) +"$BINARY" --version # MUST print a version consistent with this release, not a stale install +"$BINARY" --help ``` ### 7b: Run feature-specific tests @@ -250,7 +266,7 @@ git log v{CURRENT_VERSION}..HEAD --oneline --no-merges ### 8b: Write the changelog entry -Categorize into **Added**, **Fixed**, **Changed**. Use bold title + em-dash description matching existing style. Incorporate release notes feedback from the Step 4 persona reviews. +Read `CHANGELOG.md` first, then edit (an Edit without a prior Read fails and wastes a round-trip). Categorize into **Added**, **Fixed**, **Changed**. Use bold title + em-dash description matching existing style. Incorporate release notes feedback from the Step 4 persona reviews. ### 8c: Review with user @@ -263,13 +279,15 @@ Wait for approval. Run all mandatory checks: ```bash -# Pre-release sanity (binary builds and starts) +# Pre-release sanity (binary builds and starts) — if not already run in Step 7a cd packages/opencode && bun run pre-release -# Marker guard -bun run script/upstream/analyze.ts --markers --base main --strict +# Full preflight again, now tolerating local release commits ahead of origin/main +bun script/release-preflight.ts --version {NEXT_VERSION} --stage tag ``` +**Exit-status integrity rule:** never background a gate command with a trailing status echo (`cmd > log; echo "exit: $?"` — the echo always exits 0 and once turned a FAILED pre-release build into a reported pass). Preserve the real exit code: run in foreground, or `cmd 2>&1 | tee log` and check `$pipestatus[1]`/`PIPESTATUS[0]`, and ALWAYS read the tool's own verdict line from the log. + **Gate: ALL CHECKS MUST PASS.** Stop on failure. ### Optional: Verdaccio sanity suite @@ -282,21 +300,35 @@ docker compose -f test/sanity/docker-compose.verdaccio.yml up \ --build --abort-on-container-exit --exit-code-from sanity ``` -If Docker unavailable, skip — CI will catch it. +If Docker unavailable, skip — but mark it ⏭️ in the Step 12 summary (it was once skipped silently). CI will catch it. + +## Step 10: Commit, Tag, Push — three verified steps, one atomic push -## Step 10: Commit, Tag, Push +Do NOT chain these with `&&` into one command (a mid-chain tag failure once committed but silently never pushed). Verify each stage: ```bash -# Stage changelog + any adversarial tests +# 1. Commit (targeted staging only — never `git add -A`, worktrees carry scratch files) git add CHANGELOG.md packages/opencode/test/skill/release-v{NEXT_VERSION}-adversarial.test.ts - -# Commit echo "release: v{NEXT_VERSION}" > .github/meta/commit.txt git commit -F .github/meta/commit.txt +git log -1 --oneline # verify the release commit is HEAD -# Tag and push +# 2. Tag — then VERIFY before pushing (a silently no-op'd `git tag` once pushed a +# stale tag pointing 2641 commits away from main; the push itself triggers publish, +# so post-push verification is too late) git tag v{NEXT_VERSION} -git push origin main v{NEXT_VERSION} +test "$(git rev-parse v{NEXT_VERSION})" = "$(git rev-parse HEAD)" || { echo "TAG MISMATCH — STOP"; } + +# 3. Push branch + tag atomically — both land or neither does +git push --atomic origin HEAD:main v{NEXT_VERSION} +``` + +If `git tag` reports the tag already exists: **STOP. Never delete-and-recreate.** Report what the existing tag points to (`git log -1 v{NEXT_VERSION}`) and let the user resolve. (Preflight should have caught this; if it didn't, something changed underneath you.) + +After the push, confirm the remote agrees: + +```bash +git ls-remote origin refs/tags/v{NEXT_VERSION} # must equal the release commit SHA ``` ## Step 11: Monitor CI @@ -306,7 +338,11 @@ gh run list --workflow=release.yml --repo AltimateAI/altimate-code --limit 1 gh run watch --repo AltimateAI/altimate-code ``` -If fails: `gh run view --repo AltimateAI/altimate-code --log-failed` +Post a one-line status update to the user on every CI state change (job started/failed/passed) — do not go silent until the user asks "done?" (this happened in two of the last three releases). + +If a job fails: `gh run view --repo AltimateAI/altimate-code --log-failed`. + +**Flaky re-runs need a paper trail.** If you re-run a failed job without a code change and it passes, link a tracking issue (existing or new) with the run URL before declaring the release done — the v0.9.2 Verdaccio "no internet" flake was waved through with zero tracking and will cost the next release the same diagnosis time. ## Step 12: Verify and Close Issues @@ -314,6 +350,7 @@ If fails: `gh run view --repo AltimateAI/altimate-code --log-failed` ```bash npm info @altimateai/altimate-code version +npm view @altimateai/altimate-code dist-tags # `latest` must be this release; `beta` untouched gh release view v{NEXT_VERSION} --repo AltimateAI/altimate-code --json tagName,publishedAt,assets ``` @@ -332,7 +369,8 @@ For each PR, also check: gh pr view {PR_NUMBER} --repo AltimateAI/altimate-code --json closingIssuesReferences --jq '.closingIssuesReferences[].number' ``` -For each open issue found, comment and close: +For each open issue found, comment and close. For originating bug reports that were **already closed** by a PR merge, still post a resolution comment naming the release (users watching the issue don't see PR merges): + ```bash gh issue comment {N} --repo AltimateAI/altimate-code \ --body "Resolved in [v{NEXT_VERSION}](https://github.com/AltimateAI/altimate-code/releases/tag/v{NEXT_VERSION})." @@ -341,24 +379,32 @@ gh issue close {N} --repo AltimateAI/altimate-code ### 12c: Release summary +Every step gets a row — a skipped step must show as ⏭️ with a reason, never silently disappear: + ``` ## Release Summary: v{NEXT_VERSION} | Check | Status | Details | |-------|--------|---------| | RELEASING.md | ✅ Read | | +| Preflight (pre) | ✅ | | | Code review | ✅ | {N} SHIP, {N} HOLD — {N} P0, {N} P1, {N} P2 | +| P0-candidates verified | ✅ / n/a | {how verified, or downgrade rationale} | | Issues fixed pre-release | {N} | {descriptions} | -| Issues deferred | {N} | Filed as #{numbers} | +| Deferred → filed | {N}/{N} | #{numbers} (every deferred item has an issue link or recorded opt-out) | | Adversarial tests | ✅ | {N}/{N} passed | -| UX verification | ✅ | {N} scenarios passed | +| UX verification (Step 7) | ✅ | smoke-tested dist binary, version {V} | +| Changelog approved | ✅ | | | Pre-release check | ✅ | | +| Preflight (tag) | ✅ | | | Verdaccio | ✅ / ⏭️ | | | Marker guard | ✅ | | -| CI workflow | ✅ | | -| npm | ✅ | v{NEXT_VERSION} | +| Tag verified (local+remote SHA) | ✅ | | +| CI workflow | ✅ | {flaky re-runs: issue links} | +| npm | ✅ | v{NEXT_VERSION}, dist-tags checked | | GitHub Release | ✅ | {link} | -| Issues closed | ✅ | {N} issues | +| Issues closed/commented | ✅ | {N} issues | +| Governance notes | — | {e.g. "prerequisite PR #964 admin-merged, branch protection bypassed"} | v{NEXT_VERSION} is live! No follow-up PRs needed. ``` @@ -369,17 +415,23 @@ v{NEXT_VERSION} is live! No follow-up PRs needed. 1. **Always read RELEASING.md first.** It is the source of truth for the process. 2. **Always confirm version with user.** Never auto-release without approval. -3. **Review BEFORE testing.** The multi-persona review finds design issues (stale docs, missing timeouts, naming problems). Adversarial tests find code bugs. Different tools for different problems. Review first, then test the reviewed code. -4. **Fix actionable issues before tagging.** The whole point of reviewing early is to ship clean. If the review finds a stale doc or missing timeout, fix it on main before the tag. No follow-up PRs for things that could have been fixed in 10 minutes. -5. **Only defer what truly can't be fixed quickly.** New features, large refactors, and design decisions get deferred. Missing timeouts, stale docs, wording fixes, and small guards get fixed now. -6. **Adversarial tests cover the FINAL code.** Tests run after Step 5 fixes, so they test the code that actually ships. -7. **Never skip pre-release check.** Last gate before a broken binary ships. -8. **Always use `--repo AltimateAI/altimate-code`** with `gh` commands. -9. **Only release from main.** Feature branches should not be tagged. -10. **Changelog entries must match existing style.** Bold titles with em-dash descriptions. -11. **If CI fails after push, do NOT delete the tag.** Investigate first. -12. **npm is the source of truth for versions.** -13. **PR template is mandatory.** PRs without exact headings get auto-closed. Create issue first, then PR with `Closes #N`. -14. **No `mock.module()` in adversarial tests.** Use `Dispatcher.register()`/`reset()` or `spyOn()`. -15. **Multi-persona evaluation is not optional.** The Chaos Gremlin persona must be different each release. -16. **The release is done when the summary says "No follow-up PRs needed."** If it can't say that, something was missed. +3. **The preflight script is the gate.** If `script/release-preflight.ts` fails, stop — structural problems (diverged beta line, tag collisions, unmerged prerequisites) get resolved as their own task, never improvised inside the release. +4. **Review BEFORE testing.** The multi-persona review finds design issues (stale docs, missing timeouts, naming problems). Adversarial tests find code bugs. Different tools for different problems. Review first, then test the reviewed code. +5. **Fix actionable issues before tagging.** If the review finds a stale doc or missing timeout, fix it before the tag. No follow-up PRs for things that could have been fixed in 10 minutes. +6. **Only defer what truly can't be fixed quickly** — and every deferred item gets a durable issue link (or recorded user opt-out) before the ship gate. Chat prose is not tracking. +7. **A P0-candidate is verified or explicitly downgraded with rationale — never shipped on indirect reasoning.** +8. **Adversarial tests cover the FINAL code.** Tests run after Step 5 fixes, so they test the code that actually ships. +9. **Never skip pre-release check.** Last gate before a broken binary ships. Smoke-test the freshly built dist binary by path — never the `$PATH`-resolved command. +10. **Never trust a wrapper exit code.** No trailing status echoes on gate commands; preserve real exit codes and read the tool's own verdict line. +11. **Tags fail closed.** Verify the tag doesn't exist (local AND remote) before creating; verify it points at HEAD before pushing; push branch + tag with `--atomic`; never delete-and-recreate a colliding tag. +12. **Targeted `git add` only.** Never `git add -A` — release worktrees carry scratch files. +13. **Always use `--repo AltimateAI/altimate-code`** with `gh` commands. +14. **Release from HEAD == fresh `origin/main`.** Being literally "on main" is not required (worktrees); HEAD equality after `git fetch` is. Push via `HEAD:main`. +15. **Changelog entries must match existing style.** Bold titles with em-dash descriptions. Read the file before editing it. +16. **If CI fails after push, do NOT delete the tag.** Investigate first. Flaky re-runs get a tracking-issue link. +17. **npm is the source of truth for versions.** +18. **PR template is mandatory.** PRs without exact headings get auto-closed. Create issue first, then PR with `Closes #N`. +19. **No `mock.module()` in adversarial tests.** Use `Dispatcher.register()`/`reset()` or `spyOn()`. +20. **Multi-persona evaluation is not optional.** Personas run on `model: "sonnet"`; the Chaos Gremlin persona must be different each release; every persona delivers its review via SendMessage before idling. +21. **Keep the user informed during waits.** Status update on every CI state change; long background reviews ping every ~20-30 minutes. +22. **The release is done when the summary says "No follow-up PRs needed."** If it can't say that, something was missed — and every table row above must be filled in, including the skipped ones. diff --git a/.github/meta/release-retro-2026-07-22.md b/.github/meta/release-retro-2026-07-22.md new file mode 100644 index 0000000000..66aea3bb82 --- /dev/null +++ b/.github/meta/release-retro-2026-07-22.md @@ -0,0 +1,95 @@ +# /release Skill — Robustness Improvements (retro of v0.8.10, v0.9.1, v0.9.2) + +Evidence: full session transcripts of the last three `/release` runs (v0.8.10 2026-06-23, v0.9.1 2026-07-08/09, v0.9.2 2026-07-21), git history between tags, and a Codex code-level review of the plan against `release.yml` / `publish.ts` / `pre-release-check.ts`. + +## What's working (keep) + +- **Multi-persona review earns its cost.** It caught the `release.yml` `PREV_TAG` bug that would have generated v0.9.1 release notes omitting the entire 165-commit upstream merge (CTO persona), and discovered the CI marker guard had been silently no-op'ing on PRs (bare `catch { return [] }` on a bad base ref). +- **Adversarial tests catch real bugs pre-tag.** v0.9.1: MCP annotation-hint classifier gap. v0.9.2: `transformSnowflakeBody` crash on non-object JSON roots. +- **Targeted `git add`** (v0.9.2) correctly avoided sweeping session scratch files into the release commit — RELEASING.md still says `git add -A` and should be fixed. + +--- + +## P0 — prevent shipping the wrong artifact + +### 1. Tag creation is unsafe; both stable releases hit tag collisions — fail closed +- v0.9.1: a stale local `v0.9.1` tag made `git tag` no-op; `git push origin v0.9.1` pushed a tag pointing at a 2641-commit-divergent commit. Only luck (the stale tree predated the current `release.yml`) prevented a wrong publish. +- v0.9.2: stale local `v0.9.2` inherited from upstream OpenCode fork tag history → reactive blind `git tag -d` + recreate. +- **Fix (Step 10):** + - Preflight `git rev-parse -q --verify refs/tags/v{V}` AND `git ls-remote origin refs/tags/v{V}`. On any collision: **stop and require explicit human resolution** (log what the old tag pointed to). Never auto-delete. + - After `git tag`: assert `git rev-parse v{V} == git rev-parse HEAD` **before** pushing — post-push verification is too late, the push itself triggers CI/publish. + - One-time repo policy (separate PR, not release behavior): prune fork-inherited upstream tags and set the upstream remote to `--no-tags`. + +### 2. Atomic push; commit/tag/push as separately verified steps +v0.9.2's single `commit && tag && push` chain committed, failed at tag, silently never pushed. Also, pushing `HEAD:main` and the tag as two pushes can leave the tag published against stale main if the branch push is rejected. +- **Fix:** verify commit; verify tag SHA; then `git push --atomic origin HEAD:main v{V}` so branch+tag land together or not at all. + +### 3. Gates must carry real exit status and test the real artifact +- v0.9.1: pre-release check backgrounded as `cmd > log 2>&1; echo "exit: $?"` — trailing echo turned a FAILED build into "exit 0". Caught only by reading the log. +- v0.9.2: Step 7 smoke test ran `$PATH`-resolved `altimate --version` → stale global `0.7.3`, not the candidate; never noticed in-session. +- **Fix:** preserve the actual exit code (`tee` + `PIPESTATUS`/`wait`), never append a status echo as the last statement; smoke-test the freshly built artifact by explicit path AND the npm-pack-installed wrapper, asserting the normalized version (`0.9.3`, no `v` — build strips it). Note: `pre-release-check.ts` injects workspace `NODE_PATH` (pre-release-check.ts:110), so it can pass for the wrong reason — a hermetic variant should exist. + +### 4. CI publish gate is broken at the source (found by Codex review, verified) +`publish-npm` needs only `[build, sanity-verdaccio]` — **not `test`** (release.yml:182-184). npm can publish while the test job is red. Additional CI gaps: +- Tag-format validation lives in `github-release`, which runs **after** `publish-npm` (release.yml:301) — validate tag syntax/SHA/changelog-entry/version **before** any publish job. +- Partial-publish recovery undefined: platform packages publish concurrently, wrapper after (publish.ts:125); a mid-run failure leaves immutable partial npm state and re-runs may fail on already-published versions. Make publish idempotent/resumable and document per-phase recovery. +- Docker/AUR/Homebrew publish errors are swallowed (publish.ts:192) while RELEASING.md promises they "happen automatically" — make them gated jobs or correct the doc. +- CI smoke tests don't assert the exact released version. + +**These are workflow/script PRs, not skill prose.** + +--- + +## P1 — codify reality: preflight as a deterministic script + +### 5. Replace Step 2's "must be on main" with the invariant that actually matters +All three releases deviated (main checked out in another worktree; released from a feature branch whose HEAD equaled origin/main). The correct branch-independent invariant: **after a fresh `git fetch`, clean HEAD == `origin/main`**; push via `HEAD:main`. If it doesn't hold, stop. + +### 6. Move the deterministic checks into `script/release/preflight.ts` +Six `fix: [release]` commits in one cycle (Bun pin, Verdaccio .dockerignore, version derivation, npm E413 de-dupe, smoke-test target, marker-guard base ref) = every release debugs the pipeline live. The skill's new Step 2 = run the script. Checks (objective invariants only): +- clean worktree; HEAD == fresh origin/main (per #5) — also covers the "user just merged a PR" staleness that forced a manual interruption in v0.9.2 +- tag collision, local + remote (per #1) — fail closed +- target version: valid SemVer, strictly greater than current npm `latest`, not already published (NOT equality with package.json — it intentionally says `1.17.9`) +- prerelease ancestry scoped to the target release line: e.g. releasing `v0.9.1` with `v0.9.0-beta.*` not an ancestor of origin/main → stop (v0.9.1's Step-2 stop turned into a 17-hour merge side-quest improvised via 3 AskUserQuestion rounds) +- open PRs labeled `release-blocker` → stop +- `PREV_TAG` dry-run: evaluate release.yml's previous-tag logic locally, print which tag release notes will diff against +- marker guard full run, complete violation list in one pass (v0.9.2 took 7 serial fix rounds for 11 hunks because output was consumed one hunk at a time) + +### 7. Prerequisites fail → stop; don't improvise +The release skill should not grow an upstream-merge/admin-bypass decision tree. If preflight fails on divergence, stop `/release`, do the prerequisite work as its own task, re-invoke. If the user directs an admin `--admin` merge to satisfy a prerequisite (as in v0.9.1), the Step 12 summary must disclose that branch protection was bypassed. + +--- + +## P1 — close the loop on review output + +### 8. Every deferred finding gets a durable disposition before the ship gate +v0.8.10 deferred 3 findings; zero filed — they exist only in a chat transcript. v0.9.1/v0.9.2 filed 6 and 4 (good). Gate: each deferred item has an issue link (new or existing) or an explicit user opt-out; the summary table gets a "Deferred → filed" row so an empty list is visible. Same rule for CI jobs waved through as "flaky" (v0.9.2's Verdaccio no-internet re-run was never tracked): link a durable issue, not necessarily a new one per flake. + +### 9. Unverified P0-candidates: validate or block — no approval-string bypass +v0.9.1's Chaos Gremlin flagged possible silent global-config loss; direct verification failed twice on tooling friction and the release shipped on indirect reasoning. Rule: a P0-candidate must be either empirically validated, or explicitly downgraded with written rationale reviewed by the user. It cannot ship as a P0 with a warning label. + +### 10. Step-skipping must be visible +Step 7 (UX smoke) was silently absorbed in v0.9.1 and mis-executed in v0.9.2; optional Verdaccio was skipped without mention. Every step gets a row in the Step 12c summary table (✅/⏭️/❌) so silent omission is structurally impossible. + +--- + +## P2 — agent ergonomics (real friction, but not release safeguards) + +- **Status on state changes:** both v0.8.10 (11.5-min silent CI wait) and v0.9.1 (104-min Codex review) ended with the user typing "done?". Post an update on every CI state change; long background reviews ping every ~20-30 min. +- **Persona delivery protocol:** 3 of 5 v0.9.2 reviewers went idle without delivering and needed manual nudges. Persona prompt must end: "Send your full review via SendMessage to main as your final action." +- **Shell hygiene:** explicit working directories instead of `cd X &&` chains (2 failures in v0.8.10); Read CHANGELOG.md before Edit (v0.8.10); background suite runs tee full output to a file (v0.9.2 re-ran an 11,551-test suite just to learn which test failed). +- **Step 12 scope:** include commenting the fix/version on the originating bug-report issue (v0.9.2 user had to ask separately). +- **Docs:** update RELEASING.md — targeted staging instead of `git add -A`; document the worktree/`HEAD:main` path; stop promising Docker/AUR/Homebrew until they're gated jobs. +- **Fan-out budget:** v0.9.1 hit the weekly usage limit mid-release (7+ concurrent heavy agents); run persona reviews on cheaper models and stagger heavy fan-out in long sessions. + +--- + +## Top 5 by robustness per unit of process (Codex-concurred) + +1. Fail-closed tag checks + SHA assertions (#1) +2. Atomic `HEAD:main`+tag push with per-step verification (#2) +3. Real exit-status handling + exact-artifact/version smoke test (#3) +4. Deterministic preflight script, objective invariants only (#6) +5. Branch-independent "HEAD == fresh origin/main" base check (#5) + +Plus one CI PR that outranks all skill prose: make `publish-npm` depend on `test` and move tag validation before publish (#4). diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d2ac388818..a8d064f49f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -181,7 +181,10 @@ jobs: publish-npm: name: Publish to npm - needs: [build, sanity-verdaccio] + # altimate_change — gate publishing on the test job. Previously publish-npm only + # needed [build, sanity-verdaccio], so npm could publish while typecheck/tests + # were red (found in the 2026-07-22 release retro). + needs: [test, build, sanity-verdaccio] runs-on: ubuntu-latest timeout-minutes: 60 permissions: @@ -189,6 +192,33 @@ jobs: steps: - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + # altimate_change — validate the tag BEFORE any publish work. Tag-format + # validation previously lived in github-release, which runs AFTER publish-npm, + # so a malformed tag could publish to npm and only fail afterwards. + - name: Validate release tag + run: | + if ! echo "$CURRENT_TAG" | grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9._-]+)?$'; then + echo "::error::Invalid tag format: $CURRENT_TAG — refusing to publish" + exit 1 + fi + # Shallow checkouts may not carry the tag ref locally — fetch it so the + # comparison runs against what origin actually has. + git fetch --no-tags --depth 1 origin "refs/tags/$CURRENT_TAG:refs/tags/$CURRENT_TAG" 2>/dev/null || true + TAG_SHA=$(git rev-parse "refs/tags/$CURRENT_TAG^{commit}") + HEAD_SHA=$(git rev-parse HEAD) + if [ "$TAG_SHA" != "$HEAD_SHA" ]; then + echo "::error::Tag $CURRENT_TAG points at $TAG_SHA but workflow checked out $HEAD_SHA" + exit 1 + fi + VERSION="${CURRENT_TAG#v}" + if ! grep -q "\[$VERSION\]" CHANGELOG.md && [ "${CURRENT_TAG#*-}" = "$CURRENT_TAG" ]; then + echo "::error::CHANGELOG.md has no entry for $VERSION — stable releases require a changelog entry" + exit 1 + fi + echo "Tag validation passed: $CURRENT_TAG @ $TAG_SHA" + env: + CURRENT_TAG: ${{ github.ref_name }} + - uses: oven-sh/setup-bun@ecf28ddc73e819eb6fa29df6b34ef8921c743461 # v2 with: bun-version: "1.3.14" @@ -276,9 +306,19 @@ jobs: # comment for why this matters. The binary must start without # walking the workspace for node_modules. cd "${RUNNER_TEMP:-/tmp}" - env -u NODE_PATH "$BINARY" --version - echo "Pre-publish smoke test passed" + # altimate_change — assert the EXACT version, not just "it starts". + # The 2026-07-22 release retro found a smoke test once validated a + # stale binary (0.7.3) while releasing 0.9.2. + REPORTED=$(env -u NODE_PATH "$BINARY" --version) + EXPECTED="${CURRENT_TAG#v}" + if [ "$REPORTED" != "$EXPECTED" ]; then + echo "::error::Binary reports version '$REPORTED' but tag says '$EXPECTED'" + exit 1 + fi + echo "Pre-publish smoke test passed ($REPORTED)" fi + env: + CURRENT_TAG: ${{ github.ref_name }} - name: Publish to npm run: bun run packages/opencode/script/publish.ts diff --git a/docs/RELEASING.md b/docs/RELEASING.md index 3758c394ba..16c7072e3a 100644 --- a/docs/RELEASING.md +++ b/docs/RELEASING.md @@ -48,40 +48,57 @@ Add a new section at the top of `CHANGELOG.md`: - ... ``` -### 2. Run pre-release sanity check +### 2. Run pre-release sanity checks -**MANDATORY** — this catches broken binaries before they reach users: +**MANDATORY** — these catch broken releases before they reach users: ```bash +# Deterministic preflight: clean tree, HEAD == fresh origin/main, no tag +# collisions (local OR remote), version sanity vs npm, prerelease-line +# ancestry, release-blocker PRs, PREV_TAG dry-run, marker guard. +bun script/release-preflight.ts --version 0.5.0 --stage pre + cd packages/opencode bun run pre-release ``` -This verifies: +`pre-release` verifies: - All required NAPI externals are in `package.json` dependencies - They're installed in `node_modules` - A local build produces a binary that actually starts -Do NOT proceed if any check fails. +Do NOT proceed if any check fails. If the preflight reports a tag collision, +resolve it explicitly — never delete-and-recreate a tag blind. ### 3. Commit and tag +Stage files **individually** — never `git add -A` (release worktrees carry +scratch files that must not ship in the release commit). Releasing does not +require being literally on the `main` branch: the invariant is that a clean +HEAD equals freshly fetched `origin/main` (worktrees push via `HEAD:main`). + ```bash -git add -A +git add CHANGELOG.md git commit -m "release: v0.5.0" git tag v0.5.0 -git push origin main v0.5.0 +# Verify the tag points at HEAD BEFORE pushing — a pre-existing tag makes +# `git tag` fail, and pushing a stale tag triggers a publish of old code. +test "$(git rev-parse v0.5.0)" = "$(git rev-parse HEAD)" +# Atomic: branch and tag land together or not at all. +git push --atomic origin HEAD:main v0.5.0 ``` ### 4. What happens automatically The `v*` tag triggers `.github/workflows/release.yml` which: -1. **Builds** all platform binaries (linux/darwin/windows, x64/arm64) -2. **Publishes to npm** — platform-specific binary packages + wrapper package -3. **Creates GitHub Release** — with auto-generated release notes and binary attachments -4. **Updates AUR** — pushes PKGBUILD update to `altimate-code-bin` -5. **Publishes Docker image** — to `ghcr.io/altimateai/altimate-code` +1. **Runs release-critical tests** — typecheck + branding/install tests (gates npm publish) +2. **Builds** all platform binaries (linux/darwin/windows, x64/arm64) +3. **Validates the tag** — format, tag-SHA == checked-out SHA, CHANGELOG entry present (before any publish) +4. **Publishes to npm** — platform-specific binary packages + wrapper package +5. **Creates GitHub Release** — with auto-generated release notes and binary attachments +6. **Publishes Docker image (best-effort)** — to `ghcr.io/altimateai/altimate-code`; failures are logged but do NOT fail the release, so verify manually if you need the image +7. ~~Updates AUR~~ — currently **disabled** (the workflow step is commented out; see `publish.ts` for setup steps to re-enable) ### 5. Verify diff --git a/script/release-preflight.test.ts b/script/release-preflight.test.ts new file mode 100644 index 0000000000..360f1ba364 --- /dev/null +++ b/script/release-preflight.test.ts @@ -0,0 +1,179 @@ +import { describe, test, expect } from "bun:test" +import { + normalizeVersion, + parseSemver, + compareSemver, + isStableTagName, + selectPrevTag, + prereleaseLineTags, +} from "./release-preflight" + +describe("normalizeVersion", () => { + test("strips a leading lowercase v", () => { + expect(normalizeVersion("v0.9.3")).toBe("0.9.3") + }) + + test("strips a leading uppercase V", () => { + expect(normalizeVersion("V0.9.3")).toBe("0.9.3") + }) + + test("leaves a version with no leading v untouched", () => { + expect(normalizeVersion("0.9.3")).toBe("0.9.3") + }) + + test("leaves a prerelease version untouched aside from the v", () => { + expect(normalizeVersion("v0.9.3-beta.1")).toBe("0.9.3-beta.1") + }) +}) + +describe("parseSemver", () => { + test("parses a stable version", () => { + expect(parseSemver("0.9.3")).toEqual({ major: 0, minor: 9, patch: 3, prerelease: null }) + }) + + test("parses a prerelease version", () => { + expect(parseSemver("0.9.3-beta.1")).toEqual({ major: 0, minor: 9, patch: 3, prerelease: "beta.1" }) + }) + + test("parses multi-digit components", () => { + expect(parseSemver("12.34.567")).toEqual({ major: 12, minor: 34, patch: 567, prerelease: null }) + }) + + test("rejects a version with a leading v (call normalizeVersion first)", () => { + expect(parseSemver("v0.9.3")).toBeNull() + }) + + test("rejects a missing patch component", () => { + expect(parseSemver("0.9")).toBeNull() + }) + + test("rejects non-numeric components", () => { + expect(parseSemver("0.9.x")).toBeNull() + }) + + test("rejects garbage input", () => { + expect(parseSemver("not-a-version")).toBeNull() + expect(parseSemver("")).toBeNull() + }) +}) + +describe("compareSemver", () => { + const v = (s: string) => parseSemver(s)! + + test("orders by major first", () => { + expect(compareSemver(v("1.0.0"), v("0.9.9"))).toBeGreaterThan(0) + expect(compareSemver(v("0.9.9"), v("1.0.0"))).toBeLessThan(0) + }) + + test("orders by minor when major is equal", () => { + expect(compareSemver(v("0.10.0"), v("0.9.0"))).toBeGreaterThan(0) + }) + + test("orders by patch when major.minor is equal", () => { + expect(compareSemver(v("0.9.2"), v("0.9.1"))).toBeGreaterThan(0) + }) + + test("equal versions compare equal", () => { + expect(compareSemver(v("0.9.2"), v("0.9.2"))).toBe(0) + }) + + test("a stable version is greater than a prerelease of the same major.minor.patch", () => { + expect(compareSemver(v("0.9.2"), v("0.9.2-beta.1"))).toBeGreaterThan(0) + expect(compareSemver(v("0.9.2-beta.1"), v("0.9.2"))).toBeLessThan(0) + }) + + test("equal prerelease strings compare equal", () => { + expect(compareSemver(v("0.9.2-beta.1"), v("0.9.2-beta.1"))).toBe(0) + }) + + test("prerelease strings compare lexically", () => { + expect(compareSemver(v("0.9.2-beta.2"), v("0.9.2-beta.10"))).toBeGreaterThan(0) // "beta.2" > "beta.10" lexically + }) + + test("major/minor/patch precedence beats prerelease presence", () => { + // A higher patch prerelease still beats a lower patch stable. + expect(compareSemver(v("0.9.3-beta.1"), v("0.9.2"))).toBeGreaterThan(0) + }) +}) + +describe("isStableTagName", () => { + test("accepts vX.Y.Z", () => { + expect(isStableTagName("v0.9.2")).toBe(true) + expect(isStableTagName("v12.34.567")).toBe(true) + }) + + test("rejects prerelease tags", () => { + expect(isStableTagName("v0.9.2-beta.1")).toBe(false) + }) + + test("rejects tags without a leading v", () => { + expect(isStableTagName("0.9.2")).toBe(false) + }) + + test("rejects malformed tags", () => { + expect(isStableTagName("v0.9")).toBe(false) + expect(isStableTagName("v0.9.2.1")).toBe(false) + expect(isStableTagName("release-0.9.2")).toBe(false) + }) +}) + +describe("selectPrevTag", () => { + test("picks the highest stable tag below the current one", () => { + const tags = ["v0.9.0", "v0.9.1", "v0.9.2", "v0.8.10"] + expect(selectPrevTag(tags, "v0.9.2")).toBe("v0.9.1") + }) + + test("excludes the current tag even if present in the merged list", () => { + const tags = ["v0.9.2", "v0.9.1"] + expect(selectPrevTag(tags, "v0.9.2")).toBe("v0.9.1") + }) + + test("ignores prerelease tags — the v0.9.1 PREV_TAG bug", () => { + // Without this filter, a stray/divergent prerelease tag with a + // lexically-higher name could win a naive string sort. + const tags = ["v0.9.0", "v0.9.1-beta.5", "v0.9.1-beta.9"] + expect(selectPrevTag(tags, "v0.9.2")).toBe("v0.9.0") + }) + + test("ignores non-semver / malformed tag names", () => { + const tags = ["v0.9.0", "not-a-tag", "v0.9"] + expect(selectPrevTag(tags, "v0.9.1")).toBe("v0.9.0") + }) + + test("returns null when no eligible previous tag exists", () => { + expect(selectPrevTag([], "v0.1.0")).toBeNull() + expect(selectPrevTag(["v0.1.0"], "v0.1.0")).toBeNull() + }) + + test("sorts by semantic version, not lexical string order", () => { + // Lexically, "v0.10.0" < "v0.9.0" is false as a string compare in the + // wrong direction people expect — this asserts we sort numerically. + const tags = ["v0.9.0", "v0.10.0", "v0.2.0"] + expect(selectPrevTag(tags, "v0.11.0")).toBe("v0.10.0") + }) +}) + +describe("prereleaseLineTags", () => { + test("matches prerelease tags for the given major.minor", () => { + const tags = ["v0.9.0-beta.1", "v0.9.0-beta.2", "v0.9.1", "v0.8.5-beta.1"] + expect(prereleaseLineTags(tags, 0, 9)).toEqual(["v0.9.0-beta.1", "v0.9.0-beta.2"]) + }) + + test("does not match stable tags", () => { + expect(prereleaseLineTags(["v0.9.1", "v0.9.2"], 0, 9)).toEqual([]) + }) + + test("does not match a different major.minor line", () => { + expect(prereleaseLineTags(["v0.8.5-beta.1", "v1.0.0-beta.1"], 0, 9)).toEqual([]) + }) + + test("matches across different patch numbers within the same line", () => { + const tags = ["v0.9.0-beta.1", "v0.9.3-beta.1", "v0.9.10-rc.1"] + expect(prereleaseLineTags(tags, 0, 9)).toEqual(tags) + }) + + test("returns empty array for no matches", () => { + expect(prereleaseLineTags([], 0, 9)).toEqual([]) + expect(prereleaseLineTags(["v1.0.0"], 0, 9)).toEqual([]) + }) +}) diff --git a/script/release-preflight.ts b/script/release-preflight.ts new file mode 100644 index 0000000000..110609cdee --- /dev/null +++ b/script/release-preflight.ts @@ -0,0 +1,539 @@ +#!/usr/bin/env bun +/** + * Release Preflight — deterministic gate for `/release` and `/release-beta`. + * + * Replaces LLM-improvised release judgment calls with objective, scriptable + * checks. Born from `.github/meta/release-retro-2026-07-22.md`: three straight + * releases (v0.8.10, v0.9.1, v0.9.2) each hit at least one of these failure + * modes live — a stale local tag that nearly published a 2641-commit-divergent + * artifact, releasing with HEAD behind origin/main, an unmerged prerelease line + * turning into a 17-hour merge side-quest, and marker-guard violations + * surfacing one at a time across 7 serial fix rounds. + * + * Usage: + * bun script/release-preflight.ts --version 0.9.3 --stage pre + * bun script/release-preflight.ts --version 0.9.3 --stage tag + * bun script/release-preflight.ts --version 0.9.3-beta.1 --stage pre --allow-prerelease + * + * Exit codes: + * 0 — all checks PASS (WARN/SKIP allowed) + * 1 — at least one check FAILed + */ + +import { parseArgs } from "util" +import path from "path" +import { fileURLToPath } from "url" + +const __filename = fileURLToPath(import.meta.url) +const __dirname = path.dirname(__filename) + +// --------------------------------------------------------------------------- +// Pure helpers — semver + tag-selection logic. No git/npm/gh calls in here so +// they can be unit tested deterministically (see release-preflight.test.ts). +// --------------------------------------------------------------------------- + +export interface Semver { + major: number + minor: number + patch: number + prerelease: string | null +} + +/** Strip a leading "v" from a version/tag string. */ +export function normalizeVersion(input: string): string { + return input.startsWith("v") || input.startsWith("V") ? input.slice(1) : input +} + +/** + * Parse a normalized (no leading "v") SemVer string into its components. + * Returns null if the string isn't valid SemVer (major.minor.patch[-prerelease]). + */ +export function parseSemver(version: string): Semver | null { + const m = /^(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z-.]+))?$/.exec(version) + if (!m) return null + return { + major: Number(m[1]), + minor: Number(m[2]), + patch: Number(m[3]), + prerelease: m[4] ?? null, + } +} + +/** + * Compare two parsed SemVer values. Returns <0 if a0 if a>b. + * A stable version is always greater than any prerelease of the same + * major.minor.patch (per SemVer precedence rules), and prerelease strings + * compare lexically (good enough for our `-beta.N` convention; we don't need + * full dot-identifier numeric-vs-alphanumeric precedence for this repo). + */ +export function compareSemver(a: Semver, b: Semver): number { + if (a.major !== b.major) return a.major - b.major + if (a.minor !== b.minor) return a.minor - b.minor + if (a.patch !== b.patch) return a.patch - b.patch + if (a.prerelease === b.prerelease) return 0 + if (a.prerelease === null) return 1 // stable > prerelease + if (b.prerelease === null) return -1 + return a.prerelease < b.prerelease ? -1 : a.prerelease > b.prerelease ? 1 : 0 +} + +/** True if the tag name matches a stable release tag: v... */ +export function isStableTagName(tag: string): boolean { + return /^v[0-9]+\.[0-9]+\.[0-9]+$/.test(tag) +} + +/** + * Replicates release.yml's PREV_TAG selection: + * git tag --merged HEAD --sort=-version:refname + * | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' + * | grep -vx "$CURRENT_TAG" + * | head -1 + * + * Takes the raw (unsorted, unfiltered) list of tags already known to be + * merged into HEAD — the git call that produces that list belongs in main(), + * not here — and does the stable-filter + version-sort + exclude-current + + * take-first purely, so it's testable without git. + */ +export function selectPrevTag(mergedTags: string[], currentTag: string): string | null { + const stable = mergedTags.filter((t) => isStableTagName(t) && t !== currentTag) + const parsed = stable + .map((tag) => ({ tag, v: parseSemver(normalizeVersion(tag)) })) + .filter((x): x is { tag: string; v: Semver } => x.v !== null) + parsed.sort((a, b) => compareSemver(b.v, a.v)) // descending + return parsed.length > 0 ? parsed[0].tag : null +} + +/** + * Filter tags to the "prerelease line" for a given major.minor: tags of the + * shape vMAJOR.MINOR.PATCH-anything. Used by the ancestry check — releasing + * v0.9.1 must first confirm every v0.9.*-beta.* tag merged into origin/main. + */ +export function prereleaseLineTags(allTags: string[], major: number, minor: number): string[] { + const re = new RegExp(`^v${major}\\.${minor}\\.[0-9]+-.+$`) + return allTags.filter((t) => re.test(t)) +} + +// --------------------------------------------------------------------------- +// Thin runner — every external process call goes through here so main() +// stays readable and the pure logic above stays untestable-process-free. +// --------------------------------------------------------------------------- + +interface RunResult { + code: number + stdout: string + stderr: string +} + +function repoRoot(): string { + return path.resolve(__dirname, "..") +} + +/** Run a command, never throwing — callers inspect `.code`. */ +function run(cmd: string[]): RunResult { + const proc = Bun.spawnSync(cmd, { + cwd: repoRoot(), + stdout: "pipe", + stderr: "pipe", + }) + return { + code: proc.exitCode ?? 1, + stdout: proc.stdout ? proc.stdout.toString("utf-8").trim() : "", + stderr: proc.stderr ? proc.stderr.toString("utf-8").trim() : "", + } +} + +function git(args: string[]): RunResult { + return run(["git", ...args]) +} + +// --------------------------------------------------------------------------- +// Output formatting +// --------------------------------------------------------------------------- + +const RESET = "\x1b[0m" +const BOLD = "\x1b[1m" +const DIM = "\x1b[2m" +const RED = "\x1b[31m" +const GREEN = "\x1b[32m" +const YELLOW = "\x1b[33m" +const CYAN = "\x1b[36m" + +type Status = "PASS" | "FAIL" | "WARN" | "SKIP" + +interface CheckResult { + name: string + status: Status + detail: string +} + +const STATUS_COLOR: Record = { + PASS: GREEN, + FAIL: RED, + WARN: YELLOW, + SKIP: DIM, +} + +function printCheck(result: CheckResult): void { + const color = STATUS_COLOR[result.status] + console.log(` ${color}${BOLD}${result.status.padEnd(4)}${RESET} ${BOLD}${result.name}${RESET}`) + for (const line of result.detail.split("\n")) { + if (line.length > 0) console.log(` ${DIM}${line}${RESET}`) + } +} + +function banner(text: string): void { + const line = "─".repeat(70) + console.log(`\n${CYAN}${line}${RESET}`) + console.log(`${CYAN} ${BOLD}${text}${RESET}`) + console.log(`${CYAN}${line}${RESET}\n`) +} + +// --------------------------------------------------------------------------- +// CLI +// --------------------------------------------------------------------------- + +function printUsage(): void { + console.log(` + ${BOLD}Release Preflight${RESET} — deterministic gate for /release and /release-beta + + ${BOLD}USAGE${RESET} + bun script/release-preflight.ts --version --stage [--allow-prerelease] + + ${BOLD}OPTIONS${RESET} + --version Target release version (required) + --stage pre: HEAD must equal fresh origin/main (default) + tag: origin/main must be an ancestor of HEAD + --allow-prerelease Permit a "-beta.N" target (for /release-beta) + --help, -h Show this help message + + ${BOLD}EXIT CODES${RESET} + 0 all checks PASS (WARN/SKIP allowed) + 1 at least one check FAILed +`) +} + +async function main(): Promise { + const { values: args } = parseArgs({ + options: { + version: { type: "string" }, + stage: { type: "string", default: "pre" }, + "allow-prerelease": { type: "boolean", default: false }, + help: { type: "boolean", short: "h", default: false }, + }, + strict: false, + }) as { values: Record } + + if (args.help) { + printUsage() + process.exit(0) + } + + if (!args.version) { + console.error(`${RED}error${RESET} --version is required (e.g. --version 0.9.3)`) + printUsage() + process.exit(1) + } + + const stage = args.stage === "tag" ? "tag" : "pre" + const allowPrerelease = Boolean(args["allow-prerelease"]) + const rawVersion: string = args.version + const version = normalizeVersion(rawVersion) + const targetTag = `v${version}` + + banner(`Release Preflight — ${targetTag} (stage: ${stage})`) + + const results: CheckResult[] = [] + const add = (r: CheckResult) => { + results.push(r) + printCheck(r) + } + + // ── 1. fetch ────────────────────────────────────────────────────────────── + const fetchResult = git(["fetch", "origin", "main", "--tags"]) + if (fetchResult.code !== 0) { + add({ name: "fetch", status: "FAIL", detail: `git fetch origin main --tags failed:\n${fetchResult.stderr}` }) + } else { + add({ name: "fetch", status: "PASS", detail: "origin/main + tags fetched" }) + } + + // ── 2. clean worktree ──────────────────────────────────────────────────── + const statusResult = git(["status", "--porcelain"]) + const statusLines = statusResult.stdout.split("\n").filter((l) => l.length > 0) + const tracked = statusLines.filter((l) => !l.startsWith("??")) + const untracked = statusLines.filter((l) => l.startsWith("??")) + if (tracked.length > 0) { + add({ + name: "clean worktree", + status: "FAIL", + detail: `tracked modifications/staged changes present:\n${tracked.join("\n")}`, + }) + } else if (untracked.length > 0) { + add({ + name: "clean worktree", + status: "WARN", + detail: `untracked files present (will not be committed):\n${untracked.join("\n")}`, + }) + } else { + add({ name: "clean worktree", status: "PASS", detail: "no tracked or untracked changes" }) + } + + // ── 3. base ────────────────────────────────────────────────────────────── + const headSha = git(["rev-parse", "HEAD"]).stdout + const originMainSha = git(["rev-parse", "origin/main"]).stdout + if (stage === "pre") { + if (headSha && originMainSha && headSha === originMainSha) { + add({ name: "base", status: "PASS", detail: `HEAD == origin/main (${headSha})` }) + } else { + add({ + name: "base", + status: "FAIL", + detail: `HEAD (${headSha || "unknown"}) != origin/main (${originMainSha || "unknown"}) — stage 'pre' requires exact equality`, + }) + } + } else { + const ancestor = git(["merge-base", "--is-ancestor", "origin/main", "HEAD"]) + if (ancestor.code === 0) { + add({ + name: "base", + status: "PASS", + detail: `origin/main (${originMainSha}) is an ancestor of HEAD (${headSha}) — stage 'tag' allows local release commits ahead`, + }) + } else { + add({ + name: "base", + status: "FAIL", + detail: `origin/main (${originMainSha || "unknown"}) is NOT an ancestor of HEAD (${headSha || "unknown"})`, + }) + } + } + + // ── 4. tag collision (fail closed, never delete) ──────────────────────── + { + const localTag = git(["rev-parse", "-q", "--verify", `refs/tags/${targetTag}`]) + const localExists = localTag.code === 0 && localTag.stdout.length > 0 + const remoteLs = git(["ls-remote", "origin", `refs/tags/${targetTag}`]) + const remoteExists = remoteLs.code === 0 && remoteLs.stdout.trim().length > 0 + + if (!localExists && !remoteExists) { + add({ name: "tag collision", status: "PASS", detail: `${targetTag} does not exist locally or on origin` }) + } else { + const parts: string[] = [] + if (localExists) { + const sha = localTag.stdout + const info = git(["log", "-1", "--format=%aI %s", sha]) + parts.push(`LOCAL ${targetTag} -> ${sha} (${info.stdout || "unknown date/subject"})`) + } + if (remoteExists) { + const remoteSha = remoteLs.stdout.split("\t")[0] + parts.push(`REMOTE origin/${targetTag} -> ${remoteSha}`) + } + parts.push( + `${targetTag} already exists — resolve explicitly (inspect what it points to, decide whether to ` + + `retarget or pick a different version). This script will never delete a tag for you.`, + ) + add({ name: "tag collision", status: "FAIL", detail: parts.join("\n") }) + } + } + + // ── 5. version sanity ──────────────────────────────────────────────────── + { + const parsed = parseSemver(version) + if (!parsed) { + add({ name: "version sanity", status: "FAIL", detail: `"${rawVersion}" is not valid SemVer (X.Y.Z[-prerelease])` }) + } else if (parsed.prerelease && !allowPrerelease) { + add({ + name: "version sanity", + status: "FAIL", + detail: `"${version}" is a prerelease — pass --allow-prerelease for /release-beta targets`, + }) + } else { + const npmView = run(["npm", "view", "@altimateai/altimate-code", "version"]) + if (npmView.code !== 0) { + add({ + name: "version sanity", + status: "WARN", + detail: `could not reach npm to check current 'latest' version:\n${npmView.stderr || npmView.stdout}`, + }) + } else { + const currentLatest = npmView.stdout.trim() + const currentParsed = parseSemver(normalizeVersion(currentLatest)) + const alreadyPublished = run(["npm", "view", `@altimateai/altimate-code@${version}`, "version"]) + const isPublished = alreadyPublished.code === 0 && alreadyPublished.stdout.trim().length > 0 + + if (isPublished) { + add({ + name: "version sanity", + status: "FAIL", + detail: `@altimateai/altimate-code@${version} is already published on npm`, + }) + } else if (!currentParsed) { + add({ + name: "version sanity", + status: "WARN", + detail: `could not parse current npm 'latest' version "${currentLatest}" to compare`, + }) + } else if (compareSemver(parsed, currentParsed) <= 0) { + add({ + name: "version sanity", + status: "FAIL", + detail: `${version} is not strictly greater than current npm latest (${currentLatest})`, + }) + } else { + add({ + name: "version sanity", + status: "PASS", + detail: `${version} > current latest (${currentLatest}); not yet published; ${parsed.prerelease ? "prerelease allowed" : "stable"}`, + }) + } + } + } + } + + // ── 6. prerelease ancestry (scoped to target release line) ────────────── + { + const parsed = parseSemver(version) + if (!parsed) { + add({ name: "prerelease ancestry", status: "SKIP", detail: "skipped — target version failed to parse" }) + } else { + const allTagsResult = git(["tag", "--list"]) + const allTags = allTagsResult.stdout.split("\n").filter((t) => t.length > 0) + const lineTags = prereleaseLineTags(allTags, parsed.major, parsed.minor).filter((t) => t !== targetTag) + + if (lineTags.length === 0) { + add({ + name: "prerelease ancestry", + status: "PASS", + detail: `no v${parsed.major}.${parsed.minor}.*-* prerelease tags exist for this release line`, + }) + } else { + const offenders: string[] = [] + for (const tag of lineTags) { + const ancestor = git(["merge-base", "--is-ancestor", tag, "origin/main"]) + if (ancestor.code !== 0) offenders.push(tag) + } + if (offenders.length === 0) { + add({ + name: "prerelease ancestry", + status: "PASS", + detail: `all ${lineTags.length} prerelease tag(s) for v${parsed.major}.${parsed.minor}.* are merged into origin/main:\n${lineTags.join(", ")}`, + }) + } else { + add({ + name: "prerelease ancestry", + status: "FAIL", + detail: `prerelease tag(s) NOT merged into origin/main — merge before releasing this line:\n${offenders.join("\n")}`, + }) + } + } + } + } + + // ── 7. release-blocker PRs ─────────────────────────────────────────────── + { + const gh = run([ + "gh", + "pr", + "list", + "--repo", + "AltimateAI/altimate-code", + "--label", + "release-blocker", + "--state", + "open", + "--json", + "number,title", + ]) + if (gh.code !== 0) { + add({ name: "release-blocker PRs", status: "WARN", detail: `gh unavailable or errored:\n${gh.stderr || gh.stdout}` }) + } else { + let prs: { number: number; title: string }[] = [] + try { + prs = JSON.parse(gh.stdout || "[]") + } catch { + add({ name: "release-blocker PRs", status: "WARN", detail: `could not parse gh output:\n${gh.stdout}` }) + prs = [] + } + if (prs.length > 0) { + add({ + name: "release-blocker PRs", + status: "FAIL", + detail: prs.map((p) => `#${p.number} ${p.title}`).join("\n"), + }) + } else { + add({ name: "release-blocker PRs", status: "PASS", detail: "no open release-blocker PRs" }) + } + } + } + + // ── 8. PREV_TAG dry-run ────────────────────────────────────────────────── + { + const mergedResult = git(["tag", "--merged", "HEAD", "--sort=-version:refname"]) + const mergedTags = mergedResult.stdout.split("\n").filter((t) => t.length > 0) + const prevTag = selectPrevTag(mergedTags, targetTag) + + const allTagsResult = git(["tag", "--list"]) + const anyStableTagExists = allTagsResult.stdout + .split("\n") + .filter((t) => t.length > 0) + .some(isStableTagName) + + if (prevTag) { + add({ + name: "PREV_TAG dry-run", + status: "PASS", + detail: `release notes will diff against ${prevTag} (compare: ${prevTag}...${targetTag})`, + }) + } else if (anyStableTagExists) { + add({ + name: "PREV_TAG dry-run", + status: "FAIL", + detail: + `no previous tag found merged into HEAD, but stable tags exist in the repo — ` + + `release notes would silently omit history (the v0.9.1 bug). Check for a diverged HEAD ` + + `or an unmerged release branch.`, + }) + } else { + add({ name: "PREV_TAG dry-run", status: "PASS", detail: "no stable tags exist yet — initial release, no previous tag" }) + } + } + + // ── 9. marker guard ────────────────────────────────────────────────────── + { + const guard = run(["bun", "run", "script/upstream/analyze.ts", "--markers", "--base", "origin/main", "--strict"]) + const fullOutput = [guard.stdout, guard.stderr].filter((s) => s.length > 0).join("\n") + if (guard.code !== 0) { + add({ name: "marker guard", status: "FAIL", detail: fullOutput || `exited ${guard.code} with no output` }) + } else { + add({ name: "marker guard", status: "PASS", detail: fullOutput || "no unmarked upstream-shared changes" }) + } + } + + // ── summary ─────────────────────────────────────────────────────────────── + banner("Summary") + const width = Math.max(...results.map((r) => r.name.length)) + 2 + for (const r of results) { + console.log(` ${STATUS_COLOR[r.status]}${BOLD}${r.status.padEnd(4)}${RESET} ${r.name.padEnd(width)}`) + } + console.log() + + const failed = results.filter((r) => r.status === "FAIL") + const warned = results.filter((r) => r.status === "WARN") + console.log( + ` ${GREEN}${results.length - failed.length - warned.length} PASS/SKIP${RESET}, ${YELLOW}${warned.length} WARN${RESET}, ${RED}${failed.length} FAIL${RESET}`, + ) + console.log() + + if (failed.length > 0) { + console.error(`${RED}${BOLD}Preflight FAILED${RESET} — resolve the FAIL check(s) above before releasing.`) + process.exit(1) + } + + console.log(`${GREEN}${BOLD}Preflight PASSED${RESET}${warned.length > 0 ? ` (with ${warned.length} warning(s))` : ""}.`) +} + +// Guard CLI execution when imported as a module (e.g., by tests). +if (import.meta.main) { + main().catch((e) => { + console.error(`${RED}error${RESET} preflight crashed: ${e instanceof Error ? e.message : String(e)}`) + process.exit(1) + }) +} From 42a6251a910b9b2053540e38d5af816d6a2472ea Mon Sep 17 00:00:00 2001 From: anandgupta42 Date: Thu, 23 Jul 2026 00:09:05 -0700 Subject: [PATCH 2/4] =?UTF-8?q?fix:=20[release]=20codex-review=20fixes=20?= =?UTF-8?q?=E2=80=94=20fail-closed=20gates,=20ordering,=20PREV=5FTAG=20upp?= =?UTF-8?q?er=20bound?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Codex review of the release-hardening commit found fail-open paths; all fixed: - skill/RELEASING.md: tag verification now aborts (`|| exit 1`) instead of echo-and-continue; stage-`tag` preflight moved AFTER the release commit (it previously ran against a tree dirtied by the CHANGELOG edit and could never pass); `cd` blocks made subshells so repo-root paths stay valid; smoke test builds with `OPENCODE_VERSION` injected and asserts equality - `release-preflight.ts`: npm publish-check only treats E404 as "not published" (other errors FAIL); `gh` errors FAIL (WARN only when the binary is missing); malformed `gh` JSON FAILs instead of double-reporting; `run()` no longer throws on missing binaries; SemVer prerelease compare is numeric-aware (`beta.10` > `beta.2`); `selectPrevTag` picks the greatest stable tag strictly LOWER than the target (a merged fork-inherited `v1.18.3` must not become PREV_TAG for `v0.9.x`) — same fix applied to `release.yml`'s PREV_TAG loop - `/release-beta`: wires in the preflight (`--allow-prerelease`) and fail-closed tag collision checks before pushing the beta tag - tests: 34 -> 38 (numeric prerelease precedence, prefix ranking, fork-inherited higher-tag exclusion) Co-Authored-By: Claude Fable 5 --- .claude/commands/release-beta.md | 13 ++++- .claude/commands/release.md | 33 ++++++----- .github/workflows/release.yml | 21 ++++--- docs/RELEASING.md | 47 ++++++++++------ script/release-preflight.test.ts | 25 +++++++- script/release-preflight.ts | 97 ++++++++++++++++++++++++-------- 6 files changed, 169 insertions(+), 67 deletions(-) diff --git a/.claude/commands/release-beta.md b/.claude/commands/release-beta.md index 8bc70224ce..eda20ec753 100644 --- a/.claude/commands/release-beta.md +++ b/.claude/commands/release-beta.md @@ -91,7 +91,13 @@ DIST=$(find packages/opencode/dist -type d -name '*'"$(uname -m | sed s/arm64/ar # 4. marker guard bun run script/upstream/analyze.ts --markers --base main --strict -# 5. Local Verdaccio sanity IF docker + a native-platform build are available +# 5. Deterministic preflight (tag collisions, version sanity vs npm, prerelease +# ancestry, release-blocker PRs). For a beta cut from main use it as-is; for a +# branch beta the `base` check will FAIL by design — then rely on the manual +# tag verification in Step 4 instead. +bun script/release-preflight.ts --version X.Y.Z-beta.N --stage tag --allow-prerelease + +# 6. Local Verdaccio sanity IF docker + a native-platform build are available # (the docker image is linux; on a mac you cannot cross-build the linux NAPI dist, # so this validates the current platform only — CI covers the rest): # (cd packages/dbt-tools && bun run build) && docker compose \ @@ -108,7 +114,12 @@ The `-beta.N` suffix is what routes to the beta channel. Do NOT omit it. ```bash BETA_TAG="vX.Y.Z-beta.N" +# Fail closed on collisions — a stale local tag makes `git tag` no-op and the +# push would publish whatever the old tag points at. Never delete-and-recreate. +git rev-parse -q --verify "refs/tags/$BETA_TAG" && { echo "LOCAL TAG EXISTS — STOP"; exit 1; } +git ls-remote origin "refs/tags/$BETA_TAG" | grep -q . && { echo "REMOTE TAG EXISTS — STOP"; exit 1; } git tag "$BETA_TAG" +test "$(git rev-parse "$BETA_TAG")" = "$(git rev-parse HEAD)" || { echo "TAG MISMATCH — STOP"; exit 1; } git push origin "$BETA_TAG" # push the TAG (not necessarily main) ``` diff --git a/.claude/commands/release.md b/.claude/commands/release.md index a18a5ae71a..dd5e9db9ce 100644 --- a/.claude/commands/release.md +++ b/.claude/commands/release.md @@ -229,12 +229,12 @@ When running the full suite in the background, tee complete output to a file (`b ### 7a: Smoke test the ACTUAL release candidate -Never smoke-test the `$PATH`-resolved `altimate` — in v0.9.2 it silently validated a stale globally-installed 0.7.3. Build and run the real artifact: +Never smoke-test the `$PATH`-resolved `altimate` — in v0.9.2 it silently validated a stale globally-installed 0.7.3. Build with the target version injected (otherwise the binary reports a `0.0.0-…` preview version and no assertion is possible), then assert the exact version. Run from the repo root; use a subshell for the `cd` so later paths still resolve: ```bash -cd packages/opencode && bun run pre-release # builds dist + checks NAPI externals + binary starts +(cd packages/opencode && OPENCODE_VERSION={NEXT_VERSION} bun run pre-release) BINARY=$(find packages/opencode/dist -path '*bin/altimate*' -type f | head -1) -"$BINARY" --version # MUST print a version consistent with this release, not a stale install +test "$("$BINARY" --version)" = "{NEXT_VERSION}" || { echo "SMOKE VERSION MISMATCH — STOP"; exit 1; } "$BINARY" --help ``` @@ -276,16 +276,15 @@ Wait for approval. ## Step 9: Pre-Release Checks -Run all mandatory checks: +Run all mandatory checks from the repo root: ```bash # Pre-release sanity (binary builds and starts) — if not already run in Step 7a -cd packages/opencode && bun run pre-release - -# Full preflight again, now tolerating local release commits ahead of origin/main -bun script/release-preflight.ts --version {NEXT_VERSION} --stage tag +(cd packages/opencode && OPENCODE_VERSION={NEXT_VERSION} bun run pre-release) ``` +(The stage-`tag` preflight runs in Step 10, AFTER the release commit — running it here would fail its clean-worktree check on the just-edited CHANGELOG.) + **Exit-status integrity rule:** never background a gate command with a trailing status echo (`cmd > log; echo "exit: $?"` — the echo always exits 0 and once turned a FAILED pre-release build into a reported pass). Preserve the real exit code: run in foreground, or `cmd 2>&1 | tee log` and check `$pipestatus[1]`/`PIPESTATUS[0]`, and ALWAYS read the tool's own verdict line from the log. **Gate: ALL CHECKS MUST PASS.** Stop on failure. @@ -302,9 +301,9 @@ docker compose -f test/sanity/docker-compose.verdaccio.yml up \ If Docker unavailable, skip — but mark it ⏭️ in the Step 12 summary (it was once skipped silently). CI will catch it. -## Step 10: Commit, Tag, Push — three verified steps, one atomic push +## Step 10: Commit, Preflight, Tag, Push — verified steps, one atomic push -Do NOT chain these with `&&` into one command (a mid-chain tag failure once committed but silently never pushed). Verify each stage: +Do NOT chain these with `&&` into one command (a mid-chain tag failure once committed but silently never pushed). Run each from the repo root and verify each stage: ```bash # 1. Commit (targeted staging only — never `git add -A`, worktrees carry scratch files) @@ -313,13 +312,19 @@ echo "release: v{NEXT_VERSION}" > .github/meta/commit.txt git commit -F .github/meta/commit.txt git log -1 --oneline # verify the release commit is HEAD -# 2. Tag — then VERIFY before pushing (a silently no-op'd `git tag` once pushed a +# 2. Full preflight — now that the tree is clean again and HEAD carries the +# release commit. Stage `tag` allows local commits ahead of origin/main. +bun script/release-preflight.ts --version {NEXT_VERSION} --stage tag +# MUST print "Preflight PASSED" and exit 0. On FAIL: stop. + +# 3. Tag — then VERIFY before pushing (a silently no-op'd `git tag` once pushed a # stale tag pointing 2641 commits away from main; the push itself triggers publish, -# so post-push verification is too late) +# so post-push verification is too late). The `exit 1` is load-bearing — +# a bare echo would print the warning and keep going. git tag v{NEXT_VERSION} -test "$(git rev-parse v{NEXT_VERSION})" = "$(git rev-parse HEAD)" || { echo "TAG MISMATCH — STOP"; } +test "$(git rev-parse v{NEXT_VERSION})" = "$(git rev-parse HEAD)" || { echo "TAG MISMATCH — STOP"; exit 1; } -# 3. Push branch + tag atomically — both land or neither does +# 4. Push branch + tag atomically — both land or neither does git push --atomic origin HEAD:main v{NEXT_VERSION} ``` diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index a8d064f49f..ab432c24d2 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -361,15 +361,18 @@ jobs: # Get the previous tag. # altimate_change — pick the newest STABLE tag that is an ANCESTOR of this - # commit, excluding the current tag and any prerelease (-beta etc). Plain - # `--sort=-version:refname | head -2 | tail -1` picked the 2nd-highest tag by - # name, which for a stable release lands on a prerelease or a stray/divergent - # tag (e.g. a dangling v0.9.0), producing a bogus compare range. Restricting to - # `--merged HEAD` non-prerelease tags yields the real previous release. - PREV_TAG=$(git tag --merged HEAD --sort=-version:refname \ - | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' \ - | grep -vx "$CURRENT_TAG" \ - | head -1) + # commit AND strictly LOWER than the current tag. Excluding only equality is + # not enough: if upstream history is ever merged, fork-inherited tags (e.g. + # v1.18.3) would beat the real previous release for a v0.9.x target and the + # compare range would silently omit history. + PREV_TAG="" + for t in $(git tag --merged HEAD --sort=-version:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$'); do + if [ "$t" != "$CURRENT_TAG" ] && \ + [ "$(printf '%s\n%s\n' "$t" "$CURRENT_TAG" | sort -V | head -1)" = "$t" ]; then + PREV_TAG="$t" + break + fi + done # Generate changelog from commits between tags echo "## What's Changed" > notes.md diff --git a/docs/RELEASING.md b/docs/RELEASING.md index 16c7072e3a..5fe1f70629 100644 --- a/docs/RELEASING.md +++ b/docs/RELEASING.md @@ -34,7 +34,22 @@ The version is injected into the binary via esbuild defines at compile time. ## Release Process -### 1. Update CHANGELOG.md +### 1. Run preflight (before touching anything) + +**MANDATORY** — the deterministic gate. Run it while the tree is still clean; +it fails on a dirty worktree by design: + +```bash +# Checks: clean tree, HEAD == fresh origin/main, no tag collisions (local OR +# remote), version sanity vs npm, prerelease-line ancestry, release-blocker +# PRs, PREV_TAG dry-run, marker guard. +bun script/release-preflight.ts --version 0.5.0 --stage pre +``` + +Do NOT proceed if any check fails. If it reports a tag collision, resolve it +explicitly — never delete-and-recreate a tag blind. + +### 2. Update CHANGELOG.md Add a new section at the top of `CHANGELOG.md`: @@ -48,18 +63,12 @@ Add a new section at the top of `CHANGELOG.md`: - ... ``` -### 2. Run pre-release sanity checks +### 3. Run pre-release sanity check -**MANDATORY** — these catch broken releases before they reach users: +**MANDATORY** — this catches broken binaries before they reach users: ```bash -# Deterministic preflight: clean tree, HEAD == fresh origin/main, no tag -# collisions (local OR remote), version sanity vs npm, prerelease-line -# ancestry, release-blocker PRs, PREV_TAG dry-run, marker guard. -bun script/release-preflight.ts --version 0.5.0 --stage pre - -cd packages/opencode -bun run pre-release +(cd packages/opencode && OPENCODE_VERSION=0.5.0 bun run pre-release) ``` `pre-release` verifies: @@ -67,10 +76,9 @@ bun run pre-release - They're installed in `node_modules` - A local build produces a binary that actually starts -Do NOT proceed if any check fails. If the preflight reports a tag collision, -resolve it explicitly — never delete-and-recreate a tag blind. +Do NOT proceed if any check fails. -### 3. Commit and tag +### 4. Commit, re-preflight, tag, push Stage files **individually** — never `git add -A` (release worktrees carry scratch files that must not ship in the release commit). Releasing does not @@ -80,15 +88,20 @@ HEAD equals freshly fetched `origin/main` (worktrees push via `HEAD:main`). ```bash git add CHANGELOG.md git commit -m "release: v0.5.0" + +# Re-run preflight now that the release commit exists. Stage `tag` allows +# local commits ahead of origin/main (the tree must be clean again). +bun script/release-preflight.ts --version 0.5.0 --stage tag || exit 1 + git tag v0.5.0 # Verify the tag points at HEAD BEFORE pushing — a pre-existing tag makes -# `git tag` fail, and pushing a stale tag triggers a publish of old code. -test "$(git rev-parse v0.5.0)" = "$(git rev-parse HEAD)" +# `git tag` silently fail, and pushing a stale tag publishes old code. +test "$(git rev-parse v0.5.0)" = "$(git rev-parse HEAD)" || exit 1 # Atomic: branch and tag land together or not at all. git push --atomic origin HEAD:main v0.5.0 ``` -### 4. What happens automatically +### 5. What happens automatically The `v*` tag triggers `.github/workflows/release.yml` which: @@ -100,7 +113,7 @@ The `v*` tag triggers `.github/workflows/release.yml` which: 6. **Publishes Docker image (best-effort)** — to `ghcr.io/altimateai/altimate-code`; failures are logged but do NOT fail the release, so verify manually if you need the image 7. ~~Updates AUR~~ — currently **disabled** (the workflow step is commented out; see `publish.ts` for setup steps to re-enable) -### 5. Verify +### 6. Verify After the workflow completes: diff --git a/script/release-preflight.test.ts b/script/release-preflight.test.ts index 360f1ba364..7e7abf09ff 100644 --- a/script/release-preflight.test.ts +++ b/script/release-preflight.test.ts @@ -86,8 +86,17 @@ describe("compareSemver", () => { expect(compareSemver(v("0.9.2-beta.1"), v("0.9.2-beta.1"))).toBe(0) }) - test("prerelease strings compare lexically", () => { - expect(compareSemver(v("0.9.2-beta.2"), v("0.9.2-beta.10"))).toBeGreaterThan(0) // "beta.2" > "beta.10" lexically + test("prerelease identifiers compare numerically per SemVer (beta.10 > beta.2)", () => { + expect(compareSemver(v("0.9.2-beta.10"), v("0.9.2-beta.2"))).toBeGreaterThan(0) + expect(compareSemver(v("0.9.2-beta.2"), v("0.9.2-beta.10"))).toBeLessThan(0) + }) + + test("numeric prerelease identifiers rank below alphanumeric ones", () => { + expect(compareSemver(v("0.9.2-1"), v("0.9.2-alpha"))).toBeLessThan(0) + }) + + test("a prerelease identifier prefix ranks below its longer form", () => { + expect(compareSemver(v("0.9.2-beta"), v("0.9.2-beta.1"))).toBeLessThan(0) }) test("major/minor/patch precedence beats prerelease presence", () => { @@ -151,6 +160,18 @@ describe("selectPrevTag", () => { const tags = ["v0.9.0", "v0.10.0", "v0.2.0"] expect(selectPrevTag(tags, "v0.11.0")).toBe("v0.10.0") }) + + test("ignores merged tags HIGHER than the target (fork-inherited upstream tags)", () => { + // If upstream history is ever merged, tags like v1.18.3 become ancestors + // of HEAD. They must not become PREV_TAG for a v0.9.x release — the + // compare range would silently omit history. + const tags = ["v1.18.3", "v1.17.13", "v0.9.2", "v0.9.1"] + expect(selectPrevTag(tags, "v0.9.3")).toBe("v0.9.2") + }) + + test("returns null when only higher tags are merged", () => { + expect(selectPrevTag(["v1.18.3"], "v0.9.3")).toBeNull() + }) }) describe("prereleaseLineTags", () => { diff --git a/script/release-preflight.ts b/script/release-preflight.ts index 110609cdee..b58a498f2d 100644 --- a/script/release-preflight.ts +++ b/script/release-preflight.ts @@ -59,12 +59,35 @@ export function parseSemver(version: string): Semver | null { } } +/** + * Compare two dot-separated prerelease strings per SemVer precedence: + * numeric identifiers compare numerically (so beta.10 > beta.2), numeric < + * alphanumeric, and a shorter identifier list is lower when it's a prefix. + */ +function comparePrerelease(a: string, b: string): number { + const as = a.split(".") + const bs = b.split(".") + const len = Math.min(as.length, bs.length) + for (let i = 0; i < len; i++) { + const an = /^\d+$/.test(as[i]) + const bn = /^\d+$/.test(bs[i]) + if (an && bn) { + const diff = Number(as[i]) - Number(bs[i]) + if (diff !== 0) return diff + } else if (an !== bn) { + return an ? -1 : 1 // numeric < alphanumeric + } else if (as[i] !== bs[i]) { + return as[i] < bs[i] ? -1 : 1 + } + } + return as.length - bs.length +} + /** * Compare two parsed SemVer values. Returns <0 if a0 if a>b. * A stable version is always greater than any prerelease of the same - * major.minor.patch (per SemVer precedence rules), and prerelease strings - * compare lexically (good enough for our `-beta.N` convention; we don't need - * full dot-identifier numeric-vs-alphanumeric precedence for this repo). + * major.minor.patch, and prerelease identifiers follow SemVer precedence + * (numeric-aware, so `beta.10` > `beta.2`). */ export function compareSemver(a: Semver, b: Semver): number { if (a.major !== b.major) return a.major - b.major @@ -73,7 +96,7 @@ export function compareSemver(a: Semver, b: Semver): number { if (a.prerelease === b.prerelease) return 0 if (a.prerelease === null) return 1 // stable > prerelease if (b.prerelease === null) return -1 - return a.prerelease < b.prerelease ? -1 : a.prerelease > b.prerelease ? 1 : 0 + return comparePrerelease(a.prerelease, b.prerelease) } /** True if the tag name matches a stable release tag: v... */ @@ -90,14 +113,18 @@ export function isStableTagName(tag: string): boolean { * * Takes the raw (unsorted, unfiltered) list of tags already known to be * merged into HEAD — the git call that produces that list belongs in main(), - * not here — and does the stable-filter + version-sort + exclude-current + - * take-first purely, so it's testable without git. + * not here — and picks the greatest stable tag STRICTLY LOWER than the + * target. Excluding only equality is not enough: if upstream history is ever + * merged, fork-inherited tags like v1.18.3 would otherwise beat the real + * previous release for a v0.9.x target. */ export function selectPrevTag(mergedTags: string[], currentTag: string): string | null { - const stable = mergedTags.filter((t) => isStableTagName(t) && t !== currentTag) - const parsed = stable + const target = parseSemver(normalizeVersion(currentTag)) + const parsed = mergedTags + .filter((t) => isStableTagName(t) && t !== currentTag) .map((tag) => ({ tag, v: parseSemver(normalizeVersion(tag)) })) .filter((x): x is { tag: string; v: Semver } => x.v !== null) + .filter((x) => target === null || compareSemver(x.v, target) < 0) parsed.sort((a, b) => compareSemver(b.v, a.v)) // descending return parsed.length > 0 ? parsed[0].tag : null } @@ -129,15 +156,20 @@ function repoRoot(): string { /** Run a command, never throwing — callers inspect `.code`. */ function run(cmd: string[]): RunResult { - const proc = Bun.spawnSync(cmd, { - cwd: repoRoot(), - stdout: "pipe", - stderr: "pipe", - }) - return { - code: proc.exitCode ?? 1, - stdout: proc.stdout ? proc.stdout.toString("utf-8").trim() : "", - stderr: proc.stderr ? proc.stderr.toString("utf-8").trim() : "", + try { + const proc = Bun.spawnSync(cmd, { + cwd: repoRoot(), + stdout: "pipe", + stderr: "pipe", + }) + return { + code: proc.exitCode ?? 1, + stdout: proc.stdout ? proc.stdout.toString("utf-8").trim() : "", + stderr: proc.stderr ? proc.stderr.toString("utf-8").trim() : "", + } + } catch (e) { + // Missing binary (ENOENT) and similar spawn failures land here. + return { code: 127, stdout: "", stderr: `${cmd[0]}: not found (${e instanceof Error ? e.message : String(e)})` } } } @@ -358,8 +390,19 @@ async function main(): Promise { const currentParsed = parseSemver(normalizeVersion(currentLatest)) const alreadyPublished = run(["npm", "view", `@altimateai/altimate-code@${version}`, "version"]) const isPublished = alreadyPublished.code === 0 && alreadyPublished.stdout.trim().length > 0 + // Fail closed: a non-zero exit only means "not published" when npm + // says so (E404). The first `npm view` succeeded, so the registry is + // reachable — any other error here is unknown state, not a green light. + const publishCheckInconclusive = + alreadyPublished.code !== 0 && !/E404|404 Not Found/i.test(alreadyPublished.stderr + alreadyPublished.stdout) - if (isPublished) { + if (publishCheckInconclusive) { + add({ + name: "version sanity", + status: "FAIL", + detail: `could not determine whether @altimateai/altimate-code@${version} is already published (npm error was not E404):\n${alreadyPublished.stderr || alreadyPublished.stdout}`, + }) + } else if (isPublished) { add({ name: "version sanity", status: "FAIL", @@ -442,17 +485,23 @@ async function main(): Promise { "--json", "number,title", ]) - if (gh.code !== 0) { - add({ name: "release-blocker PRs", status: "WARN", detail: `gh unavailable or errored:\n${gh.stderr || gh.stdout}` }) + const ghMissing = gh.code !== 0 && /not found|ENOENT|no such file/i.test(gh.stderr) + if (ghMissing) { + add({ name: "release-blocker PRs", status: "WARN", detail: `gh CLI not installed — cannot check release-blocker PRs:\n${gh.stderr}` }) + } else if (gh.code !== 0) { + // gh exists but errored (auth, network, bad label…) — fail closed, we + // cannot claim there are no blockers. + add({ name: "release-blocker PRs", status: "FAIL", detail: `gh errored — blocker state unknown:\n${gh.stderr || gh.stdout}` }) } else { - let prs: { number: number; title: string }[] = [] + let prs: { number: number; title: string }[] | null = null try { prs = JSON.parse(gh.stdout || "[]") } catch { - add({ name: "release-blocker PRs", status: "WARN", detail: `could not parse gh output:\n${gh.stdout}` }) - prs = [] + prs = null } - if (prs.length > 0) { + if (prs === null) { + add({ name: "release-blocker PRs", status: "FAIL", detail: `could not parse gh output — blocker state unknown:\n${gh.stdout}` }) + } else if (prs.length > 0) { add({ name: "release-blocker PRs", status: "FAIL", From 19dd09a425ab8dcc0cba0dde3b30945ec96ad675 Mon Sep 17 00:00:00 2001 From: anandgupta42 Date: Thu, 23 Jul 2026 00:25:19 -0700 Subject: [PATCH 3/4] =?UTF-8?q?fix:=20[release]=20address=20PR=20#1032=20r?= =?UTF-8?q?eview=20=E2=80=94=20strict=20SemVer,=20fail-closed=20npm,=20aut?= =?UTF-8?q?horitative=20tag=20check?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses all 7 chatgpt-codex-connector review comments: - P1 prerelease ancestry now checks against HEAD, not origin/main — cutting beta.N+1 from a release branch no longer trips on beta.N being unmerged - P1 npm unreachable/unparseable is now FAIL, not WARN — an inconclusive registry query must not defer failure to the irreversible release workflow - P1 release.yml validates the tag via `git ls-remote origin` (with annotated `^{}` peel) instead of an unforced fetch that a force-moved tag would defeat - P1 prerelease targets are also compared against the current `beta` dist-tag so a stale beta.N cannot move new beta installs backward - P2 `parseSemver` + the workflow tag regex now enforce semver.org rules: no leading zeros in numeric identifiers, no empty prerelease identifiers - P2 ci.yml marker-guard job runs the preflight test suite (root bunfig.toml pins `bun test` away from the repo root, so it runs from `script/`) - P2 marker-guard check FAILs when analyze.ts degrades to pattern-only detection because the upstream remote is missing tests: 38 -> 41 (leading-zero rejection, empty-identifier rejection, zero/alphanumeric prerelease acceptance) Co-Authored-By: Claude Fable 5 --- .github/workflows/ci.yml | 6 +++ .github/workflows/release.yml | 26 ++++++++--- script/release-preflight.test.ts | 18 ++++++++ script/release-preflight.ts | 77 +++++++++++++++++++++++++++----- 4 files changed, 111 insertions(+), 16 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 66cfbd0a6b..7bb116a6eb 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -509,6 +509,12 @@ jobs: run: bun test working-directory: script/upstream + - name: Run release preflight tests + # Root bunfig.toml pins `bun test` away from the repo root, so run from + # script/ — same pattern as the marker parser tests above. + run: bun test release-preflight.test.ts + working-directory: script + - name: Check for missing altimate_change markers run: | if [[ "${{ github.event_name }}" == "push" ]]; then diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index ab432c24d2..1d481ee55c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -197,17 +197,31 @@ jobs: # so a malformed tag could publish to npm and only fail afterwards. - name: Validate release tag run: | - if ! echo "$CURRENT_TAG" | grep -qE '^v[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9._-]+)?$'; then + # Strict SemVer: no leading zeros in numeric identifiers, no empty + # prerelease identifiers (rejects v01.2.3, v1.2.3-01, v1.2.3-a..b). + SEMVER_ID='(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)' + if ! echo "$CURRENT_TAG" | grep -qE "^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-${SEMVER_ID}(\.${SEMVER_ID})*)?$"; then echo "::error::Invalid tag format: $CURRENT_TAG — refusing to publish" exit 1 fi - # Shallow checkouts may not carry the tag ref locally — fetch it so the - # comparison runs against what origin actually has. - git fetch --no-tags --depth 1 origin "refs/tags/$CURRENT_TAG:refs/tags/$CURRENT_TAG" 2>/dev/null || true - TAG_SHA=$(git rev-parse "refs/tags/$CURRENT_TAG^{commit}") + # Ask origin directly what the tag points to — the event payload and + # the checkout's local tag ref are both stale if the tag was force-moved + # between the push event and this job. An unforced `git fetch` would + # refuse to clobber the local tag, so ls-remote is the authority. + LS=$(git ls-remote origin "refs/tags/$CURRENT_TAG" "refs/tags/$CURRENT_TAG^{}") || { + echo "::error::Could not query origin for tag $CURRENT_TAG" + exit 1 + } + # Annotated tags list a peeled ^{} line pointing at the commit; prefer it. + TAG_SHA=$(echo "$LS" | grep '\^{}' | cut -f1 | head -1) + [ -z "$TAG_SHA" ] && TAG_SHA=$(echo "$LS" | cut -f1 | head -1) + if [ -z "$TAG_SHA" ]; then + echo "::error::Tag $CURRENT_TAG no longer exists on origin — refusing to publish" + exit 1 + fi HEAD_SHA=$(git rev-parse HEAD) if [ "$TAG_SHA" != "$HEAD_SHA" ]; then - echo "::error::Tag $CURRENT_TAG points at $TAG_SHA but workflow checked out $HEAD_SHA" + echo "::error::Tag $CURRENT_TAG points at $TAG_SHA on origin but workflow checked out $HEAD_SHA (tag moved since the push event?)" exit 1 fi VERSION="${CURRENT_TAG#v}" diff --git a/script/release-preflight.test.ts b/script/release-preflight.test.ts index 7e7abf09ff..7e344d6c0f 100644 --- a/script/release-preflight.test.ts +++ b/script/release-preflight.test.ts @@ -55,6 +55,24 @@ describe("parseSemver", () => { expect(parseSemver("not-a-version")).toBeNull() expect(parseSemver("")).toBeNull() }) + + test("rejects leading zeros in numeric identifiers (SemVer rule)", () => { + expect(parseSemver("01.2.3")).toBeNull() + expect(parseSemver("1.02.3")).toBeNull() + expect(parseSemver("1.2.03")).toBeNull() + expect(parseSemver("1.2.3-01")).toBeNull() + }) + + test("rejects empty prerelease identifiers", () => { + expect(parseSemver("1.2.3-")).toBeNull() + expect(parseSemver("1.2.3-alpha..1")).toBeNull() + expect(parseSemver("1.2.3-alpha.")).toBeNull() + }) + + test("accepts zero and alphanumeric prerelease identifiers", () => { + expect(parseSemver("1.2.3-0")).toEqual({ major: 1, minor: 2, patch: 3, prerelease: "0" }) + expect(parseSemver("1.2.3-0a.1")).toEqual({ major: 1, minor: 2, patch: 3, prerelease: "0a.1" }) + }) }) describe("compareSemver", () => { diff --git a/script/release-preflight.ts b/script/release-preflight.ts index b58a498f2d..d2f8caccb7 100644 --- a/script/release-preflight.ts +++ b/script/release-preflight.ts @@ -46,10 +46,15 @@ export function normalizeVersion(input: string): string { /** * Parse a normalized (no leading "v") SemVer string into its components. - * Returns null if the string isn't valid SemVer (major.minor.patch[-prerelease]). + * Follows semver.org rules (minus build metadata): no leading zeros in + * numeric identifiers, no empty prerelease identifiers. Returns null if the + * string isn't valid SemVer (major.minor.patch[-prerelease]). */ +const SEMVER_RE = + /^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?$/ + export function parseSemver(version: string): Semver | null { - const m = /^(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z-.]+))?$/.exec(version) + const m = SEMVER_RE.exec(version) if (!m) return null return { major: Number(m[1]), @@ -380,10 +385,13 @@ async function main(): Promise { } else { const npmView = run(["npm", "view", "@altimateai/altimate-code", "version"]) if (npmView.code !== 0) { + // Fail closed: without the registry we cannot verify monotonicity or + // that the version is unpublished — deferring that failure to the + // irreversible release workflow is exactly the wrong place for it. add({ name: "version sanity", - status: "WARN", - detail: `could not reach npm to check current 'latest' version:\n${npmView.stderr || npmView.stdout}`, + status: "FAIL", + detail: `could not reach npm to verify version state — retry when the registry is reachable:\n${npmView.stderr || npmView.stdout}`, }) } else { const currentLatest = npmView.stdout.trim() @@ -411,8 +419,8 @@ async function main(): Promise { } else if (!currentParsed) { add({ name: "version sanity", - status: "WARN", - detail: `could not parse current npm 'latest' version "${currentLatest}" to compare`, + status: "FAIL", + detail: `could not parse current npm 'latest' version "${currentLatest}" — cannot verify monotonicity`, }) } else if (compareSemver(parsed, currentParsed) <= 0) { add({ @@ -420,11 +428,45 @@ async function main(): Promise { status: "FAIL", detail: `${version} is not strictly greater than current npm latest (${currentLatest})`, }) + } else if (parsed.prerelease) { + // A prerelease target must also move the `beta` dist-tag forward — + // being greater than stable `latest` is not enough. Publishing + // 0.10.0-beta.4 while `beta` points at 0.10.0-beta.5 would move new + // beta installs BACKWARD. + const distTags = run(["npm", "view", "@altimateai/altimate-code", "dist-tags", "--json"]) + let betaCurrent: string | null = null + if (distTags.code === 0) { + try { + betaCurrent = (JSON.parse(distTags.stdout) as Record).beta ?? null + } catch { + betaCurrent = null + } + } + const betaParsed = betaCurrent ? parseSemver(normalizeVersion(betaCurrent)) : null + if (distTags.code !== 0) { + add({ + name: "version sanity", + status: "FAIL", + detail: `could not read npm dist-tags to compare against the current beta channel:\n${distTags.stderr || distTags.stdout}`, + }) + } else if (betaParsed && compareSemver(parsed, betaParsed) <= 0) { + add({ + name: "version sanity", + status: "FAIL", + detail: `${version} is not strictly greater than the current beta dist-tag (${betaCurrent}) — new beta installs would move backward`, + }) + } else { + add({ + name: "version sanity", + status: "PASS", + detail: `${version} > latest (${currentLatest})${betaCurrent ? ` and > beta (${betaCurrent})` : "; no beta dist-tag"}; not yet published; prerelease allowed`, + }) + } } else { add({ name: "version sanity", status: "PASS", - detail: `${version} > current latest (${currentLatest}); not yet published; ${parsed.prerelease ? "prerelease allowed" : "stable"}`, + detail: `${version} > current latest (${currentLatest}); not yet published; stable`, }) } } @@ -448,22 +490,27 @@ async function main(): Promise { detail: `no v${parsed.major}.${parsed.minor}.*-* prerelease tags exist for this release line`, }) } else { + // Check ancestry against HEAD (the release candidate), not origin/main: + // for /release, HEAD equals or descends from origin/main so the result + // is the same, and for /release-beta cutting beta.N+1 from a branch, + // the beta.N tag is an ancestor of HEAD but not of origin/main — + // checking origin/main would wrongly block every branch beta. const offenders: string[] = [] for (const tag of lineTags) { - const ancestor = git(["merge-base", "--is-ancestor", tag, "origin/main"]) + const ancestor = git(["merge-base", "--is-ancestor", tag, "HEAD"]) if (ancestor.code !== 0) offenders.push(tag) } if (offenders.length === 0) { add({ name: "prerelease ancestry", status: "PASS", - detail: `all ${lineTags.length} prerelease tag(s) for v${parsed.major}.${parsed.minor}.* are merged into origin/main:\n${lineTags.join(", ")}`, + detail: `all ${lineTags.length} prerelease tag(s) for v${parsed.major}.${parsed.minor}.* are ancestors of HEAD:\n${lineTags.join(", ")}`, }) } else { add({ name: "prerelease ancestry", status: "FAIL", - detail: `prerelease tag(s) NOT merged into origin/main — merge before releasing this line:\n${offenders.join("\n")}`, + detail: `prerelease tag(s) NOT in this release candidate's history — merge before releasing this line:\n${offenders.join("\n")}`, }) } } @@ -551,6 +598,16 @@ async function main(): Promise { const fullOutput = [guard.stdout, guard.stderr].filter((s) => s.length > 0).join("\n") if (guard.code !== 0) { add({ name: "marker guard", status: "FAIL", detail: fullOutput || `exited ${guard.code} with no output` }) + } else if (/falling back to pattern-based detection/i.test(fullOutput)) { + // analyze.ts exits 0 on this fallback, but without the upstream remote + // it only pattern-matches packages/opencode/src/ and can miss unmarked + // changes in other upstream-owned packages. A release machine must have + // the real remote — fail closed. + add({ + name: "marker guard", + status: "FAIL", + detail: `upstream remote unavailable — guard ran in degraded pattern-only mode, coverage incomplete.\nAdd it: git remote add upstream https://github.com/anomalyco/opencode.git && git fetch upstream --no-tags\n${fullOutput}`, + }) } else { add({ name: "marker guard", status: "PASS", detail: fullOutput || "no unmarked upstream-shared changes" }) } From e06bcde32e969221005ee34dbe4f248a705048e9 Mon Sep 17 00:00:00 2001 From: anandgupta42 Date: Thu, 23 Jul 2026 00:25:58 -0700 Subject: [PATCH 4/4] chore: drop internal release-retro notes from the PR Co-Authored-By: Claude Fable 5 --- .github/meta/release-retro-2026-07-22.md | 95 ------------------------ script/release-preflight.ts | 2 +- 2 files changed, 1 insertion(+), 96 deletions(-) delete mode 100644 .github/meta/release-retro-2026-07-22.md diff --git a/.github/meta/release-retro-2026-07-22.md b/.github/meta/release-retro-2026-07-22.md deleted file mode 100644 index 66aea3bb82..0000000000 --- a/.github/meta/release-retro-2026-07-22.md +++ /dev/null @@ -1,95 +0,0 @@ -# /release Skill — Robustness Improvements (retro of v0.8.10, v0.9.1, v0.9.2) - -Evidence: full session transcripts of the last three `/release` runs (v0.8.10 2026-06-23, v0.9.1 2026-07-08/09, v0.9.2 2026-07-21), git history between tags, and a Codex code-level review of the plan against `release.yml` / `publish.ts` / `pre-release-check.ts`. - -## What's working (keep) - -- **Multi-persona review earns its cost.** It caught the `release.yml` `PREV_TAG` bug that would have generated v0.9.1 release notes omitting the entire 165-commit upstream merge (CTO persona), and discovered the CI marker guard had been silently no-op'ing on PRs (bare `catch { return [] }` on a bad base ref). -- **Adversarial tests catch real bugs pre-tag.** v0.9.1: MCP annotation-hint classifier gap. v0.9.2: `transformSnowflakeBody` crash on non-object JSON roots. -- **Targeted `git add`** (v0.9.2) correctly avoided sweeping session scratch files into the release commit — RELEASING.md still says `git add -A` and should be fixed. - ---- - -## P0 — prevent shipping the wrong artifact - -### 1. Tag creation is unsafe; both stable releases hit tag collisions — fail closed -- v0.9.1: a stale local `v0.9.1` tag made `git tag` no-op; `git push origin v0.9.1` pushed a tag pointing at a 2641-commit-divergent commit. Only luck (the stale tree predated the current `release.yml`) prevented a wrong publish. -- v0.9.2: stale local `v0.9.2` inherited from upstream OpenCode fork tag history → reactive blind `git tag -d` + recreate. -- **Fix (Step 10):** - - Preflight `git rev-parse -q --verify refs/tags/v{V}` AND `git ls-remote origin refs/tags/v{V}`. On any collision: **stop and require explicit human resolution** (log what the old tag pointed to). Never auto-delete. - - After `git tag`: assert `git rev-parse v{V} == git rev-parse HEAD` **before** pushing — post-push verification is too late, the push itself triggers CI/publish. - - One-time repo policy (separate PR, not release behavior): prune fork-inherited upstream tags and set the upstream remote to `--no-tags`. - -### 2. Atomic push; commit/tag/push as separately verified steps -v0.9.2's single `commit && tag && push` chain committed, failed at tag, silently never pushed. Also, pushing `HEAD:main` and the tag as two pushes can leave the tag published against stale main if the branch push is rejected. -- **Fix:** verify commit; verify tag SHA; then `git push --atomic origin HEAD:main v{V}` so branch+tag land together or not at all. - -### 3. Gates must carry real exit status and test the real artifact -- v0.9.1: pre-release check backgrounded as `cmd > log 2>&1; echo "exit: $?"` — trailing echo turned a FAILED build into "exit 0". Caught only by reading the log. -- v0.9.2: Step 7 smoke test ran `$PATH`-resolved `altimate --version` → stale global `0.7.3`, not the candidate; never noticed in-session. -- **Fix:** preserve the actual exit code (`tee` + `PIPESTATUS`/`wait`), never append a status echo as the last statement; smoke-test the freshly built artifact by explicit path AND the npm-pack-installed wrapper, asserting the normalized version (`0.9.3`, no `v` — build strips it). Note: `pre-release-check.ts` injects workspace `NODE_PATH` (pre-release-check.ts:110), so it can pass for the wrong reason — a hermetic variant should exist. - -### 4. CI publish gate is broken at the source (found by Codex review, verified) -`publish-npm` needs only `[build, sanity-verdaccio]` — **not `test`** (release.yml:182-184). npm can publish while the test job is red. Additional CI gaps: -- Tag-format validation lives in `github-release`, which runs **after** `publish-npm` (release.yml:301) — validate tag syntax/SHA/changelog-entry/version **before** any publish job. -- Partial-publish recovery undefined: platform packages publish concurrently, wrapper after (publish.ts:125); a mid-run failure leaves immutable partial npm state and re-runs may fail on already-published versions. Make publish idempotent/resumable and document per-phase recovery. -- Docker/AUR/Homebrew publish errors are swallowed (publish.ts:192) while RELEASING.md promises they "happen automatically" — make them gated jobs or correct the doc. -- CI smoke tests don't assert the exact released version. - -**These are workflow/script PRs, not skill prose.** - ---- - -## P1 — codify reality: preflight as a deterministic script - -### 5. Replace Step 2's "must be on main" with the invariant that actually matters -All three releases deviated (main checked out in another worktree; released from a feature branch whose HEAD equaled origin/main). The correct branch-independent invariant: **after a fresh `git fetch`, clean HEAD == `origin/main`**; push via `HEAD:main`. If it doesn't hold, stop. - -### 6. Move the deterministic checks into `script/release/preflight.ts` -Six `fix: [release]` commits in one cycle (Bun pin, Verdaccio .dockerignore, version derivation, npm E413 de-dupe, smoke-test target, marker-guard base ref) = every release debugs the pipeline live. The skill's new Step 2 = run the script. Checks (objective invariants only): -- clean worktree; HEAD == fresh origin/main (per #5) — also covers the "user just merged a PR" staleness that forced a manual interruption in v0.9.2 -- tag collision, local + remote (per #1) — fail closed -- target version: valid SemVer, strictly greater than current npm `latest`, not already published (NOT equality with package.json — it intentionally says `1.17.9`) -- prerelease ancestry scoped to the target release line: e.g. releasing `v0.9.1` with `v0.9.0-beta.*` not an ancestor of origin/main → stop (v0.9.1's Step-2 stop turned into a 17-hour merge side-quest improvised via 3 AskUserQuestion rounds) -- open PRs labeled `release-blocker` → stop -- `PREV_TAG` dry-run: evaluate release.yml's previous-tag logic locally, print which tag release notes will diff against -- marker guard full run, complete violation list in one pass (v0.9.2 took 7 serial fix rounds for 11 hunks because output was consumed one hunk at a time) - -### 7. Prerequisites fail → stop; don't improvise -The release skill should not grow an upstream-merge/admin-bypass decision tree. If preflight fails on divergence, stop `/release`, do the prerequisite work as its own task, re-invoke. If the user directs an admin `--admin` merge to satisfy a prerequisite (as in v0.9.1), the Step 12 summary must disclose that branch protection was bypassed. - ---- - -## P1 — close the loop on review output - -### 8. Every deferred finding gets a durable disposition before the ship gate -v0.8.10 deferred 3 findings; zero filed — they exist only in a chat transcript. v0.9.1/v0.9.2 filed 6 and 4 (good). Gate: each deferred item has an issue link (new or existing) or an explicit user opt-out; the summary table gets a "Deferred → filed" row so an empty list is visible. Same rule for CI jobs waved through as "flaky" (v0.9.2's Verdaccio no-internet re-run was never tracked): link a durable issue, not necessarily a new one per flake. - -### 9. Unverified P0-candidates: validate or block — no approval-string bypass -v0.9.1's Chaos Gremlin flagged possible silent global-config loss; direct verification failed twice on tooling friction and the release shipped on indirect reasoning. Rule: a P0-candidate must be either empirically validated, or explicitly downgraded with written rationale reviewed by the user. It cannot ship as a P0 with a warning label. - -### 10. Step-skipping must be visible -Step 7 (UX smoke) was silently absorbed in v0.9.1 and mis-executed in v0.9.2; optional Verdaccio was skipped without mention. Every step gets a row in the Step 12c summary table (✅/⏭️/❌) so silent omission is structurally impossible. - ---- - -## P2 — agent ergonomics (real friction, but not release safeguards) - -- **Status on state changes:** both v0.8.10 (11.5-min silent CI wait) and v0.9.1 (104-min Codex review) ended with the user typing "done?". Post an update on every CI state change; long background reviews ping every ~20-30 min. -- **Persona delivery protocol:** 3 of 5 v0.9.2 reviewers went idle without delivering and needed manual nudges. Persona prompt must end: "Send your full review via SendMessage to main as your final action." -- **Shell hygiene:** explicit working directories instead of `cd X &&` chains (2 failures in v0.8.10); Read CHANGELOG.md before Edit (v0.8.10); background suite runs tee full output to a file (v0.9.2 re-ran an 11,551-test suite just to learn which test failed). -- **Step 12 scope:** include commenting the fix/version on the originating bug-report issue (v0.9.2 user had to ask separately). -- **Docs:** update RELEASING.md — targeted staging instead of `git add -A`; document the worktree/`HEAD:main` path; stop promising Docker/AUR/Homebrew until they're gated jobs. -- **Fan-out budget:** v0.9.1 hit the weekly usage limit mid-release (7+ concurrent heavy agents); run persona reviews on cheaper models and stagger heavy fan-out in long sessions. - ---- - -## Top 5 by robustness per unit of process (Codex-concurred) - -1. Fail-closed tag checks + SHA assertions (#1) -2. Atomic `HEAD:main`+tag push with per-step verification (#2) -3. Real exit-status handling + exact-artifact/version smoke test (#3) -4. Deterministic preflight script, objective invariants only (#6) -5. Branch-independent "HEAD == fresh origin/main" base check (#5) - -Plus one CI PR that outranks all skill prose: make `publish-npm` depend on `test` and move tag validation before publish (#4). diff --git a/script/release-preflight.ts b/script/release-preflight.ts index d2f8caccb7..44ff44de8e 100644 --- a/script/release-preflight.ts +++ b/script/release-preflight.ts @@ -3,7 +3,7 @@ * Release Preflight — deterministic gate for `/release` and `/release-beta`. * * Replaces LLM-improvised release judgment calls with objective, scriptable - * checks. Born from `.github/meta/release-retro-2026-07-22.md`: three straight + * checks. Born from the 2026-07-22 release retro: three straight * releases (v0.8.10, v0.9.1, v0.9.2) each hit at least one of these failure * modes live — a stale local tag that nearly published a 2641-commit-divergent * artifact, releasing with HEAD behind origin/main, an unmerged prerelease line