releaseWizard: fix changelog forward-port for a release on an older…

[dsmiley]

… major branch

logchange.py forward-port failed releasing 9.11.0: it tried to cherry-pick 9.x feature commits onto branch_10x and main.

• Only cherry-pick commits touching changelog/v{version}/. Selecting everything under changelog/ picked up every feature commit with a changelog entry since the release branch diverged from the target.
• Pull each target branch (fast-forward only) before cherry-picking, so the final push isn't rejected.
• Resolve modify/delete conflicts confined to changelog/unreleased/ by removing the entry; -X ours doesn't settle those. Skip a pick that is already applied.
• Discard other versions' regenerated version-summary.md files, which otherwise block the next checkout.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

[dsmiley]

Disclaimer: I don't do python. Just vibing with Claude here as triage issues I found. Nonetheless it had this to say when I asked for verification:

 What the real run exercised:                                                                                                                            │ 120 +    else:                                                                          
                                                                                                                                                          │ 121 +        print("  Nothing left to apply — skipping.")                               
  - Narrowed commit selection: 6 changelog-only commits per target instead of 77.                                                                         │ 122 +        git(["cherry-pick", "--skip"], cwd=git_root)                               
  - Pull before pick: all three targets fast-forwarded (they were 77, 71 and 16 behind).                                                                  │ 123 +    return True                                                                    
  - Conflict resolution: modify/delete conflicts in changelog/unreleased/ were auto-removed and the pick continued — confirmed in the log for branch_9x (4│ 124 +                                                                                   
    files).                                                                                                                                               │ 125 +                                                                                   
  - Restore after regenerate: the stray v10.0.0/version-summary.md change no longer blocked the checkout of branch_9x.                                    │ 126  def strip_unreleased_block(changelog_path: Path, dry_run=False):                   
                                                                                                                                                          │ 127      """Remove the [unreleased] block that logchangeGenerate emits.                 
  Caveats worth stating honestly:                                                                                                                         │ 128                                                                                     
                                                                                                                                                          │ 398          msg_b = f"Regenerate CHANGELOG.md for v{version}"                          
  - Two runs, not one clean pass. The first run did branch_10x and main, then failed before branch_9x. I added the restore fix and the second run         │ 399          print(f"  Committing: {msg_b!r}")                                          
    finished. The final script has not done a start-to-finish run from a clean state.                                                                     │ 400          git(["commit", "-m", msg_b], cwd=git_root, dry_run=dry_run)                
  - Skip-empty-pick path never ran. On the re-run, git reported 0 commits to pick for branch_10x and main, so that code is untested.                      │ 401 +    git(["restore", "changelog/"], cwd=git_root, dry_run=dry_run)                  
  - --push was not used. You pushed by hand, so the script's push step is untested with these changes (it wasn't modified).                               │ 402                                                                                     
  - --dry-run not run with the changes.                                                         

[janhoy]

Thanks David for fixing this. It will be the third attempt at a not-terrible RM experience handling these yml files. I was close but not close enough. Below is some (AI) comments...

🤖 The research and this comment were generated by Claude Code. I've read it through and agree with the findings, but treat the details as machine-produced and verify anything that matters.

I checked out the branch and verified the claims empirically — the core fix is correct, and the bug it fixes is definitely real. Against the actual 9.11.0 history (upstream/main...upstream/branch_9_11):

pathspec	commits selected
changelog/ (old)	183 — every feature commit with a changelog entry since the branch cut
changelog/v9.11.0/ (new)	6 — exactly the right ones

I also reproduced the modify/delete case in a scratch repo and confirmed that -X ours -X no-renames really does leave a UD unmerged path that -X ours cannot settle, that git diff --diff-filter=U reports it, and that the git rm + cherry-pick --continue recovery produces the correct tree. And the narrowing is safe for the commits that matter: prepare stages git add changelog as a single commit touching both unreleased/ and v{version}/, so it's still selected, and the .gitkeep commit the narrowing now excludes is independently re-created per target by ensure_unreleased_gitkeep.

A few things below, only the first two of which I'd consider worth acting on before merge.

1. --cherry-pick now costs ~3.5 min per target, for no benefit

Measured on this repo:

git log --cherry-pick --right-only upstream/main...upstream/branch_9_11 -- changelog/v9.11.0/ ...
→ 3m22s

Git computes patch-ids for the entire symmetric difference before path limiting prunes it — that's 6652 commits for main and 6576 for branch_10x. So roughly 7 minutes of completely output-free hang, in a wizard step that runs with confirm_each_command: false and --push. This is exactly the cross-major scenario the PR is about, so it'll bite on the next 9.x release.

For comparison, the candidate list without --cherry-pick takes 0.04s. Filtering those 6 candidates by patch-id against the target's own 6 pathspec-touching commits gives the identical answer in well under a second — I verified both produce the same result (zero commits for 9.11.0, since main already has them).

The simplest option is to drop --cherry-pick altogether: recover_cherry_pick already --skips a pick that is empty because it's already applied, so that is what provides re-run idempotency now. Relatedly, the step-4 comment still credits --cherry-pick with making forward-port idempotent, which is stale either way.

2. recover_cherry_pick can delete an unreleased entry it shouldn't

The recovery removes any conflicting path under changelog/unreleased/. The pre-existing stale pass just below it is deliberately more careful — it only removes unreleased files that have a counterpart in changelog/v{version}/. Without that guard, an identically-named entry on the target that belongs to a different version gets silently git rm'd, losing a contributor's pending changelog entry on a branch the script then pushes.

Applying the same counterpart check, and falling through to the existing error path when it doesn't hold, would keep the two code paths consistent.

3. git restore changelog/ is unguarded destruction of uncommitted work

cmd_forward_port has no clean-tree precondition, and the git checkout <release_branch> in step 1 will happily carry non-conflicting local modifications along. A git status --porcelain assertion at the top of the function would make both new restore calls safe, and would also protect the new pull and the per-target checkouts.

Related: restore only touches tracked files, so the "blocks the next checkout" problem isn't fully closed. A target branch whose version folder has no tracked version-summary.md gets an untracked one, which still blocks checkout if the next branch tracks that path. Probably not reachable today, but worth a comment if you don't want to handle it.

4. The release branch isn't pulled, only the targets

Step 6 pushes release_branch too, so if it's behind the remote the push is rejected after all the target work is done. The same git pull --ff-only in step 1 would make this symmetric with the stated goal ("so the final push isn't rejected").

Nits

• recover_cherry_pick ignores dry_run. It's unreachable in dry-run today (a dry-run git() returns rc 0 and never raises), but it's unguarded, so it would git rm and cherry-pick --continue for real the day that changes.
• unmerged_paths is exposed to core.quotepath: a non-ASCII entry filename (changelog names derive from JIRA summaries) comes back quoted and the following git rm then fails. git diff -z, or git ls-files -u -z, avoids it. The pre-existing commit_touches_unreleased has the same latent issue.
• In releaseWizard.yaml, "but not yet on the target to branch_10x, main, and branch_9x" reads a bit garbled now. More importantly, the description doesn't mention that the script now git pull --ff-onlys each target — RMs should know it moves their local branches.
• git restore needs git >= 2.23. Fine in practice; just noting there's no documented minimum anywhere.

[janhoy]

See review feedback in other comment

[dsmiley]

I don't have enough interest/know-how to review python or the release wizard details (like this). Maybe just have your AI do what it thinks is best and we merge.

[janhoy]

I had Claude Code (Fable) work through my own review comments above (or rather Claude Opus's comments), one commit per item, on this branch. I also merged in main. Nothing new in the design, this is the review applied:

• Dropped --cherry-pick from the commit selection. Measured against the real 9.11.0 history: 191s per target with it, 0.03s without, same six commits. Re-runs stay idempotent since an already-applied pick comes out empty and gets --skipped.
• recover_cherry_pick now only removes a conflicting unreleased entry if it also exists in changelog/v{version}/, same guard as the stale-entry pass. Anything else falls through to the manual-resolve error.
• Forward-port refuses to run on a dirty tree, and the git restore is paired with a git clean limited to changelog/*/version-summary.md so an untracked summary can't block the next checkout either.
• The release branch is pulled --ff-only like the targets, so the final push doesn't get rejected.
• Nits: -z on the git path listings (non-ASCII entry names), --dry-run honoured in recover_cherry_pick, wizard text reworded and now mentions the clean-tree requirement and the pulls, git 2.23 noted in the docstring.

🤖 Implemented and verified with Claude Code. Every fix was reproduced failing first and then passing in a scratch repo with a fake gradle: 9x/9_11/10x/main branches, a 9.x feature commit with a changelog entry, a modify/delete conflict, non-ASCII entry name, untracked version-summary, same-name entry without counterpart, dirty tree, local branches behind origin, --push, re-run and --dry-run. I've read through the diff and I'm fine with it.

I think this is ready to merge once CI is happy.

[janhoy]

Hopefully, changelog handling will now be more friction-less for future RM's. It was a good call to move most of the mechanics into logchange.py and not as individual commands in wizard as in first revision.

@sigram you may want to port this change into your release branch branch_10_1 before next RC to save yourself from some pain. If you don't want to merge it into the release branch, it also works placing the script somewhere else and running it from there, preferably based on the 10x version and not the main version...

[sigram]

@janhoy will do, thanks!

[janhoy]

@dsmiley I set milestone 9.x since there will likely be another 9.x release. Feel free to merge and backport so that @sigram can use it in the 10.1 release. Or is it so that the old script is likely to work for the 10.1 release, that the fix is mostly for previous-major-branch use case?