Fix branch switch failure with symlink changes - #1077
Open
jojinkb wants to merge 1 commit into
Open
Conversation
When switching to the push branch, go-git removes worktree files that differ between the checkout ref and the push branch. The BoundOS billy filesystem resolves a trailing symlink path component, so removing a symlink path operates on its target instead of the symlink itself (go-git/go-billy issue 135). When the target has already been deleted by a preceding change of the same checkout, the branch switch fails with "remove <target>: no such file or directory", permanently failing the reconciliation until the push branch is manually rebased or deleted. When the target still exists, it is deleted in place of the symlink, corrupting the worktree with stale symlinks and missing files that then get committed to the push branch. Wrap the worktree filesystem handed to the gogit client so that Remove and RemoveAll operate on the given path itself, never on the target it may link to, and treat removal of an already-absent path as a no-op, matching git checkout semantics. The upstream fix exists only on the go-billy v6 pre-release line and won't be backported to v5; the wrapper can be removed once go-git and go-billy are bumped to v6. Assisted-by: Claude Code/claude-fable-5 Signed-off-by: Jojin <jojin.kb@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1037
When an
ImageUpdateAutomationuses apush.branchthat has fallen behind the checkout ref and the changes between the two branches involve symlinks,SwitchBranchcan fail with:Once hit, every reconcile fails the same way until the push branch is manually rebased or deleted.
Root cause
go-git/go-billy#135: go-billy v5's
osfs.BoundOSresolves a trailing symlink path component (BoundOS.abs→SecureJoinwalks every component, including the leaf), soRemove/RemoveAllon a symlink path operate on the symlink target instead of the symlink itself.During the branch switch, go-git removes worktree files that differ between the two branches, in lexical path order. Two failure modes follow:
Modifychange, handled asRemove+ re-create.Removeresolves the symlink to its target — which the same checkout has already deleted (e.g.deploy/_stacks/config.yamlsorts beforedeploy/<service>/config.yaml) — and fails with ENOENT. This reproduces the exact error in Controller errors when attempting to delete a non-existent file #1037, including the error naming the target path rather than the symlink being switched away from.Status()and get committed to the push branch byCommitAndPush.The go-billy fix exists only on the v6 (pre-release) line and won't be backported to v5 (see maintainer comments on go-git/go-billy#135), so waiting for a dependency bump would leave affected automations wedged indefinitely. source-controller hits the same root cause in fluxcd/source-controller#1921.
Fix
Wrap the worktree filesystem handed to the gogit client (
worktreeFSininternal/source/worktree_fs.go):RemoveandRemoveAllresolve intermediate path components securely likeBoundOSdoes, but keep the final component unresolved so they operate on the path itself, never on a symlink target, and treat removal of an already-absent path as a no-op — matchinggit checkoutsemantics (and what go-billy v6 /os.Rootdo). Storage configuration moves frombuildGitConfigtoCheckoutSource, where the working directory is known.The wrapper is entirely controller-side and can be dropped once go-git and go-billy are bumped to v6.
Test plan
TestSourceManager_CheckoutSource_symlinkSwitchBranchreproduces the scenario from the issue end-to-end (push branch behind the checkout ref, per-service config replaced by symlinks to a new_stacksfile). On unpatched code it fails with the exact error from the issue (could not checkout to branch '...': remove .../deploy/_stacks/config.yaml: no such file or directory); with the fix the switch succeeds, the worktree matches the push branch state (symlink targets untouched, no stale symlinks) and is clean, so nothing bogus can be committed.TestWorktreeFS_*cover symlink-preserving removal, no-op removal of absent paths, root protection, and that relative paths cannot escape the worktree root.go test -race ./internal/source/passesgo test -race ./internal/controller/(envtest) — tick after local verification