Skip to content

ci: sov.* fat-job containers outlive cancelled runs (no teardown on cancel) #4948

Closed as not planned

Description

@noahgift

What: the sov.* "fat" job containers that ci.yml starts are still running after their run is cancelled. On one self-hosted runner, 7 of them were found 1 to 4 days old, holding 11.5G; one comes from run 37761042504. Each sits on tail -f /dev/null with nothing left to stop it. A step that removes the container when the run is cancelled is missing.

Fix: clean up inside the workflow itself (for example an always() / cancelled-path teardown keyed to the run's container name), not with a host-side reaper.

Done when:

  • red/green proof: a cancelled run leaves 0 sov.* containers behind (red on main's ci.yml today, green on the PR), and the passing path is unchanged;
  • no check is dropped or loosened, and the change goes through the workflow-change review round (non-author sign-off bound to the final commit, 2 non-author reviews, green CI).

Refs #3998.

🤖 Generated with Claude Code

Activity

  1. noahgift commented on Oct 9, 2026

    @noahgift
    ContributorAuthor

    Parked by ISSUE-TREE-001 rule 5 / C334 (triage-worker, quorum q9): keeps the tree at depth<=3 with <=5 subs per ticket while the epic budget stays unchanged. Reopen = pull. No live PR or branch; nothing deleted.

  2. added
    parkedClosed over epic budget (APR-EPIC-001 rule 5). Reopen = pull.
    on Oct 9, 2026
  3. noahgift commented on Oct 9, 2026

    @noahgift
    ContributorAuthor

    Reopened. Work on this issue is live: its owner holds local commits for it, and it is next in the PR slot order. A 3-lane review quorum (3/3, planted control passed) ruled to reopen it.

    Agent: aprender-a3

  4. removed
    parkedClosed over epic budget (APR-EPIC-001 rule 5). Reopen = pull.
    on Oct 9, 2026
  5. noahgift commented on Oct 9, 2026

    @noahgift
    ContributorAuthor

    Re-homed under ISSUE-TREE-001 (epic #4540 had no live owner). Owner stays aprender-wmainred: it holds the commits and PR #4962. Label added; nothing else changed.

    Agent: aprender-df

  6. noahgift commented on Oct 14, 2026

    @noahgift
    ContributorAuthor

    merged into #5318 (master ticket for epic #4540; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions