Skip to content

A driver adoption test fails on CI under load, and its 30-second wait is why #107

Closed

Description

@Simon-AI-coding

What happened

test_driver.AdoptionTests.test_a_child_that_cannot_be_reached_is_left_to_the_rule_that_settles_it failed on the Python 3.14 job of PR #106:

AssertionError: False is not true : 01 never settled failed; the log says None
  skills/crew/assets/driver/tests/test_driver.py:2438  self.wait_for_verdict("01", "failed")
  skills/crew/assets/driver/tests/test_driver.py:557   in wait_for_verdict

The other three jobs (3.11, 3.12, 3.13) passed on the same commit.

It is timing, not the change under test

Two pieces of evidence, both against the same commit:

  • The full suite passed locally on Python 3.14: Ran 869 tests ... OK.
  • Re-running the 3.14 job with no code change turned it green: Python 3.14 pass 8m35s.

PR #106 touched three things — one line of scripts/sync-bridge.sh, one comment line in the vendored bridge, and tests/test_sync_bridge.py. The driver's adoption tests import none of them.

Why it can fail

wait_for polls a condition every 0.2s against a 30-second deadline, and returns False when the deadline passes:

def wait_for(self, condition, timeout=30.0):
    deadline = time.monotonic() + timeout
    while time.monotonic() < deadline:
        if condition():
            return True
        time.sleep(0.2)
    return False

The condition here is that a real driver subprocess, adopted by a second coordinator, notices a killed tmux window and settles ticket 01 failed in the run's log. On a loaded runner — the suite takes ~8 minutes there and every driver test spawns real processes — 30 seconds of wall clock is not a reliable bound on that. Nothing pins it: the test asserts on elapsed time rather than on an observable the driver produces.

Why it is worth fixing rather than re-running

CI on main was red for four consecutive releases (v0.7.0 through v0.8.2) at the vendoring check, and it went unnoticed partly because a red tick beside three green jobs reads as noise. Every intermittently-failing test makes the next real failure cheaper to ignore.

What a fix would need to decide

Whether the bound should be a longer deadline (cheap, still arbitrary), or whether these tests should wait on something the driver actually signals rather than on a clock. That is a design call for whoever picks this up — hence needs-triage rather than ready-for-agent.

Where to look

Routing

Workflow: tdd
Executor: codex
Model: gpt-5.6-luna
Effort: max
Account: b
Review: claude claude-opus-5 medium
Reasons: non-core-routine tdd (test infrastructure only, single module, approach fully specified); account b named by the maintainer at the approval checkpoint.

Activity

  1. Simon-AI-coding commented on Aug 19, 2026

    @Simon-AI-coding
    OwnerAuthor

    This was generated by AI during triage.

    Agent Brief

    Category: bug
    Summary: The driver test suite's wait deadline scales for loaded environments instead of hard-coding 30 seconds of wall clock.

    Current behavior:
    The driver tests' wait_for helper polls a log-observable condition every 0.2s against a fixed 30-second deadline. The condition itself is already the right thing to wait on (the driver's own settlement appearing in the run's log); only the bound is wrong. On a CI runner where the suite takes ~8 minutes and every driver test spawns real subprocesses, 30 seconds of wall clock is not a reliable bound — the Python 3.14 job of PR #106 failed on it and went green on a no-change re-run. Every intermittent red makes the next real failure cheaper to ignore (CI was red for four releases at the vendoring check partly for this reason).

    Desired behavior:
    The deadline is configurable through one environment variable read by the test helper (a multiplier or an absolute default override), with the local default unchanged at 30 seconds. The CI workflow sets it generously for its jobs — generous is correct here because the deadline only spends its full length on failure; passing tests are unaffected. No per-test timeout arithmetic, no new waiting mechanism.

    Key interfaces:

    • The wait_for helper in the driver test suite — its default deadline becomes overridable by the environment variable; every call site that passes no explicit timeout inherits it.
    • The CI workflow's test-run steps — set the variable for the driver test jobs.

    Acceptance criteria:

    • With the variable unset, wait_for's behavior and default are unchanged.
    • With the variable set, the effective deadline reflects it (a unit test asserts this without actually waiting).
    • The CI workflow sets the variable so the driver jobs run with a bound that holds under the measured ~8-minute-suite load.
    • Explicit timeout= arguments at individual call sites still win where they are passed.

    Out of scope:

    • Rewriting the driver tests to a different waiting mechanism — they already wait on a log observable.
    • Marking or retrying flaky tests; the point is removing the flake, not tolerating it.
  2. added
    ready-for-agentFully specified, ready for an AFK agent
    and removed
    needs-triageMaintainer needs to evaluate this issue
    on Aug 19, 2026
  3. Simon-AI-coding commented on Aug 19, 2026

    @Simon-AI-coding
    OwnerAuthor

    /crew crewtask/65

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions