Skip to content

Flaky test: test_hung_scan_tools_are_unavailable fails under load #136

Open

Description

@lukemaj

What: tests/test_health.py test_hung_scan_tools_are_unavailable passes reliably regardless of machine load.
Why: It relies on wall-clock timing (a /bin/sleep 0.3 stub against a patched SCAN_TIMEOUT_SECONDS = 0.05) and failed 5 of 5 runs after the full suite under load, on both main 1fbbb0f and PR #134's head.
So what: Small test fix; whoever picks it up.

Evidence

  • Muse Spark 1.3 review of PR fix: treat pending gh pr checks as pending (#132) #134 (2026-10-19, run on cavallo): full suite 860 tests, 1 failure in this test on head and on base; it passed once on base by timing luck. CI proof passed, so it is load-dependent.
  • tests/test_health.py around lines 115-116.

Elon record

  • Requirements and who asked: reliable proof runs for every PR (operations verification); found during the 2026-10-19 review follow-up Luke requested.
  • Deleted: changing health.py behavior; only the test's dependence on timing is in scope.
  • Bottleneck: one timing-based assertion.
  • Checked myself: the test source lines above; the reviewer's reported reruns (not reproduced by me).

Acceptance criteria

  • The test proves the hung-tool path without depending on scheduler timing (for example a stub that never exits, or an injected clock).
  • It passes repeatedly under parallel load.

Non-goals

  • Changing health scan semantics.

Blockers

None.

Required proof

  • 20 consecutive full-suite runs, or the test in a loop under load, all passing; independent review.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions