Skip to content

test: count event-loop turns in the OpenClaw stop test - #459

Merged
AnuradhaKaruppiah merged 1 commit into
NVIDIA:mainfrom
mnajafian-nv:test/openclaw-stop-cancellation-flake
Oct 20, 2026
Merged

AnuradhaKaruppiah merged 1 commit into
NVIDIA:mainfrom
mnajafian-nv:test/openclaw-stop-cancellation-flake

Conversation

@mnajafian-nv

@mnajafian-nv mnajafian-nv commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Overview

The OpenClaw stop-cancellation test could fail with TimeoutError when a process pause exhausted its 0.2 s cancellation deadline. This change replaces that deadline and the 1 s wait for cleanup to start with waits bounded by 100 event-loop turns.

In this test, cleanup uses mocks and in-process tasks; progress does not require an external response or a timer to expire. The test still checks that stop() reaches the blocked cleanup step, completes cleanup after cancellation, and propagates CancelledError before the test releases that step. Production stop() is unchanged.

Details

  • The test covers cancellation during client close, process wait and log-task cleanup.
  • Python 3.14 CI failures occurred on PRs that do not change stop() or this test: the [process] case on Windows in #452 and #443, and the [logs] case on linux-arm64 in #453.
  • After cancellation, local probes observed completion in 1 to 4 event-loop turns on Python 3.14.7 and up to 7 on Python 3.11.14, below the 100-turn limit.
  • While running tests/adapters/ on an M-series Mac with Python 3.14.7, explicitly triggered full garbage collections at this test took 160.7 to 219.1 ms, comparable to the old 0.2 s deadline.

Validation

  • Each case 200 times under CPU load (40 busy processes on 10 cores): before this change, 1 of 600 failed on Python 3.14 with the same TimeoutError and 0 of 600 on 3.11. After it, 0 of 600 failed on both.
  • Each case 200 times with a 250 ms stall inside stop() after the cancellation, Python 3.14: 600 of 600 failed before this change and 0 of 600 after it, and 0 of 600 after it with a second 250 ms stall before the blocked step.
  • Three injected regressions, 10 runs each, fail every case: stop() continuing to wait on the blocked step after the cancellation, never reaching it, and swallowing the cancellation.
  • tests/adapters/test_openclaw.py: 58 passed, 5 skipped. pre-commit run --files tests/adapters/test_openclaw.py passed.
  • Full Python suite not rerun locally.

Where should the reviewer start?

run_event_loop_until in test_openclaw_stop_completes_cleanup_before_propagating_cancellation in tests/adapters/test_openclaw.py.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • Relates to: none

  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.

  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Summary by CodeRabbit

  • Tests
    • Improved cancellation test reliability by polling for cleanup progress across bounded event-loop turns.
    • Added verification that the cancelled stop operation completes and remains cancelled before cleanup is released.

The test gave stop() 0.2 s of wall-clock time to propagate its
cancellation. After the cancellation, stop() awaits only mocks and
in-process tasks and finishes in 1 to 7 event-loop turns, so only a
process pause could fail it: a pause of more than 0.2 s while the test
waited could raise TimeoutError. A full garbage collection at this test
takes 161 to 219 ms on an M-series Mac with Python 3.14 when
tests/adapters/ runs, and the Python 3.14 jobs on Windows and
linux-arm64 have been failing this test on pull requests that do not
change stop() or this test.

Wait for each step by counting event-loop turns (at most 100) instead
of seconds. The test still fails when stop() keeps waiting on the
blocked cleanup step after the cancellation, never reaches it, or
returns without re-raising the cancellation.

Signed-off-by: mnajafian-nv <mnajafian@nvidia.com>
@mnajafian-nv
mnajafian-nv requested a review from a team as a code owner October 20, 2026 13:57
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA/NeMo-Fabric/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Enterprise
  • Run ID: 4d31198c-be9d-415f-b51b-602859c099d6
📥 Commits

Reviewing files that changed from the base of the PR and between 3134226 and 6a6e00b.

📒 Files selected for processing: 1
  • tests/adapters/test_openclaw.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout: 27
  • GitHub Check: Test (Python 3.12, linux-arm64)
  • GitHub Check: Test (Python 3.12, macos-arm64)
  • GitHub Check: Test (Python 3.13, macos-arm64)
  • GitHub Check: Test (Python 3.11, windows-amd64)
  • GitHub Check: Test (Python 3.14, macos-arm64)
  • GitHub Check: Test (Python 3.13, linux-arm64)
  • GitHub Check: Test (Python 3.14, linux-arm64)
  • GitHub Check: Test (Python 3.12, windows-amd64)
  • GitHub Check: Test (Python 3.14, linux-amd64)
  • GitHub Check: Test (Python 3.11, macos-arm64)
  • GitHub Check: Test (Python 3.11, linux-arm64)
  • GitHub Check: Test (Python 3.13, linux-amd64)
  • GitHub Check: Qwen Code E2E
  • GitHub Check: Test (Python 3.12, linux-amd64)
  • GitHub Check: Test (Python 3.11, linux-amd64)
  • GitHub Check: Test (Python 3.14, windows-amd64)
  • GitHub Check: Test (x86_64)
  • GitHub Check: Test (Python 3.13, windows-amd64)
  • GitHub Check: Relay 0.9 (Python 3.12, linux-amd64)
  • GitHub Check: Cline E2E
  • GitHub Check: OpenCode E2E
  • GitHub Check: Pre-commit
  • GitHub Check: Hermes adapter (upstream Relay 0.10, Python 3.14)
  • GitHub Check: Test (arm64)
  • GitHub Check: Test adapters (Node 22.19.0)
  • GitHub Check: Test adapters (Node 24)
  • GitHub Check: Test (Node 20.18.3)
🧰 Additional context used
📓 Path-based instructions: 1
Tests should cover the behavior promised by the changed API surface, including error paths, lifecycle cleanup, and SDK/native parity where relevant.

⚙️ CodeRabbit configuration file

Files:

  • tests/adapters/test_openclaw.py
🔇 Additional comments: 1
tests/adapters/test_openclaw.py: 1

1145-1154: LGTM!

Also applies to: 1158-1161, 1164-1164

Walkthrough

The cleanup-cancellation test now uses bounded event-loop polling to verify cleanup startup and stop-task cancellation before it releases the blocked cleanup operation.

Changes

Cleanup cancellation

Layer / File(s) Summary
Verify cleanup cancellation
tests/adapters/test_openclaw.py
The test polls for cleanup startup and stop-task cancellation without relying on wall-clock timers. It releases the blocked cleanup operation after confirming cancellation.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to 6a6e0

No actionable merge risk is established; the test-only change is suitable to merge with normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title follows Conventional Commits format with the allowed lowercase type test, an imperative summary, 54 characters, and no trailing period. It accurately describes the test change.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The pull request description is complete and follows the repository template. It explains the change, identifies where the reviewer should start, documents validation results and limitations, includes…
Full details: Description check

Explanation

The pull request description is complete and follows the repository template. It explains the change, identifies where the reviewer should start, documents validation results and limitations, includes the required issue section, and confirms both contribution checklist items.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@mnajafian-nv mnajafian-nv self-assigned this Oct 20, 2026
@AnuradhaKaruppiah
AnuradhaKaruppiah merged commit c7c5964 into NVIDIA:main Oct 20, 2026
44 checks passed
@mnajafian-nv
mnajafian-nv deleted the test/openclaw-stop-cancellation-flake branch October 20, 2026 16:33

This branch was previously deployed

1 inactive deployment
fern — 6a6e00bf Deployed Oct 20, 2026 by AnuradhaKaruppiah via Clean up docs preview #2194
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants