perf(runtime): bound scheduler service latency

[Coldwings]

Description

Bound scheduler service delay while a worker remains continuously runnable.
Workers now periodically transfer cross-thread submissions and non-blockingly
poll pending I/O instead of waiting for the runnable workload to become idle.

The contract is deliberately cooperative and expressed in completed task
dispatches from the main worker loop, not elapsed time: user code must return to
the scheduler before a service bound can advance. Inline resumptions delivered
as part of one backend completion batch are not separate worker-loop dispatches.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Performance improvement (optimization that improves speed/memory usage)
• Documentation (changes to documentation, comments, or examples)
• Refactoring (code changes that neither fix bugs nor adds features)
• Tests (adding or modifying tests)
• Build/CI (changes to build system, CI configuration, or dependencies)

Related Issues

Closes #1028

Changes Made

Core Changes

• Reuse the existing owner-local execution counter to open an external-service
  point every 256 completed worker-loop task dispatches without another hot-path
  counter mutation.
• Transfer published MPSC/overflow submissions at each external-service point.
• Give pending I/O one non-blocking backend poll every 16,384 completed
  worker-loop dispatches while the worker remains runnable; CPU-only workloads
  do not make an added I/O syscall.
• Preserve local LIFO ordering, affinity and active-I/O ownership, work
  stealing, shutdown, and resize/drain behavior.
• Add a deterministic regression that pauses a service point, injects a remote
  task and ready eventfd I/O, and proves both complete while the local yield
  backlog is still running on epoll and io_uring.
• Document the cooperative bound and the caller responsibility to suspend.

API Changes (if applicable)

No signature changes. Scheduler behavior now guarantees cooperative service
points under persistent runnable work, including continuously stolen work:

• external submissions: at most 256 completed worker-loop task dispatches;
• pending I/O: at most 16,384 completed worker-loop task dispatches.

These are not wall-clock deadlines. A non-suspending coroutine can still occupy
its worker indefinitely, and one backend poll may deliver a completion batch
inline before control returns to the worker loop.

Testing

Unit Tests

• Added new tests for the changes
• Updated existing tests if needed
• All tests pass locally

Integration Tests

• Tested with existing examples
• Tested in real-world scenarios (if applicable)

Sanitizer Testing

• Tested with ASAN (AddressSanitizer)
• Tested with TSAN (ThreadSanitizer)
• No new warnings or errors

Test Results

Strict GCC 12.2 Debug build with developer warnings and warnings-as-errors:
  elio_tests, elio_tests_asan, elio_tests_tsan,
  scheduler_service_benchmark, quick_benchmark: PASS

Normal full suite:
  792 test cases, 12097 assertions: PASS

ASAN scheduler suite:
  78 test cases, 597 assertions: PASS

TSAN scheduler suite:
  78 test cases, 597 assertions: PASS (no race reports)

Service regression, explicit epoll and io_uring:
  normal / ASAN / TSAN: PASS
  100 consecutive normal runs: PASS

Three interleaved, CPU-pinned Release runs of
scheduler_service_benchmark --trials 40 --backlog-ms 20 measured:

Metric	main mean	This branch mean	Change
I/O p99	20,079.57 us	200.23 us	99.0% lower (100.3x)
Remote inbox p99	20,077.49 us	10.64 us	99.95% lower (1,887.5x)
Total backlog yields	54,681,143	54,946,051	0.5% higher

Two additional interleaved, CPU-pinned quick_benchmark runs measured mean
yield at 21.35 -> 20.31 ns, scheduler reschedule at 21.18 -> 21.64 ns (+2.2%),
and external submission at 50.15 -> 50.81 ns (+1.3%). These local timings are
evidence for selecting the 256 / 16,384 quanta, not CI thresholds.

Checklist

Code Quality

• My code follows the project's code style
• I have added/updated comments for complex logic
• I have removed any debug code, TODOs, or commented-out code
• My changes generate no new warnings

Documentation

• I have updated documentation (wiki, README, code comments)
• I have added examples for new features (if applicable)
• I have updated API documentation (if applicable)

Testing

• I have added tests that prove my fix is effective or my feature works
• New and existing unit tests pass locally with my changes
• I have tested with ASAN and TSAN

Compatibility

• My changes are backward compatible (or I've documented breaking changes)
• I have considered the impact on existing users
• I have updated CHANGELOG.md (if applicable)

Performance (if applicable)

• I have considered the performance impact
• I have added benchmarks for performance-critical changes

Screenshots / Diagrams

Not applicable.

Additional Notes

The separate quanta are intentional. External queue inspection is cheap enough
for a tight bound; a backend poll has measurable cost, so its wider bound keeps
pending-I/O throughput close to the no-service baseline while still eliminating
unbounded starvation.

Reviewer Guidance

Areas requiring special attention:

• Whether using completed-resume count as the cooperative bound is documented
  clearly enough and does not imply a wall-clock guarantee.
• Direct completion delivery from poll(0) followed by external-inbox transfer
  into the owner-LIFO deque.
• Resize/drain and active-I/O ownership remaining unchanged.

Questions for reviewers:

• Is there a continuously runnable path that bypasses run_task() and therefore
  bypasses these service points?
• Should these fixed internal quanta remain implementation policy, or is there a
  concrete use case that justifies public configuration now?

---

Thank you for contributing to Elio! 🎉

[Copilot]

Pull request overview

Adds cooperative scheduler service points to prevent persistent local work from starving remote submissions and pending I/O.

Changes:

• Services external work every 256 scheduler dispatches and I/O every 16,384.
• Adds epoll/io_uring regression coverage.
• Documents service guarantees and performance trade-offs.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
include/elio/runtime/worker_thread.hpp	Defines service quanta and state.
include/elio/runtime/scheduler.hpp	Implements periodic service points.
tests/unit/test_scheduler.cpp	Tests service under local backlog.
README.md	Advertises bounded cooperative service.
CHANGELOG.md	Records the behavioral change.
wiki/API-Contracts.md	Updates the scheduler contract.
wiki/API-Reference.md	Documents service semantics.
wiki/Performance-Tuning.md	Explains bounds and performance trade-offs.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Coldwings]

@copilot Please review the current head only. Please do not modify code or push commits.

[Copilot]

@copilot Please review the current head only. Please do not modify code or push commits.

Reviewed the current head.

The documentation fix is consistent and accurate across all four surfaces — the inline scheduler comments, API-Contracts.md, API-Reference.md, and Performance-Tuning.md all now say "completed worker-loop task dispatches" rather than "resumptions", and every location explicitly notes that coroutines resumed inline during a backend poll(0) completion batch do not consume the dispatch budget.

The implementation at scheduler.hpp:1908–1910 correctly gates on tasks_executed_local_, which is incremented only inside run_task() in the main worker loop, so the stated contract and the code match. No issues found with the current head.